Banks ‘sleepwalking’ into AI governance failure, insiders warn

TL;DR:

• Governance and engineering specialists warn banks are pushing agentic AI into KYC, AML, underwriting and customer servicing faster than oversight can keep up.
• The core worry has shifted from bias and hallucinations to accountability: who is liable when a semi-autonomous system makes a regulated decision.
• Critics argue prompt-based safeguards are “intent, not enforcement”, and that firms need runtime telemetry able to reconstruct an AI decision months later.

A growing chorus of governance professionals say financial services is “sleepwalking into a governance failure that makes previous scandals look minor”, as banks embed generative and agentic AI into the workflows regulators watch most closely. The warning, aired in a widely shared exchange among AI-governance leaders, reframes the risk debate away from model accuracy and toward who carries the liability when autonomous systems decide.

Prompt-based guardrails are not controls

The flashpoint was the US Federal Reserve’s replacement of its SR 11-7 model-risk framework — the rulebook that governed model risk for fifteen years — with SR 26-2 in April. Alexandra Car of Breeple.ai argued the successor “ignores the very tech being deployed into KYC, AML, and credit underwriting at record speed”, leaving firms to assume existing guardrails suffice. Several specialists pushed back on that assumption. “Probabilistic compliance is not compliance,” said Marcos Oliveira of Block 64; “prompt-based guardrails are intent, not enforcement.” The proposed fix is runtime telemetry and code-level controls capable of reconstructing an agent’s decision chain after the fact — because, as Oliveira put it, “you cannot audit what you cannot reconstruct.”

Looking forward

For UK institutions the timing matters: firms are racing toward EU AI Act compliance deadlines while leaning heavily on vendor assurances, even though they retain regulatory responsibility for third-party systems’ behaviour. The warning sits awkwardly against the FCA’s decision to lean on existing frameworks rather than write new AI rules, and compounds separate alarms over AI-generated code outrunning bank testing. The question regulators are starting to ask, specialists say, is no longer “do you have AI policies?” but “can you explain how this outcome was formed, challenged and defended under pressure?” Banks that build governance infrastructure at the same pace as deployment — not after it — will be the ones left standing.