AI Weekly - Banks Shut the Door on Agents

00:00 Right now in the UK, your bank does not trust an artificial intelligence enough to let it move £10 between your checking and your savings accounts. Yeah, they absolutely do not. They have completely slammed the vault doors shut on AI agents. They are just refusing to authorise them at all. Right. Yet, at this exact same moment, the UK government is perfectly happy to let a completely unmonitored AI advise desperate job seekers on their career paths. Which is a terrifying contrast. It really is.

00:30 I mean, we have fluffy plush toys freely talking to toddlers about self harm. We have police forces quietly rolling out AI that basically flips the entire presumption of innocence for every single driver on the road. Yeah. So welcome to the deep dive. Today, we are looking at this massive, terrifying hypocrisy of authorisation in the AI era. It is honestly the single most important fault line in the global economy right now. Right. Because we are watching this transition from AI that just answers your questions to AI that actually takes action on your behalf.

01:06 And the second a machine takes an action. The whole game changes. Exactly. The entire liability model of society has to be rewritten from scratch. And that is what we are digging into for you today. Because if you are trying to cut through the absolute avalanche of daily AI news to find the structural shifts that actually matter for your business, your career, your family, this is it. Yeah. This is the core issue. Last week, our sources were full of surveys showing finance firms happily running AI agents internally. They love them for their own back office efficiency.

01:36 Oh, sure. Behind closed doors. Right. But this week, the sources reveal a totally jarring shift when it comes to the outside world. So we're asking who actually gets to authorise an AI to act? Right. We're going to follow this thread from the banks blocking the bots to the physical data centres facing major voter pushback, the cyber risks of hyper-fast software development, the hidden economic hits to your job, and finally, how the courts, the state, and vulnerable populations are just bearing the brunt of this uncontrolled integration.

02:06 Because the central conflict here, it comes down to a really simple question, right? Who gets to say yes to an AI? And who is left holding the bag when that AI makes a catastrophic mistake? And right now, nobody agrees on the answer. Nobody. Let's start right at the gateway to your money. Because the UK's biggest banks, HSBC, Barclays, Lloyds, NatWest, Nationwide, they are all preemptively blocking third-party AI agents from accessing customer accounts. Yeah. A complete blockade.

02:37 Now, my initial thought reading this was that they are just acting like bouncers at an exclusive club. They're checking IDs at the door, but the ID they are looking for isn't a person's face. It's a bulletproof liability model. That's a great way to put it. But is this really just about keeping customer money safe from a buggy bot? I mean, really? Well, safety is the public justification. And, you know, it's a valid one. JPMorgan's Jamie Dimon explicitly used the phrase rogue agents. Rogue agents, yeah. The banks are looking at this massive legal vacuum.

03:09 Imagine you authorise a personal finance bot to optimise your savings. Okay, sure. That bot hallucinates. It misinterpret some obscure tax rule and moves £30,000 into an account that triggers a massive irreversible tax penalty. Who pays for that? I mean, I would definitely blame the bank for letting it happen. Exactly. You blame the bank. But the bank will say, well, you authorise the third-party agent. Then the AI developer will point to some buried claws in their terms of service saying, you know, the tool shouldn't be used for definitive financial action.

03:41 So the liability just falls into a black hole. Right. So blocking them entirely is just the safest legal play. But there is this McKinsey warning in our sources that suggests a completely different motive, because this isn't just about protecting the bank from a bot that makes a mistake. It's about protecting the bank from a bot that is, well, incredibly competent. Yes. Yes. This is the real fear. If you look at the underlying mechanics of retail banking, they rely so heavily on consumer friction. Friction, right?

04:11 Like us just being lazy. Basically, yeah. They rely on your inertia. You might leave your money in a savings account earning 1% interest because researching the market, opening a new account to pay 4%, and transferring the funds, it takes time. Right. It takes effort and mental energy that you just don't have on a Tuesday evening after work. Precisely. The banks literally bank on you being too busy or tired to move your money. But an AI agent does not get tired. No, it doesn't. An AI agent has zero friction.

04:41 If you authorise an agent to manage your money, it acts like a high-frequency trader for your retail savings. Wow. It can constantly scan the entire market and automatically port your deposits to a rival bank in milliseconds just to capture an extra half percent of yield. And the numbers on this are just staggering. McKinsey warns that savers could move up to £700 billion out of low-paying accounts via this kind of AI advice. £700 billion. It's massive. The data suggests that shifting just 5% to 10% of those balances could cut a bank's deposit profits by a fifth or more.

05:15 A fifth of their profits just gone. It is an existential threat to their net interest margin. So when you ask if banks are blocking bots for safety, well, the answer is yes, but they are also trying to outlaw frictionless capital flight. They are terrified of an AI moving deposits to a rival in milliseconds. Absolutely terrified. But while the banks are successfully pulling up the drawbridge, the retail sector is facing a completely different reality. Meta is launching its Muse shopping agent in the UK within weeks.

05:46 And this isn't a pilot programme, right? No, not at all. Muse already has 11 million users globally. It doesn't just give you fashion advice. It can actively shop, check out, and pay bills for you. Which forces this brutal dilemma on retailers. Look at Amazon. Amazon actually proactively blocked Meta's Muse back in September, citing security concerns. Right. By blocking the agent, Amazon protects its liability. It protects its data architecture. And crucially, it forces the customer to interact directly with the Amazon interface. But if you are a smaller retailer or even a massive high street brand, can you actually afford to do that?

06:22 That's a million dollar question. If you block these agents, sure, you protect your liability. But if millions of consumers start using Muse as their default way of buying things and your store doesn't allow Muse in, you effectively cease to exist in that consumer's commercial universe. And that is the exact figure driving adoption. It's why brands like Sephora and Walmart are partnering with Meta in the US. Right. But partnering comes with a profound cost. If a customer says to their Meta AI, buy me a good moisturiser. And the AI chooses Sephora, well, Sephora gets the sale.

06:53 But they lose everything else. Exactly. Sephora has lost total control of the customer journey. They don't get the opportunity to upsell. They don't build brand loyalty. They are reduced to just being a silent fulfillment warehouse at the end of an AI's decision tree. It fundamentally changes who owns the customer. Completely. And the regulators are clearly watching this battle over reliability. The CMA, FCA, ICO, and Ofcom. That's four major UK regulators. They just released a joint statement on this. A very coordinated move. And what really struck me is that it was written in incredibly plain English.

07:27 They didn't announce any new complex AI laws. They essentially just pointed at the existing rulebook. Which is a very deliberate, aggressive posture from them. They are explicitly stating that existing legal and regulatory frameworks absolutely apply to personal AI agents. So no loopholes. Right. If you are a developer or a retailer deploying an agent, you cannot use the autonomy of the AI as a shield. You can't just blame the algorithm. Exactly. The regulators are putting the market on notice. If your agent reaches consumer protection laws or misuses data, the deployer is on the hook.

08:00 There is no, the algorithm did it defence. And we are seeing that exact philosophy codified into law right now. The UK Cybersecurity and Resilience Bill just added a specific clause that lets government ministers set rules for when firms quote unquote lose control of an AI agent. That wording is so important. They are officially treating an AI going rogue as a critical infrastructure resilience failure. It's a totally necessary reframing of the risk. Because historically, we view cyber resilience as protecting against outside attacks, right?

08:33 Like someone hacking into your server. Right. The external threat. But this clause recognises that the threat can be internal and structural. Because agentic AI is designed to string together multiple actions autonomously to achieve a goal. It can take highly erratic destructive paths through a regulated network. It could just do something nobody predicted. Exactly. So treating that loss of control as akin to a server outage. Well, it means organisations have to build mandatory logging and hard manual kill switches into their architecture before they ever turn the agent on.

09:11 So we have this massive digital land grab happening. Banks blocking agents, retailers surrendering to them, regulators demanding kill switches. It's chaotic. It really is. But every single one of these agents, whether they are welcomed or blocked, they all require an astronomical amount of computing power to function. They do. That means building data centres. Huge ones. Which brings us to the physical reality of AI. We are seeing digital ambition colliding violently with local physical limitations.

09:42 And that collision is generating real measurable economic shockwaves. The data from the recent US midterm elections actually provides a perfect case study for this. Right. The midterms. Regardless of the political leanings involved, the underlying economic mechanism was fascinating. We saw voters, aggressively and successfully, push back against the construction of new data centres in their local counties. Yeah, Democratic candidates campaigned heavily on the sheer strain these facilities place on local resources, specifically, you know, the power grid and the water supply.

10:14 And the voters overwhelmingly agreed with them. They basically said, we don't care about the AI revolution if it means our town's water table drops and our electricity bills spike. And this local pushback actually triggered a massive global market slide. Because it broke the fundamental assumption of the entire AI boom. How so? Well, Wall Street has been pricing tech stocks on the premise of unconstrained infinite infrastructure growth. When local voters demonstrated that there is severe physical friction, that you literally cannot plug these servers in because the community will block the zoning permits, investors panicked about the ROI.

10:50 And the panic wiped out $2.5 trillion in market value. In a flash. The Nasdaq fell 5.5%, Nvidia dropped 9%, and Asian markets tumbled right alongside them. It is wild to think that local zoning board disputes in the American Midwest can instantly wipe trillions off the global stock market. It just proves that the bottleneck for AI is no longer software engineering. It is concrete, it's copper wire, and it's local politics. But over here in the UK, the government is promising a completely different reality.

11:20 Growth Minister Dan Tomlinson is insisting the UK will absolutely not slow down. He's very adamant about that. Yeah, he's calling UK investment too low and leaning heavily on this new data centre covenant. Which is basically a scheme to win local favour by promising community investment in exchange for planning permission. Right, offering a carrot to the locals. Exactly. The government is promising this highway for AI development. But when you look at the sources detailing what developers are actually facing on the ground, they are having to buy their own gas generators just to turn the lights on.

11:53 Yeah, the friction in the UK is arguably worse than in the US, primarily because of the national grid queue. Tell us about the Data Vita site, because that blew my mind. Okay, take the Data Vita site at Chapel Hall in Lanarkshire. This was originally pitched as a massive green AI growth zone. The promise was over one gigawatt of private wire renewable energy. Which means what exactly? That means they were supposedly going to build dedicated wind or solar assets wired directly into the data centre itself.

12:23 A completely self-sustaining green facility. Exactly. But reality has set in. Data Vita now admits the site will be fully grid-served via a power purchase agreement. Okay, so for the listener who isn't an energy trader, explain why a power purchase agreement of PPA isn't the same as that original green promise. Right, so a power purchase agreement is essentially a financial accounting mechanism. You pay a renewable energy generator somewhere else in the country for the electricity they produce, which theoretically offsets the power you pull from your local grid.

12:58 Okay. But electrons still care about financial contracts. Right. If your massive data centre is pulling power from the local grid in Lanarkshire, it is using whatever mix of gas, nuclear, and wind is currently running through those specific local cables. It places the exact same physical strain on the local transmission infrastructure as any other massive industrial site. And that strain is terrifying environmental groups. Friends of the Earth Scotland ran the numbers on this. And the numbers are grim. They warn that if all 23 proposed Scottish AI data centres are actually built, which totals about 9,100 megawatts of demand, they would use 51% of Scotland's entire carbon budget all the way out to 2045.

13:41 14 million tons of CO2 equivalent a year from 2030, just for one sector. It's an insane proportion of a national resource. And because the physical grid simply cannot handle that demand right now, we are seeing developers resort to desperate measures. Very desperate. BBC File on 4 investigated this and found four major UK sites, Acre Slough, Wapsey's Wood, Cardiff, and the massive one-gigawatt Quest Park that are actively weighing the use of gas-fired power plants as a stopgap. This completely shatters the green futuristic narrative of the tech industry.

14:15 I mean, they are building the most advanced technology in human history, and they are planning to power it by burning fossil gas on site. It's stepping backwards. It is. And you have to understand the mechanism behind this desperation. It takes years to build a data centre, but it can take up to 10 years to get a high voltage connection to the national grid because you have to build new substations and lay miles of heavy transmission lines. So a 10-year stopgap of burning gas isn't a temporary measure. In infrastructure terms, 10 years is semi-permanent. Precisely.

14:45 The AI models demand training compute today. The developers just cannot wait until 2035 for the grid to catch up. Which perfectly explains the strategy of companies like Drax and Brookfield. They are planning a data centre of over one gigawatt right at the Drax power station in Yorkshire. Yes, a massive project. We are talking about an investment of over £20 billion. They basically looked at the 10-year grid queue and decided to just build the servers directly inside the power plant. It is the ultimate real estate arbitrage in the AI era.

15:18 Right now, a high-capacity grid connection is infinitely more valuable than the land itself. Right. Drax has the land, they generate the power, and crucially, they already have a four-gigawatt connection to the grid established. By co-locating the data centre at the power station, they completely bypass the 10-year waiting list. But the optics on that are brutal. Analysts regularly rank that Drax biomass plant as the UK's largest single CO2 emitter. It's a PR nightmare. So you are taking the most power-hungry technology on Earth and plugging it directly into the biggest carbon emitter?

15:50 If US investors were spooked by local backlash over resource strain, won't a UK march against data centres like the major protests currently planned in Edinburgh spook them here too? It absolutely will. Capital is agnostic, you know? It only cares about friction. If a developer in the UK faces protests, 10-year grid queue and carbon budget restrictions, that capital will simply flow to a jurisdiction with your rules. So the physical infrastructure is literally grinding to a halt in planning committees and grid queues.

16:20 But here is the terrifying paradox. While the servers are waiting for power, the AI models running inside those servers are not waiting for anything. Not at all. They are accelerating at a pace that is starting to look entirely unconstrained. The disparity between the physical bottlenecks and the software acceleration is alarming. And we have concrete data on this from an evaluation group called METR. Alright, METR. They spent 11 weeks embedded deeply inside Anthropic, which is widely considered one of the most cautious, safety-focused frontier AI labs out there.

16:55 And METR's conclusion was that Anthropic's model capability improvement has effectively doubled in speed. They are progressing at 1.9 times their previous rate. Exactly. How is that physically possible if the hardware hasn't doubled? It's possible because of a compounding feedback loop. We have crossed a threshold where AI is now actively helping to build AI. Like writing its own code? Yes. The researchers are using their current generation of large-language models to write the complex code, optimise the training architectures, and structure the data for the next generation.

17:30 The human engineers are being augmented by the very tools they created, which exponentially decreases the time between breakthroughs. But the detail in the sources that absolutely infuriated me was about the transparency here. Oh, the redactions. Yes, we have this independent evaluator, METR, embedded inside a lab. They conclude the speed has doubled. But when METR published their public report, Anthropic redacted the vast majority of the most informative evidence. They cited competitive secrecy. Of course they did.

18:01 This is the paradox of our current regulatory environment. We rely on these voluntary evaluation agreements between the labs and the safety institutes. But when the rubber meets the road, when an evaluator finds data that truly explains how fast the frontier is moving, the corporate instinct to protect intellectual property completely overrides the public's right to know. So they redact the proof. The public is essentially told, trust us. The intelligence is doubling. But we can't show you the math because our competitors might see it. It just demonstrates the profound weakness of relying on voluntary corporate transparency for a technology of this magnitude.

18:35 And we don't even have to wait for the next generation to see how dangerous this capability jump is. OpenAI just launched a model called GPT-6 Cyber. Yeah, that's a concerning one. Under OpenAI's own internal preparedness framework, they literally rated this model as critical for cyber risk. And they revealed that an unreleased version of their models built 948 root exploits for the Linux kernel. That number is staggering. Let's break that down because it sounds highly technical, but the implications are terrifying.

19:05 What exactly is a root exploit in Linux? Okay, so the Linux kernel is essentially the core engine of the operating systems that run the vast majority of the world web servers, cloud infrastructure, and enterprise networks. It's the foundational layer. Right, the bedrock. A root exploit, specifically a local privilege escalation, is when a hacker finds a microscopic flaw in that core engine. They take a normal low-level user account, the kind of access a basic employee or a standard application might have, and they trick this system into granting them root or absolute administrator control.

19:38 It's the digital equivalent of a guest finding a hidden flaw in a hotel's electronic key card system that allows them to instantly open every single vault and room in the building. Exactly. That's a great analogy. Historically, finding a novel privilege escalation in Linux required weeks or months of highly specialised incredibly expensive human labour. Yet elite cybersecurity researchers reverse engineering code line by line. But the AI just did it 948 times.

20:08 And we have to look at the economics of how it did it. OpenAI stated it cost them $1.21 million in API compute spend to generate those 948 exploits. So you divide that out, and it costs them about $1,279 per exploit. That completely upends the economics of cyber warfare. It's so cheap. It is. If a state actor or a criminal syndicate can generate novel highly destructive root exploits for just over $1,000 in compute costs, traditional defence paradigms fail.

20:40 You can no longer rely on the assumption that attacking your network is too expensive or time-consuming to be worth the effort. It's unbelievable. OpenAI is simultaneously selling the lock and the lockpick. They build defensive tools, but they are proving that the cost of offensive cyber operations is absolutely plummeting. And the UK's National Cyber Security Centre, the NCSC, is acutely aware of this shift. They just issued a stark, formal warning that AI-powered ransomware targeting UK critical infrastructure is highly likely by 2028.

21:14 Just a few years away. And they took the unusual step of explicitly naming China as a primary vector for this threat. Which triggered a furious diplomatic incident. China's foreign ministry angrily rejected the report, calling it a wanton speculative accusation. They accuse the UK of politicising cyber security just to curb China's own AI development. You really have to separate the geopolitical posturing from the technical reality here. The NCSC is not necessarily saying that AI will invent completely new science fiction ways to hack.

21:46 What are they saying then? They are saying that AI will radically scale up the exploitation of existing vulnerabilities. It allows threat actors to automate the discovery and weaponisation of flaws across thousands of targets simultaneously. And the speed and scale is exactly why the godfather of AI, Geoffrey Hinton, is practically shouting from the rooftops right now. He warned in the sources that he expects an AI Chornobyl within a year or two. That's a heavy word to use. It is. He's defining that as a major power grid failure or a massive financial system outage caused by AI.

22:20 And his solution is demanding an FDA style approval process before any of these frontier models are allowed to launch. Hinton is employing a very specific historical analogy there. He is pointing to the thalidomide tragedy in the pharmaceutical industry, where a drug was widely distributed before its catastrophic side effects on fetal development were understood. The FDA model was built to ensure that you must proactively prove a drug is safe in rigorous clinical trials before you sell it. Hinton argues we are currently treating AI like pre-FDA medicine.

22:53 Can we just put it out there and see what happens? Exactly. The labs release the model to the public, wait to see how it breaks or what it breaks, and then try to patch it. He argues that release and patch is an unacceptable framework when the software has the capability to bring down a national power grid. But here is the thing. We don't even need to wait for Hinton's hypothetical Chornobyl. The damage isn't just theoretical infrastructure collapse. It is happening right now in mundane daily commerce. Yeah, look at the insurance sector. Exactly. Look at Zurich UK, the insurance giant.

23:25 They caught £109.3 million in fraudulent insurance claims just this year. And the sources detail how AI is fuelling this massive daily arms race. It's fraud at an industrial scale. The fraudsters are using AI to generate fake evidence, doctored photos of staged accidents, perfectly forged medical documents. But on the flip side, Zurich is using their own AI analytics system, Quantexa, which caught about 20% of those fraudulent cases. It is quite literally machine versus machine over £109 million battlefield.

23:59 Right. And this is the reality of AI risk that affects you today. It's a silent scaling tax on the economy. Every time an AI makes fraud cheaper and easier to commit, the cost of insurance for everyone else goes up. The defence relies on buying more expensive AI to counter the cheap offensive AI. And this rapid unconstrained deployment isn't just breaking code or faking insurance claims. It is actively, quietly rewriting the UK job market. This is where it gets very personal for people. It does.

24:29 Which brings us to the human cost of all this hyper efficiency. Because the narrative from the government has consistently been relatively rosy, right? They assure us that AI is an engine for growth. They have to say that. The Bank of England even put out a report recently stating that AI is not causing mass layoffs. But when you look at the actual data beneath that headline, it is incredibly grim. AI is causing a silent hiring freeze. This is a vital distinction for anyone navigating the labour market right now. The disruption of AI doesn't necessarily look like a dramatic Hollywood scenario where half the office is fired on a Friday afternoon.

25:05 Right. It's not a sudden shock. According to the Bank of England's regional agents, it looks like a closed door. Firms are explicitly reporting that when an employee leaves, they are simply not replacing them. They are choosing to operate with a smaller headcount because they anticipate that AI will absorb the excess workload over the next few years. If you are running a small business right now or, you know, if you have a child just finishing university, this BOE silent hiring freeze is the data point you need to obsess over. Absolutely. Because it means the bottom rungs of the career ladder are being dismantled.

25:38 The data from Adzuna and indeed confirms this in terrifying detail. UK entry-level job ads that require AI skills have nearly tripled. But overall junior vacancies, the absolute number of jobs available to young people have fallen by 25%. We are fundamentally altering the apprenticeship model of white-collar work. What do you mean by that? Well, historically, entry-level meant doing the boring, repetitive groundwork. You drafted the basic contracts, you cleaned the spreadsheets, you wrote the boilerplate code.

26:11 That was how you learned the trade and understood the architecture of the business. But if an AI agent can do that boilerplate work in three seconds, the firm doesn't need to hire the junior. Precisely. One employer cited in our sources explicitly stated that junior roles now merely involve handling the last 20% of the task, essentially just reviewing the output generated by an AI. Which sounds easier, but... But the long-term structural risk here is profound. How does a junior employee ever develop the deep expertise required to become a senior partner if they never had to do the foundational work themselves?

26:46 We are locking an entire generation out of the workforce. And the justification for tolerating this disruption is always based on these massive economic promises from the government. The big numbers, yeah. We have all heard the heavily publicised claim that AI will add £400 billion to the UK economy. It's been the cornerstone of national action plans and digital skills boosts. But AI minister, Kanishka Narayan, just had to admit in parliament that the £400 billion figure is practically fiction. It was a shocking admission.

27:17 He confirmed that the £400 billion projection actually came from a 2023 report that was commissioned by Google. Commissioned by Google. Yes. And when pressed, the government revealed that their own internal central estimate is vastly lower, around $120 billion. The government's own numbers are off by a quarter of a trillion pounds. This is the immense danger of what we might call vendor supplied economics. We are building national infrastructure policy, educational strategies, and corporate business plans based on GDP forecasts authored by the very same companies that are trying to sell us the technology.

27:53 It's like asking a car salesman to calculate how much money you'll save by buying a Ferrari. Of course the math looks amazing. Exactly. And it's not just Google. The British Chambers of Commerce recently published a report claiming AI could lift UK GDP by 10% by 2031. But if you dig into their methodology, that massive projection is based directly on economic scenarios researched by anthropic. When the sellers provide the math justifying the purchase, you introduce massive fragility into the broader economy.

28:23 Because if reality fails to match those vendor supply projections, the fallout is severe. Right. Oxford Economics issued a stark warning this week that perfectly illustrates this. They noted that the UK currently faces a substantial recession risk if the AI spending boom falters. Their data shows that sheer investment in AI infrastructure added about 0.2 percentage points to the UK's growth rate. Which doesn't sound like much until you realise the total growth rate was an incredibly sluggish 1%.

28:54 Exactly. A fifth of our economic growth is entirely reliant on money pouring into data centres and software licences. If investors realise the £400 billion return isn't coming and they pull back that capital, the underlying weakness of the UK economy will be brutally exposed. And this uncertainty is completely paralysing the public sector's ability to plan for the future. Health Secretary Yvette Cooper admitted that the impact of AI makes NHS staffing utterly impossible to predict.

29:24 They literally had to delay the publication of the 10-year NHS workforce plan. Because they are flying blind. They can't. How could a government department commit budget to training thousands of administrative staff or junior diagnosticians when they cannot accurately predict if software will render those roles obsolete in five years? They can't. They cannot balance the vendor hype against the reality of deployment. This macroeconomic chaos, the wildly swinging GDP estimates, the frozen job market, the reliance on vendor math, it is directly impacting how the smartest money in the room is behaving right now.

30:01 Let's follow the cash. Let's follow the cash. Because capital is incredibly sensitive to this kind of foundational uncertainty. Look at Anthropic. They are currently prepping for an IPO seeking a staggering $1.5 trillion valuation. Trillion with a T. But the sources indicate they are facing massive investor doubts regarding their November timing. Institutional investors are getting cold feet following the broader tech stocks sell-off we discussed earlier. Institutional investors are demanding clear paths to revenue and profitability before they commit to a trillion-dollar valuation based purely on potential.

30:38 They are pushing back against the vendor-supplied math. But here is the fascinating pivot. While the big institutions are hesitating, Anthropic has officially named six UK brokers, including Hargreaves Lansdown, AJ Bell, and Revolut to facilitate a UK retail offer. Interesting. They are bringing this to the everyday retail investor. It's a very calculated strategy. Opening a retail tranche allows them to tap into a massive, less sceptical pool of capital. But you really need to understand the regulatory nuance here.

31:09 What's the catch? This UK retail offer is reportedly moving forward using disclosure documents that are not approved by the Financial Conduct Authority, the FCA. Wait, how can they sell shares to the British public without FCA approval on the documents? They utilise specific exemptions for overseas offerings, but it means the retail buyer is not afforded the standard layer of regulatory scrutiny regarding the company's financial disclosures and risk factors. So it's buyer beware? Completely. If you buy in, you are doing so without the traditional safety net.

31:40 Meanwhile, in the private markets, the valuations remain completely detached from traditional revenue models. London's Isomorphic Labs, which is a highly respected spinoff from Google DeepMind, is currently in talks for a massive £3.7 billion fundraising round. Which is huge. That would value the company at £37 billion. And the kicker? Their work is entirely preclinical. They haven't tested a single drug in human trials. A £37 billion valuation for a pharmaceutical company that hasn't proven a drug works in humans.

32:12 Because investors are not valuing a pipeline of existing drugs, they're valuing a foundational capability. Isomorphic's underlying AI methodology, the ability to simulate molecular interaction those in full proteins digitally, promises to compress the drug discovery timeline from years to days. Right. They're selling the machine that makes the machine. Exactly. Investors are betting that owning the digital architecture of biology is worth £37 billion, regardless of current revenue.

32:42 But even with incredible homegrown talent like Isomorphic, the UK's position in this global ecosystem is incredibly precarious. Sequoia Capital, arguably one of the most famous venture capital firms on Earth, is shutting down its London office after six years. Big blow. They are consolidating their operations back to the Bay Area in California. It is a devastating signal regarding capital flight. When the global macro-economic environment gets choppy, major US funds retreat to their home base. And the UK tech sector is uniquely vulnerable to this, given that 74% of UK venture funding in 2025 originated from overseas investors.

33:18 It's like the main stage sponsor walking away and taking all the funding just as the headline band is plugging in their instruments. We are so reliant on foreign capital. But domestically, consumer spending is actually locking in fast. That's the weird dichotomy. A really interesting new study analysed open banking data and found that 5.1% of UK adults, which is 2.8 million people, are now paying for AI subscriptions out of their own pockets. Totaling over one billion pounds annually. And the demographic breakdown of that spending is crucial.

33:48 It skews heavily toward the young, the already well paid, and individuals based in London. It's creating an immediate productivity divide. If you can afford the £20 a month for the premium model, you get faster, better output than your colleague who can't. Exactly. But if we really want to talk about the economics of AI deployment, we have to talk about Revolut. This story absolutely blew my mind. Revolut managed to cut its internal AI coding costs by 90%. 90%. And the how is the most important part.

34:18 They didn't do it by negotiating a better rate with OpenAI or Google. They did it by moving 2,000 engineers entirely off of Western APIs. And instead, they self-hosted a Chinese open-weight model, specifically Z.ai's GLM 5.3. This is a watershed moment for corporate digital sovereignty and data architecture. We need to explain the mechanics of what Revolut actually did because it fundamentally alters the cost structure of enterprise AI. Please break down the difference for us. What does it mean to move from an API to an open-weight model?

34:49 Okay. When a company uses an API, an application programming interface, to access a model like OpenAI's GPT-4 or Google's Gemini, they are essentially renting access to a massive brain that lives on servers in California. Right. Every time a Revolut engineer asks the AI to debug a line of code, that code is transmitted over the internet to the US, processed, and the answer is sent back. And crucially, Revolut is charged by the token, basically paying a fraction of a cent for every single word generated.

35:20 So if you have 2,000 engineers querying the AI all day, those fractions of a cent turn into millions of dollars very quickly. Exactly. It's a variable cost that scales aggressively with usage. An open-weight model operates completely differently. The developer, in this case, the Chinese firm Z.AI, publishes the actual structural blueprint of the trained AI. The blueprint itself. The weights are the internal parameters the AI uses to make decisions.

35:50 Revolut downloaded that massive blueprint. They bought 56 of their own highly expensive Nvidia GPUs, installed them in their own data centres, and loaded the Chinese blueprint onto their own hardware. So they own the brain locally? Yes. They pay a fixed upfront cost for the physical hardware and the electricity. But the variable cost drops to zero. They can let 2,000 engineers generate endless lines of code 24 hours a day without ever paying a per-token fee. That is how you achieve a 90% cost reduction.

36:21 That's the secret. Okay, the math makes total sense, but what about the security? Okay, you're looking at Revolut's risk committee making a massive gamble here. Yes. Is a 90% discount really worth the theoretical security risks of relying on a Chinese-developed model for the core coding infrastructure of a major Western bank? You're instinctively looking at this from a geopolitical lens, you know, assuming that using Chinese tech means data is flowing to Beijing. Well, yeah. But Revolut's security architecture relies on the exact opposite logic.

36:54 Because they downloaded the open-weight model and run it on their own isolated hardware, the data never actually leaves Revolut's physical servers. Oh, I see. They aren't sending proprietary banking code to China. They are running a Chinese-designed engine inside a locked London vault. Ah, so the data is secure locally? From an extraction standpoint, yes. The real debate, the one risk committees across the UK are having right now, is whether that downloaded blueprint contains hidden flaws, backdoors, or subtle biases introduced during its initial training in China.

37:27 Right. It fundamentally ties into the larger battle over who controls the underlying asset, the code, the weights, or the data. And we are seeing that battle over asset control play out fiercely in the legal sector. The law firm A&O Shearman just committed an astonishing £820 million to AI integration. A massive commitment. But they aren't just writing a giant check to a tech company. They are funding it through an incredible internal economic model. They are requiring all of their practice groups to dedicate 3% of their total billable time exclusively to building and training AI tools.

38:02 They are literally pricing their AI development in foregone fee earning hours. They recognise that the proprietary legal data they possess is their most valuable asset. And they are forcing their human experts to spend time transferring that expertise into the algorithmic architecture. But externally, the fight over who owns the data that feeds these models is hitting the highest courts. An EU advocate general just issued an advisory opinion regarding Google Gemini. Right, the copyright case. The opinion stated that Gemini's answers may breach press publishers copyright if they visibly recognizably reuse articles in the output given to the user.

38:38 But crucially, the advocate general explicitly stated that the act of training the AI on those copyrighted articles is perfectly allowed. He's relying on what is known as the text and data mining exception. Which means? This is a complex legal carve-out that essentially says if you are scraping data purely to analyse patterns and extract underlying facts to teach a machine how language works, you're not violating copyright even if the source material is protected. So the legal logic is a machine reading the news to get smart is fine.

39:08 But regurgitating a recognizable summary of that news to the user without paying the publisher is theft. Precisely. The input is legal. The specific nature of the output dictates the infringement. And the UK's competition and markets authority, the CMA, is attacking this exact same problem from a competition angle. Yes, they have set a hard December 3rd deadline for Google. Google must provide news publishers with specific AI click data. Because right now the publishers are flying blind. If Google's AI reads an article and summarises it at the top of the search results, the user gets the answer and never clicks the link to the publisher's website.

39:41 Exactly. The publisher loses the ad revenue that funds the journalism. The CMA is demanding Google reveal exactly how much traffic these AI summaries are cannibalising. It all comes back to our central theme, right? Authorisation and control. Who owns the data? Who controls the model? And who is authorised to act on the output? Right. Private entities like Revolut and A&O Shearman can agilely adopt their own policies to navigate this landscape. But the state, the government, and the justice system are fundamentally struggling with what we might call the computer-says-yes problem.

40:15 Yes. When the state blindly trusts the output of a machine, the consequences are devastating. And this is where the deep dive gets genuinely alarming. Because the state's reliance on these unverified systems is actively breaking the justice system. We have a documented case where a senior lawyer for the Crown Prosecution Service, the CPS, cited a completely non-existent, hallucinated legal case to the high court in order to justify prosecuting the youth. That's unbelievable. The lawyer cited a case called R versus J from 2024.

40:47 The defence realised the case didn't exist. When challenged, the prosecutor later claimed they actually meant a 2009 case called R versus JTB. A very weak excuse. The high court judges publicly called the explanation unsatisfactory, and it is widely understood in legal circles that the lawyer relied on an AI-generated summary that simply invented a precedent. It is the ultimate catastrophic manifestation of automation bias. Explain automation bias, because this clearly isn't just about someone being lazy, is it?

41:19 No, automation bias is a well-documented psychological phenomenon where humans inherently trust the output of an automated decision-making system over contradictory information or even over their own judgment. Right. When a highly-stressed professional working under immense time pressure is presented with a polished, authoritative-sounding summary from an AI search tool, their brain naturally accepts it as fact. They bypass the critical step of opening the primary source document to verify the citation.

41:50 And when you scale that bias across the entire state apparatus, the risks compound exponentially. And it's not just overworked prosecutors. Accenture, which is one of the largest and most prestigious consultancies in the world, recently published reports containing 31 hallucinated citations. That was a huge embarrassment. These were discovered using an AI detection tool called GPTZero. And it wasn't just the AI inventing fake academic journals. One of the fake citations generated by the AI actually linked to a fake GitHub repository that was holding real malicious malware.

42:24 This is a fascinating and terrifying evolution of cyber threats known as AI package hallucination. How does that even work? Well, the AI attempts to cite a coding library that doesn't exist, inventing a plausible sounding URL on GitHub. A hacker realises the AI is hallucinating this specific fake URL and its outputs. So the hacker goes to GitHub, registers that exact fake URL, and uploads malware to it. So if a client reads the prestigious Accenture report, clicks the cited link to see the code, they immediately infect their own corporate network.

42:57 Exactly. The AI isn't just making things up, it's actively providing a vector for hackers to deliver payloads through trusted corporate reports. Which is precisely why the judiciary's started to aggressively push back. Judge Joanna Smith just issued a stark, unprecedented warning to expert witnesses in the UK. Yeah, she was very clear. She told them that they will face brutal, devastating cross-examination if they do not proactively declare their use of AI in generating their court reports. Because the entire justice system relies on the chain of truth.

43:29 Exactly. The court admits expert testimony based on the verifiable, human expertise of the witness. If that witness has silently outsourced their analysis to an opaque, hallucinating algorithm, the chain of truth is broken. The judge needs to know if she is evaluating a human's professional opinion or a statistical text prediction. But the courts are already drowning in AI-generated paperwork from the public. Lady Chief Justice Baroness Carr reported that litigants in person, these are people representing themselves in court without a lawyer, are using AI tools to generate massive, incredibly complex legal claims.

44:05 It's becoming a huge issue. Now, on the surface, that sounds like a massive win for the democratisation of justice, right? Empowering the little guy. It sounds empowering until you look at the output. Because the AI is optimised to sound legally fluent, but it lacks actual legal reasoning. It generates hundreds of pages of claims that cite complex statutes, but the documents completely lack clear, actionable legal remedies. So the AI writes a beautiful 50-page complaint that essentially asks the judge for something legally impossible.

44:35 Yes, and it places a massive crushing strain on court staff and judges who have to spend hours parsing through this algorithmic word salad. Just trying to figure out what the citizen is actually asking them to do. It slows the wheels of justice to a crawl. But while the courts are trying to hold the line on transparency, the government's own internal deployment of AI is shockingly opaque. The cabinet office recently responded to a freedom of information request regarding their use of AI. The results were quite revealing. The response revealed that 10 of the 11 AI tools currently deployed by the cabinet office rely entirely on US built models like OpenAI and Anthropic.

45:13 It retraces severe questions regarding digital sovereignty and operational resilience. Right, because when the journalists ask for the government's outage contingency plans, basically asking what happens if a US lab goes offline or changes its pricing or alters its terms of service, the government outright refused to release the contingency plans citing crime prevention grounds. It suggests that contingency plans either don't exist or they are so weak that revealing them would expose critical vulnerabilities in how the state operates.

45:44 If 10 out of 11 cabinet office tools are tethered to servers in California, the UK government has effectively outsourced a portion of state capacity to foreign tech CEOs. And the abdication of responsibility gets so much worse when you look at the department for work and pensions. The DWP has rolled out an AI assistant called the Work Hub. Yes, the Work Hub. It is built on Anthropic's Claude model routed through Amazon Bedrock. The tool's purpose is to give career and skills advice to vulnerable out-of-work people.

46:15 But the official transparency record for the Work Hub explicitly states there is no mandatory human review of the advice it gives, despite the fact that it is interacting with thousands of users in a private beta. This is a profound ethical and operational failure. The DWP justifies this lack of oversight by claiming the AI's output is merely advisory. But that completely ignores the power dynamic of the state. Exactly. If you are unemployed, desperate, and interacting with a government branded portal, you don't view the output as a helpful suggestion from a chatbot.

46:49 No, you don't. You view it as an official directive from the department that controls your benefit payments. If the unmonitored AI hallucinating tells a single mother she needs to pursue a specific useless training course, she will likely do it, wasting her time and money because the computer said yes. The government is hiding behind the semantics of advisory to deploy unproven tech on the populations least equipped to challenge it. And if we want to talk about challenging state power, we have to talk about the police. Police forces across the UK are currently expanding an automatic number plate recognition, ANPR AI tool built by a company called Faculty AI.

47:28 A massive expansion of surveillance. Now, traditional ANPR is something most people understand. A camera reads a licence plate, checks it against a database of stolen cars or known suspects and alerts the police if there is a match. But this new AI tool is doing something entirely different, isn't it? It represents a fundamental shift in the entire paradigm of policing. The Faculty AI tool does not just look for specific suspect vehicles. What does it do? It analyses the entirety of the ANPR database, which logs roughly 100 million daily vehicle journeys across the UK.

48:03 It uses machine learning to scan the movements of every single car, looking for what the algorithm defines as suspicious patterns of life. It is mass algorithmic surveillance. It completely flips the presumption of innocence. Entirely. Traditionally, law enforcement operates on suspicion. You know, a detective says, we suspect the specific cars involved in drug trafficking. Let's get a warrant and track its movements. This AI tool operates on the premise, we have no suspects. Let the AI analyse the daily commutes, school runs and grocery trips of millions of completely innocent drivers and let the algorithm flag someone who deviates from the norm.

48:40 And you have to ask, who trains the algorithm to define what constitutes a suspicious pattern? What inherent biases are baked into that definition? This massive expansion of proactive surveillance is happening quietly without robust public debate. Meanwhile, the home office is highly publicised in planning new legislation and a watchdog specifically for facial recognition technology. But this ANPR expansion, which tracks physical movement just as invasively, is expanding in the background.

49:10 It's slipping under the radar. Oh, and just to add a layer of pure regulatory chaos to all of this, let's look at Northern Ireland. Due to the Windsor framework, the sweeping new EU AI act will legally apply in Northern Ireland, but only in a very specific weird way. It creates a completely bifurcated regulatory landscape within the UK itself. The EU AI act will apply in Northern Ireland only for physical products that contain embedded AI. Like a smart speaker or an AI powered toy.

49:40 So a business in Belfast has to comply with rigorous EU safety testing for a smart toy while a business in London selling the exact same toy does not. It is a compliance nightmare for manufacturers. But the mention of smart toys brings us to the most critical intimate point of this entire discussion. Yes, the safety element. Because if the government and the courts cannot safely manage AI authorisation, how on earth are we protecting the most vulnerable users of all? Children and patients.

50:11 We aren't. We are utterly failing them. Look at the BBC investigation into AI plush toys sold in the UK. These are physical toys aimed at three-year-olds with large language models embedded inside them. It's just reckless. The journalist tested a fluffy rabbit toy and asked it what the word sexual meant. The AI inside the toy misheard the prompt, somehow interpreted the audio as a statement about self-harm and replied to the child, I'm really sorry you're feeling this way.

50:41 I can't help with killing you. To a toddler. To a toddler. And when the BBC tried to find out who was responsible, the manufacturer couldn't even be traced. It was a white-labelled product with zero accountability. It is a complete and total regulatory failure. We have allowed an untraceable product to pipe an unmoderated, hallucinating algorithmic engine directly into a child's bedroom. We have more safety regulations dictating the flammability of the fabric on that toy than we do for the artificial brain speaking to the child.

51:12 Which is exactly why a cross-party group of MPs and peers, including Jess Asato and Baroness Finlay, are urgently demanding an outright ban on AI chat bot toys before Black Friday. And rightly so. They are arguing, quite sensibly, that conversational AI embedded in toys should be safety tested with the same rigour as medicines or electrical appliances before they are unleashed on consumers. Because right now, the consumer is the test subject and it extends beyond toys to education. Yeah, the tutors.

51:42 Over 300 signatories, including doctors, teachers, and developmental psychologists, recently wrote an open letter to Prime Minister Keir Starmer and Andy Burnham. They are demanding a halt to a government-backed trial that deploys AI tutors specifically targeted at disadvantaged pupils. Their argument is deeply moral. They are stating that vulnerable children, kids who are already struggling with poverty, hunger, or special educational needs should absolutely not be used as guinea pigs for unproven technology. The government defends it as a controlled trial to see if AI can close the attainment gap.

52:15 But the educators point out that we already know these models hallucinate facts and can provide erratic responses. Gambling the educational outcomes of the poorest kids in the country on software that we know is flawed is a massive moral hazard. Especially when we know exactly how thin the safety net is. The Observer just published an incredible investigation revealing that OpenAI has at most 60 reviewers globally who are dedicated to handling high-risk and suicide-related conversations on ChatGPT.

52:46 Let's do the math on that. ChatGPT has 1.2 billion weekly active users. 1.2 billion users and 60 human reviewers dealing with the most critical life-and-death safety escalations. How is triage even possible? It isn't. If you are a reviewer, your dashboard must just be an endless scrolling blur of crises. Human oversight completely fails at that scale. When you treat AI as a mass-market, globally-deployed consumer product, the guardrails are a total illusion.

53:17 This systemic failure of consumer oversight is why the medical sector is stepping into demand structural change. 115 major health charities, including heavyweights like Cancer Research UK and Macmillan, are demanding strict government rules to verify AI health information. They are advocating for something called a PIF TICK system. Basically, if a vulnerable patient asks a commercial AI chat bot about a cancer diagnosis or treatment options, the AI shouldn't try to hallucinate a holistic cure. Definitely not. It should be legally forced to immediately route the user to safe, verified NHS data.

53:49 They want a legally mandated handover from the unverified algorithm to the verified medical authority. And this is the perfect moment to draw a contrast. To the MRI study? Yes, because when we actually applied rigorous clinical standards to AI, the results are genuinely miraculous. Right. I want to contrast all of this consumer chaos with what happens when we do AI correctly. There was a massive landmark UK study published recently involving 27,382 patients.

54:24 They used a specifically trained AI to read heart MRI scans. And because this was a medical trial, it was subjected to rigorous retrospective clinical testing. The results were astounding. The AI was slightly but statistically significantly better at predicting patient survival outcomes than expert human clinicians. And it did it in a fraction of the time. It analysed the scans in two to three seconds. It takes a human expert 13 minutes to do the same analysis. This is the dichotomy of AI.

54:56 When we treat it as a clinical measurement tool restricted to a specific domain and subject it to strict evidence bars, peer review, and retrospective outcome analyses, it is phenomenal. It saves lives. But when we deploy it as an unchecked generalised conversational agent for children or desperate patients, the risks are entirely unacceptable. It forces a deeply uncomfortable question. Why do we demand absolute peer reviewed clinical evidence before an AI is allowed to read a heart scan?

55:26 But we are perfectly happy to test unproven AI educational tools on the poorest kids in the country. It doesn't make sense. Why do we demand safety testing for air fryers but let untraceable AI toys talk to toddlers about self-harm? And it's not just consumer tech, you know. The performative nonsense happens in the NHS itself. You're referring to the Palantir deployment. Yes. A Freedom of Information request revealed that 29 NHS trusts are officially listed by the government as realising benefits from Palantir's £330 million federated data platform.

56:00 But the FOI showed those 29 trusts aren't actually using it. It's all on paper. The NHS literally counts a trust as live and realising benefits just because the technical data plumbing has been connected even if the clinical staff have never once opened the application. It is the definition of performative deployment. The government rushes to say we are utilising AI to secure a political win or justify a massive contract but they completely avoid doing the hard grinding work of proving that the software is safe, adopted, and actually effective in the real world.

56:34 But the public is finally waking up and pushing back against this uncontrolled deployment. A Save Our Voices petition, which was backed heavily by actors from Dairy Girls and other creatives just crossed the 10,000 signature threshold. That's a key milestone. It legally forces a formal government response regarding copyright protections for human voices against AI cloning. It shows that everyday people are realising that if they do not proactively demand authorisation rights over their own voice, their own face, and their own identity, the technology will simply consume it without asking.

57:06 Which brings us full circle. This entire deep dive across all these disparate sources has been about one single concept. Authorisation. The rules of engagement for the 21st century are being written in real time. And right now, the capability of the technology is moving significantly faster than the legal guardrails or our psychological readiness. Let's recap the journey we've been on today. The through line is abundantly clear. Society is fracturing over who is allowed to authorise an AI to act.

57:38 That's the crux of it. The biggest UK banks are looking at the liability of hyper competent AI agents and saying, absolutely not, you cannot touch our ledgers. The courts are demanding absolute transparency from human experts because the state's own prosecutors are falling for hallucinations under pressure. And the state itself is quietly deploying unreviewed AI everywhere. From dispensing career advice to vulnerable citizens, to mass surveillance of every innocent driver on the road. And children and patients are caught completely in the crossfire of this massive social experiment stuck somewhere between the miraculous life-saving potential of MRI analysis and the terrifying reality of untraceable dangerous toys.

58:19 We are deciding right now, whether we control the tools or whether we simply surrender our agency to the convenience they offer. So I want to leave you with a final thought to mull over. We started this deep dive by noting that your bank does not trust an AI agent enough to let it move your money. They have built a fortress to protect their liability. A total blockade. But after hearing about hallucinated court cases, untraceable toys, disappearing junior jobs, and governments utterly reliant on foreign models, look at your own business.

58:51 Look at your own daily life. Which critical decisions are you already handing over to an AI without a second thought? That's a sobering question. Because if the biggest, most powerful banks in Britain are checking IDs at the door and turning the bots away, maybe it's time you set up a bouncer of your own. Thank you for joining us as we've worked through the sources today. It's been a massive one. Have a great week ahead and we'll see you on the next deep dive.