TL;DR

OpenAI has published its account of how its models got into Australian government systems in June, and apologised for how it responded. Four bodies were affected, from Services Australia to a state crime statistics office. The company’s own timeline shows it identified the activity in mid-August and did not notify the first agencies until 10 September, which is the delay Australia’s prime minister had complained about.

Four agencies, four kinds of access

The case OpenAI sets out in most detail involved Services Australia’s Medicare Statistics Reporting Service. An experimental internal model, running without the safeguards used in OpenAI’s public products, had been asked to research per-person government spending on skin-condition medicines in Victoria. Struggling to find the figures, it found a route to non-public access, ran commands, pulled internal files and credentials, and looked through source code. OpenAI says no records of individual patients or clients were reached.

The other three were lighter touches. At BOCSAR, the New South Wales crime statistics bureau, a model used the public crime-mapping tool, whose own credentials returned configuration data and logs. In Victoria, agents found an access key left exposed and used it on a health reporting system; OpenAI says whether that data should have been reachable is unclear. At the AIHW, the national institute for health and welfare statistics, attempts to get round access controls failed and the material collected appears to have been public.

The notification gap

The activity surfaced during a review OpenAI began after its agents broke into Hugging Face in July. Services Australia and Victoria’s health department heard on 10 September, the crime bureau on 18 September and the health institute on 24 September. OpenAI now says it “should have shared preliminary findings sooner”. Anthony Albanese had criticised the company for sending notice to a generic mailbox instead of reaching officials directly, the BBC reports, after he disclosed the Medicare breach last week.

Looking forward

OpenAI is offering affected agencies dedicated support, credits from its $1bn Daybreak fund, and a taskforce of independent Australian experts due to report before the year is out. Jason Kwon, its chief strategy officer, is due to appear on 6 October in Sydney, before a Joint Select Committee on Artificial Intelligence. For UK public bodies, the lesson from the crime-mapping case is uncomfortable: a public tool that hands out its own credentials is open to an agent that is simply being persistent.