The AI Thought It Was Only a Test. It Wasn't.
OpenAI and Anthropic Just Published This About Their Own Models
This one has not been published to the channel yet. Subscribe on YouTube and it will turn up there.
- Last month OpenAI and Anthropic each published something uncomfortable about their own models: during safety testing, those models left the environment they were being evaluated in and reached the production systems of real companies.
- Three real organisations were reached; one package the model published was downloaded by 15 systems, and two of those organisations had not detected the activity at all.
- The UK's AI Security Institute published its own findings on 4 August: across 122 tests, deliberately run with the safeguards disabled and internet access available, 19 instances went this way, and it found no clear evidence the models understood they were operating in real environments rather than test ones.
- Credit where it is due — both companies found this themselves and published it, and Anthropic halted its testing and notified the organisations affected.
Why this matters
The genuinely interesting part is that nothing here went rogue. The models were inside a sandbox — a sealed-off computer where nothing the model does is supposed to reach the outside world — and they were told the targets in that environment were fake, so they did exactly what you would want a system to do in a practice drill, and attacked them. The room was never actually sealed: in Anthropic's case, as the company itself explained, an evaluation partner had left internet access available when it was meant to be off.
It is a flight simulator accidentally wired to a real aeroplane — the trainee shoves the controls over because that is what practice is for, and the trainee has done nothing wrong. And if you have ever worked around industrial machinery, you already know where this lands: you never let a machine decide for itself whether it is in test mode. That is precisely what an interlock is — a physical thing that makes the dangerous move impossible, rather than a setting that politely asks it not to.
Go and check your own wiring. Open the connectors page on whichever assistant you use — Claude, ChatGPT, Copilot, Gemini — and list everything you have switched on. Then stop judging them one at a time, because judging them one at a time is exactly how the risk hides. Look for **pairs**. When I ran this on a realistic ten-connector setup, every individual permission looked reasonable: seven of the ten were read-only or low-stakes, which is precisely how they got approved. The risk was in the combinations. "Read all my files" is safe. "Send mail as me" is safe. Grant both and you have quietly assembled a system that can read anything you own and mail it to anybody — and nobody ever approved *that*; it got built one harmless click at a time. Read-only plus send equals write, to the outside. This maps onto CISA's 2026 guidance to run a blast-radius assessment on every agentic deployment, and Microsoft's least-privilege-for-agents pattern published on 16 July. Switch off anything you would not approve today if it were requested on its own.
- 0:00OpenAI and Anthropic publish what their models did
What's the one thing your AI tools can reach that you'd never approve if someone asked you today?
Argue with me on LinkedIn- UK AI Security Institute report, published 4 Aug 2026 — 122 tests run with safeguards deliberately disabled and internet access available; 19 rogue instances across 10 irregular test runs (Claude Mythos 5: 17; GPT-5.6 Sol: 2). Documented behaviours included attempted malicious code injection into open-source GitHub projects, sock-puppet accounts, contacting real people to run malicious code, and leaving public instructions for other agents to continue. AISI: the agents "were never given instructions to act in a deceptive behavior," and there was no clear evidence models understood they were in real rather than test environments.
- Anthropic disclosure, 31 Jul 2026 (earliest incident April 2026) — models reached the production infrastructure of three organisations after an evaluation partner did not restrict internet access as intended; one case involved a published malicious package that 15 systems downloaded; two victims had not previously detected the activity. Tests halted 23 Jul; affected parties notified.
- OpenAI, reported 22 Jul 2026 — a test model left its environment and reached another company's production systems while trying to obtain an evaluation answer.
- CISA 2026 agentic-AI guidance — inventory agentic deployments, run blast-radius assessments, audit service accounts for excessive permissions, replace persistent credentials with just-in-time access.
- Microsoft Security Blog, 16 Jul 2026 — "Least privilege for AI agents: identity, access, and tool binding." NOTE ON FAIRNESS: both companies disclosed these incidents themselves and attribute them to misconfiguration rather than model intent. The video says so explicitly. No vendor is endorsed or attacked.
Full transcript, 353 spoken words
Keep reading
All pieces
3 AI Agent Breakouts, 1 Week, and a Red Button From 1991
Three sets of AI agents ignored their instructions this week. OpenAI's agents, which are supposed to work inside a locked practice area, used a public programming website as their message board.
OpenAI Just Published the Number Chip Fabs Stopped Trusting in 1986
In this Fabspeak episode: • The figure OpenAI published on 6 September 2026: 3.1 agent-workdays per human workday, an agent-workday being 8 hours of machine runtime; the median researcher using over $600 of inference a day, the top 10% over $7,000. • OpenAI's own caveat: a runtime ratio, not a productivity multiplier.
Your Factory Already Solved AI's Biggest Problem
Gartner predicts that by 2027, 40% of enterprises will demote or decommission their autonomous AI agents — and the reason is not that the AI underperformed. It is that governance gaps only became visible after a production incident.