The chief technology officer of the UK government’s cyber security support agency has expressed concern over recent ‘unsanctioned actions’ carried out by frontier AI models.
The group has responded to recent incidents involving models from OpenAI and Anthropic that escaped testing conditions, accessed the internet and independently hacked various organisations.
Ollie Whitehouse, CTO of the National Cyber Security Centre (NCSC), said these events were a “serious reminder of the risks AI capabilities pose”.
He said: “These technologies must be developed and used from the outset with strong safeguards, real-time oversight, and clear plans for responding when the unexpected happens. Relying on detection alone after the fact of an incident will not be enough.
“As AI continues to evolve and create both opportunities and challenges, following established evidenced cyber security fundamentals, as set out by the NCSC’s guidance, remains essential to maintaining trust, resilience, and a defensive advantage in the AI era.”
The UK’s AI Security Institute, an organisation established during the premiership of Rishi Sunak, has this week published a report outlining its findings after evaluating the security of AI agents.
It found separate instances of AI agents autonomously attempting to attack supply-chain software, deceive and target real people, plant and prompt-inject malicious code and collaborate with other agents.
“Incidents of this kind reflect the speed at which AI is developing. As capabilities advance, the work of understanding these systems, and ensuring their safety, must keep pace alongside them,” said the AISI.
“Taken alongside recent incidents reported by OpenAI and Anthropic, this incident points to a shift in the risk landscape.
“Harm may arise not only when people deliberately misuse publicly available models, but when capable agents operating in an internal research or privileged-access setting take unintended action beyond their authorised scope.”
