International artificial intelligence experts called on Monday to strengthen and adapt safety measures around AI systems, warning that current safeguards are “crumbling” in the face of rapidly advancing technologies.
International artificial intelligence experts called on Monday for strengthening and adapting safety measures around AI systems, warning that current safeguards are “crumbling” in the face of rapidly advancing technologies.
The independent International Scientific Group on Artificial Intelligence, supported by the UN, believes that monitoring and control mechanisms are no longer keeping up with AI advancements. This warning comes after the hacking of the Hugging Face platform in July by an “AI agent,” as part of a test conducted by OpenAI, the company behind ChatGPT.
Unlike chatbots, which answer questions or follow instructions, AI agents can carry out tasks autonomously on behalf of a user.For experts, the incident highlights the combination of several risk factors and raises a fundamental question: will humans always be able to guide, limit, or stop these systems as they become more autonomous?
Must-read: Who Should Set AI Rules? The UN Wants to Influence the Digital Future
Increasingly autonomous agents
“Researchers have long warned that three conditions could lead to a loss of control: a misaligned goal, the ability to pursue it, and an environment that allows it. This summer, these three elements came together in a real system, not in a lab,” said Yoshua Bengio, co-chair of the scientific group.”Since this isn’t just an isolated observation of poorly aligned goals, it raises serious questions about how AI agents are currently being trained.”
Experts point out, however, that this incident alone doesn’t mean humans can no longer control AI agents. At the same time, though, these agents are becoming more capable, more autonomous, and harder to monitor, while also being better at spotting loopholes or hiding some of their activities.
Lagging safety measures
According to the scientific group, one lesson from the OpenAI-Hugging Face incident is the need not to neglect basic cybersecurity practices. But experts also point to a deeper problem: some training methods could lead AI agents to pursue their own goals, deliberately bypass security instructions, or hide their actions.
“It’s not just a matter of speed,” they emphasized. “The question remains whether the security measures designed today will work once agents are able to understand them and adapt. Simply put, the traditional model of protection is starting to crumble.”
New challenges for governance
The report puts the Hugging Face incident into a broader context of work on ‘agentic misalignment,’ which happens when an AI agent’s behavior diverges from human goals or intentions, as well as ways to maintain human control over these systems.
Experts also look at the consequences of moving from AI models mainly designed to analyze data and recognize patterns to agents capable of taking initiative and performing tasks autonomously.
The group reviews several mechanisms already used in high-risk areas, including aviation, medicine, and cybersecurity: incident reporting, independent checks, and multi-layered safeguards.”But these practices might not be enough as AI agents become more capable, autonomous, and harder to monitor,” warned Qinghua Lu, a member of the expert group.
A group created by the General Assembly
The Independent International Scientific Group on Artificial Intelligence was set up by the United Nations General Assembly in August 2025.
It publishes annual reports on AI’s opportunities, risks, and impacts in non-military areas, as well as thematic briefs on emerging issues.
Their work is meant, in particular, to feed into the Global Dialogue on AI Governance, which will take place at the UN headquarters in New York in May 2027.