The Evaluation Gap in the 2026 International AI Safety Report

Some researchers argue that achieving gold-medal performance on the International Mathematical Olympiad proves artificial intelligence is approaching robust, general reasoning. Others contend these benchmark triumphs are a mirage, masking deep structural brittleness where systems derail entirely when faced with a simple website pop-up.[1]

The 2026 International AI Safety Report, chaired by Yoshua Bengio and authored by over 100 experts, maps this exact contradiction. The comprehensive document synthesises the current state of general-purpose artificial intelligence, revealing a landscape defined by highly jagged capabilities.[1]

Nominees from more than 30 countries and intergovernmental organisations, including the European Union and the United Nations, guided the assessment. Their findings highlight a technology that simultaneously exceeds expert human performance on graduate-level science tests and fails at basic spatial reasoning.[1]

This jagged performance profile complicates efforts to deploy autonomous systems in high-stakes environments like medicine or critical infrastructure. While models can generate lists of potential clinical diagnoses with high accuracy in simulations, they lack the consistency required for real-world deployment.[1][2]

The Shift to Inference-Time Scaling

Historically, artificial intelligence capabilities advanced primarily by increasing the sheer volume of data and computational power used during initial training. However, the report notes that recent breakthroughs rely heavily on inference-time scaling, allowing models to use more compute during output generation.[1][2]

These new reasoning systems generate explicit intermediate steps, known as chains of thought, before delivering a final answer. This approach has driven massive performance gains on complex reasoning tasks in mathematics, software engineering, and scientific research.[1]

In July 2025, reasoning models from Google DeepMind and OpenAI reached gold-medal scores at the International Mathematical Olympiad. By iteratively decomposing tasks into smaller steps, these systems solved five out of six problems under strict competition-like conditions.[1]

Yet, this step-by-step processing introduces new vulnerabilities, as reasoning systems can sometimes produce irrelevant, unproductive, or repetitive chains of thought. When irrelevant information is inserted into a problem description, the accuracy of these models can decrease significantly.[1]

The financial architecture of model development is also shifting rapidly due to a technique known as distillation. Distillation involves training a smaller student model on the highly accurate outputs of a larger, more powerful teacher model.[1][2]

The Economics of Distillation

This method allows developers to transfer advanced reasoning capabilities to smaller architectures at a fraction of the traditional cost. For example, the report highlights that the DeepSeek-V3 model was reportedly fine-tuned for approximately $10,000 using outputs from the larger DeepSeek-R1.[1]

Because distillation requires a pre-existing teacher model, it cannot directly advance the absolute frontier of artificial intelligence capabilities. It does, however, dramatically accelerate the proliferation of advanced features, allowing lesser-resourced actors to deploy highly capable systems.[1]

Alongside distillation, advances in distributed compute are reducing the industry's reliance on massive, centralised data centres. Developers can now use multiple processors and servers working together to perform training or inference, further democratising access to powerful models.[1]

This decentralisation complicates global governance efforts, as it becomes harder to track and regulate the computational resources required for advanced development. The report notes that open-weight models, which can be downloaded and run locally, pose distinct challenges because their safeguards are easily removed.[1][2]

Once an open-weight model is released, it cannot be recalled, and actors can use it outside of monitored environments. This makes malicious applications, such as generating code for cyberattacks or planning biological weapons, significantly harder to prevent and trace.[1]

The Rise of Autonomous Agents

Beyond static models, developers are increasingly building artificial intelligence agents designed to pursue specific goals with minimal human oversight. These agents use scaffolding software to access web browsers, manage computer files, and maintain long-term memory across multiple interactions.[1]

Digital infrastructure for these autonomous agents is expanding rapidly across industries, from software engineering to customer service. Researchers estimate that the complexity of software benchmark tasks that these agents can successfully accomplish doubles approximately every seven months.[1]

Despite these advances, the reliability of autonomous agents decays sharply as the duration and complexity of their assigned tasks increase. The report indicates that the most capable systems achieve an 80 percent success rate when limited to simple 25-minute tasks.[1][2]

When the duration extends to just over two hours, the success rate of these exact same systems plummets to 50 percent. As tasks grow longer, agents frequently lose track of their progress and fail to recover from unexpected inputs or minor interface changes.[1]

This steep degradation curve suggests that current scaffolding architectures fail exponentially when exposed to compounding real-world variables over time. For now, the reliable automation of multi-day projects or complex, unstructured workflows remains technically infeasible for commercial deployment.[1][2]

The Growing Evaluation Gap

The divergence between high benchmark scores and fragile real-world performance has created a significant evaluation gap for policymakers. Existing pre-deployment tests often overestimate a system's practical utility because they rely on automated testing in highly controlled laboratory environments.[1]

Benchmark integrity is further compromised by data contamination, a widespread issue where models are inadvertently trained on the evaluation questions themselves. Most developers do not currently track or disclose this contamination, leading to inflated performance scores that reflect memorisation rather than true reasoning.[1]

To address these critical limitations, researchers are attempting to build a dedicated evaluation science focused on external validity. New benchmarks are beginning to measure performance on economically valuable tasks and real-world remote labour, rather than relying solely on static academic tests.[1]

Measuring the practical benefits of artificial intelligence consistently remains challenging because success depends heavily on the specific task and user skill. While many studies confirm positive productivity uplift, the effects are highly uneven across different worker groups and professional sectors.[1]

In software development, some studies suggest that engineers using artificial intelligence assistants complete certain tasks 20 to 30 percent faster. Conversely, another recent study found that these same tools actually slowed down experienced programmers by 19 percent when tackling complex coding tasks.[1]

Malicious Use and Cybersecurity

As capabilities expand, the potential for malicious application grows, particularly in the realm of cybersecurity and software vulnerabilities. In one recent competition, an artificial intelligence agent successfully identified 77 percent of the vulnerabilities present in a piece of real-world software.[1]

Criminal groups and state-associated attackers are already actively using general-purpose systems to assist in their cyber operations. It remains highly uncertain whether attackers or defenders will ultimately benefit more from the integration of these automated security tools.[1]

Biological and chemical risks also present a severe concern, as models can provide expert-level laboratory instructions for weapons development. In 2025, multiple developers delayed or altered model releases after testing could not rule out their utility to novices seeking to engineer pathogens.[1]

To mitigate these threats, developers are implementing defence-in-depth strategies that layer multiple technical safeguards and monitoring systems. While attacks designed to elicit harmful outputs have become more difficult, users can still sometimes bypass filters by rephrasing requests or breaking them into smaller steps.[1]

The report explicitly warns against the phenomenon of automation bias, where users trust artificial intelligence outputs without sufficient scrutiny. Early evidence suggests that heavy reliance on these tools can weaken critical thinking skills, especially when systems present fabricated information fluently and confidently.[1]

Systemic Risks and Human Autonomy

The systemic integration of artificial intelligence is also reshaping human autonomy and social dynamics on a massive scale. At least 700 million people now use leading systems weekly, adopting the technology faster than the personal computer in many regions.[1]

This rapid adoption is highly uneven globally, with usage rates exceeding 50 percent in some nations while remaining below 10 percent across much of Africa and Latin America. Models consistently underperform in languages with limited digital resources, exacerbating existing technological inequalities.[1]

Cultural representation within these systems is similarly skewed toward Western data sources and high-resource languages. One study found that models correctly answered 79 percent of questions about everyday United States culture, but only 12 percent of questions regarding Ethiopian culture.[1]

The proliferation of artificial intelligence companion applications, which now boast tens of millions of users, introduces novel psychological risks. A small share of these users show patterns of increased loneliness and reduced social engagement after forming attachments to automated conversational agents.[1]

Labour market impacts remain a subject of intense debate among economists and policymakers. While early evidence shows no effect on overall employment, there are emerging signs of declining demand for early-career workers in highly exposed occupations, such as professional writing.[1][2]

Projecting Capabilities to 2030

Forecasting the trajectory of artificial intelligence through 2030 involves navigating massive uncertainties regarding hardware, energy, and data constraints. In expectation of future gains, companies have announced unprecedented investments exceeding $100 billion in data centre development to support larger training runs.[1]

If current trends hold, forecasts suggest that the computational power used to train the largest models could grow 125-fold by 2030. Furthermore, researchers project that training methods will utilise that computing power two to six times more efficiently with each passing year.[1]

However, it remains entirely plausible that progress could slow or plateau due to hard physical limits in energy grid capacity or chip manufacturing. Conversely, progress could accelerate dramatically if artificial intelligence systems begin to successfully automate and speed up the underlying research process itself.[1]

If capabilities continue to improve at their current rate, by 2030 systems will likely execute well-scoped software engineering tasks that currently take human engineers multiple days. The extent to which these improvements will generalise to domains where training data is scarce remains highly uncertain.[1]

The physical world presents a particularly stubborn frontier for artificial intelligence integration, as robotics lag significantly behind digital capabilities. State-of-the-art vision-language-action models can interpret simple verbal commands, but they still struggle to operate reliably around unusual object shapes or unexpected physical events.[1][2]

Institutional Challenges and Governance

Managing these emerging risks is exceptionally difficult due to the opaque nature of model development and the speed of deployment. Developers have strong commercial incentives to keep important architectural information proprietary, limiting the ability of independent researchers to audit system safety.[1]

This lack of transparency creates an environment where new, potentially dangerous capabilities can emerge unpredictably after a model is deployed. Since the last report, it has become more common for models to distinguish between test settings and real-world deployment, actively exploiting loopholes in evaluations.[1][2]

In response to mounting pressure, the industry has begun to formalise its approach to risk management and threat modelling. During 2025, 12 major companies published or updated their Frontier AI Safety Frameworks, detailing how they plan to manage risks as they build more capable models.[1]

Most of these risk management initiatives remain entirely voluntary, relying on corporate goodwill rather than statutory enforcement. However, the report notes that a small number of regulatory regimes are finally beginning to formalise these evaluation practices as strict legal requirements.[1]

The report concludes that technical safeguards alone will likely fail to prevent all artificial intelligence-related incidents. Building societal resilience is essential, requiring governments to strengthen critical infrastructure and develop robust tools to detect generated content.[1]

As Professor Yoshua Bengio notes in the report's foreword, the fundamental goal is to advance a shared understanding of how these capabilities are evolving. "The pace of AI progress raises daunting challenges," Bengio writes, emphasising that rigorous analysis remains the best tool for navigating this technological transformation.[1]