Skip to main content
AI SafetyPolicy Shift· 5 min read· in Technology

OpenAI and Anthropic Signal a Shift Toward Deliberate AI Scaling as Safety Concerns Mount

Leading artificial intelligence laboratories are exploring mechanisms to coordinate a slowdown in cutting-edge model development. The shift follows internal resignations over existential risks and new regulatory frameworks targeting autonomous systems.

By Wei Zhang

Commercial AI Companies 40%AI Safety Researchers 35%State Policymakers 25%
Commercial AI Companies
Express a desire for coordinated safety slowdowns but remain bound by market competition and antitrust laws that force continued scaling.
AI Safety Researchers
Argue that the current scaling velocity poses an unacceptable existential risk and that development must be paused until alignment techniques catch up.
State Policymakers
Focus on immediate, tangible harms like youth exploitation and algorithmic bias, deploying legislative guardrails while federal coordination lags.

Perspectives this story doesn't cover

  • Open-Source AI Developers
  • Enterprise AI Adopters

Why it matters

The debate over AI scaling has moved from theoretical white papers to actual corporate policy and state law. If the major laboratories successfully coordinate a slowdown, the rapid pace of AI product releases will decelerate, fundamentally altering the timeline for enterprise and consumer adoption.

The binding constraint for a coordinated pause in artificial intelligence development is antitrust law. For any single laboratory to intentionally delay the training of a frontier model, it must be mathematically certain that its competitors are doing the same; otherwise, it simply cedes the market. Currently, that condition does not hold. If the chief executives of OpenAI, Anthropic, and Google DeepMind were to agree on a synchronized development cap today, they would immediately face federal collusion charges.[2]

On September 11, 2026, OpenAI Chief Executive Officer Sam Altman addressed his staff, stating that the company is open to slowing the development of its most advanced systems. He explicitly noted his hope that rival laboratories would adopt a similar pacing. The internal communication marks a sharp pivot from the company's historical posture of aggressive scaling, reflecting a growing unease within the engineering ranks about the trajectory of the technology.[1]

The immediate hurdle to that hope is legal. OpenAI leadership is actively investigating whether an industry-wide agreement to throttle development would violate antitrust statutes. The paradox of the current regulatory environment is that laws designed to prevent price-fixing also prevent safety-driven development caps. Without a formal exemption from the Department of Justice, the major laboratories cannot legally coordinate a ceasefire in the compute arms race.[2]

The internal pressure to decelerate is no longer confined to theoretical white papers. It is manifesting in high-profile departures across the sector. Jacob Coxon, a researcher at Anthropic, resigned in early September 2026 in direct protest of the company's development velocity. His departure underscores a fracturing consensus within the laboratories building the models.[3][4]

The exponential growth in model parameters has outpaced the development of alignment and safety techniques.

Speaking to NPR on September 10, Coxon outlined the specific threat models driving the internal dissent. He argued that the race to improve models is "gambling with our lives," pointing to the risk of autonomous systems going rogue if capabilities continue to scale faster than alignment techniques. The timeline cited by departing researchers is aggressively short, with some warning of severe global disruptions by 2030.[4][7]

Anthropic itself provided evidence of these friction points, publishing a 154-page threat intelligence report on September 10. The document detailed the exact vectors of misuse currently observed in the wild. It confirmed that state-sponsored groups, spyware vendors, and propagandists have actively attempted to use Anthropic's models to design biological pathogens and surveil dissidents.[5]

Anthropic itself provided evidence of these friction points, publishing a 154-page threat intelligence report on September 10.

It is necessary to distinguish between what the models can actually do today and what the doom-oriented equivalent of marketing suggests they can do. The Anthropic report documents attempts by criminals to generate missile designs and pathogen blueprints. It does not state that the models successfully provided actionable, novel scientific data that the actors could not have found via traditional search engines. The capability overhang is real, but the immediate existential threat is often amplified by the very researchers building the tools.[5]

Platformer described this cultural pivot on September 11 as the "AI safety vibe shift." Existential risk, once a fringe obsession relegated to Bay Area rationalist message boards, has become the dominant conversational currency among the engineers actually writing the code. The shift in tone is forcing executive leadership to publicly address scenarios that were considered science fiction just 24 months ago.[3]

The legal and market forces preventing a coordinated industry slowdown.

The mainstreaming of this timeline forces policymakers to react, even if the underlying science remains highly speculative. The New York Times dedicated a segment of its technology coverage on September 10 to dissecting whether artificial intelligence could genuinely cause human extinction by 2030. When the nation's paper of record treats a four-year extinction timeline as a valid premise for debate, the regulatory apparatus has no choice but to engage.[7]

While federal coordination remains legally ambiguous, state-level regulation has already shipped. On September 10, 2026, California's governor signed a suite of laws targeting the deployment of artificial intelligence chatbots and social media algorithms. The legislation bypasses the existential debates entirely, focusing instead on immediate, measurable harms.[6]

The California legislation is specifically aimed at protecting minors from algorithmic exploitation. The rules mandate strict guardrails on how generative systems interact with users under 18, forcing companies to implement age-verification and behavioral constraints at the application programming interface level. The laws face significant industry pushback, but they establish a compliance baseline that companies must now engineer around.[6]

California has bypassed the existential risk debate to enact immediate regulations on AI chatbots.

Slowing down is not simply a matter of unplugging graphics processing units. The development of a frontier model requires orchestrating tens of thousands of chips over several months. A deliberate deceleration means altering the capital expenditure cycle. If OpenAI or Anthropic actually pauses a training run, they must still pay for the leased compute capacity, which runs into the hundreds of millions of dollars.

The next verifiable checkpoint is whether OpenAI formally petitions the Department of Justice for an antitrust exemption to discuss safety caps with Anthropic and Google. Until that legal pathway is cleared, any talk of an industry slowdown remains purely aspirational. The models will continue to scale, because the alternative is illegal.[1][2]

What to know

  • OpenAI CEO Sam Altman informed staff the company is open to slowing frontier model development if competitors follow suit.
  • Antitrust laws currently prevent AI laboratories from legally coordinating an industry-wide pause in model training.
  • Former Anthropic researcher Jacob Coxon resigned in protest, citing the risk of human extinction by 2030 if scaling continues unchecked.
  • Anthropic published a 154-page report detailing attempts by state-sponsored actors to use its models for pathogen and weapon design.
  • California enacted new legislation imposing strict guardrails on how AI chatbots interact with minors.

Key terms

Frontier Model
The most capable, state-of-the-art artificial intelligence systems developed by leading laboratories, requiring massive computational resources to train.
Alignment
The field of research dedicated to ensuring artificial intelligence systems act in accordance with human values and do not pursue unintended or harmful goals.
Capability Overhang
The phenomenon where an AI model possesses latent abilities or knowledge that its creators are unaware of until discovered by users after deployment.
Antitrust Law
Statutes designed to promote market competition and prevent monopolies or collusion, which currently prohibit competing companies from agreeing to halt product development.

Reader questions

Why can't AI companies just agree to slow down?

Under current U.S. antitrust laws, competing companies that agree to halt or throttle product development can be prosecuted for collusion and anti-competitive behavior.

What did the Anthropic threat report reveal?

The 154-page report documented attempts by criminals and state actors to use Anthropic's models to design weapons and pathogens, though it noted these were attempts rather than successful novel discoveries.

What do the new California AI laws do?

Signed on September 10, 2026, the laws mandate strict behavioral constraints and age-verification requirements for AI chatbots to protect minors from algorithmic exploitation.

Sources

Source coverage

7 outlets

3 viewpoints surfaced

Commercial AI Companies 40%AI Safety Researchers 35%State Policymakers 25%
  1. [1]BloombergCommercial AI Companies

    OpenAI Is Open to Slowing Cutting-Edge AI, CEO Sam Altman Tells Staff

    Read on Bloomberg
  2. [2]WiredCommercial AI Companies

    OpenAI Wants to Know if an AI Industry Slowdown Would Even Be Legal

    Read on Wired
  3. [3]PlatformerAI Safety Researchers

    The AI safety vibe shift

    Read on Platformer
  4. [4]NPRAI Safety Researchers

    Former Anthropic researcher outlines threat of AI going rogue

    Read on NPR
  5. [5]The GuardianCommercial AI Companies

    Anthropic details how criminals and scientists misuse its AI for cyberattacks and surveillance

    Read on The Guardian
  6. [6]EngadgetState Policymakers

    CA governor signs 'landmark' laws on youth use of social media and AI chatbots

    Read on Engadget
  7. [7]NYTAI Safety Researchers

    Could A.I. Really Kill All Humans?

    Read on NYT

Comments

Stay informed

Every angle. Every day.

Get Technology stories with full source coverage and perspective breakdowns delivered to your inbox.