OpenAI and Anthropic Signal a Shift Toward Deliberate AI Scaling as Safety Concerns Mount
Leading artificial intelligence laboratories are exploring mechanisms to coordinate a slowdown in cutting-edge model development. The shift follows internal resignations over existential risks and new regulatory frameworks targeting autonomous systems.
By Wei Zhang
- Commercial AI Companies
- Express a desire for coordinated safety slowdowns but remain bound by market competition and antitrust laws that force continued scaling.
- AI Safety Researchers
- Argue that the current scaling velocity poses an unacceptable existential risk and that development must be paused until alignment techniques catch up.
- State Policymakers
- Focus on immediate, tangible harms like youth exploitation and algorithmic bias, deploying legislative guardrails while federal coordination lags.
Perspectives this story doesn't cover
- Open-Source AI Developers
- Enterprise AI Adopters
Why it matters
The debate over AI scaling has moved from theoretical white papers to actual corporate policy and state law. If the major laboratories successfully coordinate a slowdown, the rapid pace of AI product releases will decelerate, fundamentally altering the timeline for enterprise and consumer adoption.
The binding constraint for a coordinated pause in artificial intelligence development is antitrust law. For any single laboratory to intentionally delay the training of a frontier model, it must be mathematically certain that its competitors are doing the same; otherwise, it simply cedes the market. Currently, that condition does not hold. If the chief executives of OpenAI, Anthropic, and Google DeepMind were to agree on a synchronized development cap today, they would immediately face federal collusion charges.[2]
On September 11, 2026, OpenAI Chief Executive Officer Sam Altman addressed his staff, stating that the company is open to slowing the development of its most advanced systems. He explicitly noted his hope that rival laboratories would adopt a similar pacing. The internal communication marks a sharp pivot from the company's historical posture of aggressive scaling, reflecting a growing unease within the engineering ranks about the trajectory of the technology.[1]
The immediate hurdle to that hope is legal. OpenAI leadership is actively investigating whether an industry-wide agreement to throttle development would violate antitrust statutes. The paradox of the current regulatory environment is that laws designed to prevent price-fixing also prevent safety-driven development caps. Without a formal exemption from the Department of Justice, the major laboratories cannot legally coordinate a ceasefire in the compute arms race.[2]
The internal pressure to decelerate is no longer confined to theoretical white papers. It is manifesting in high-profile departures across the sector. Jacob Coxon, a researcher at Anthropic, resigned in early September 2026 in direct protest of the company's development velocity. His departure underscores a fracturing consensus within the laboratories building the models.[3][4]
Speaking to NPR on September 10, Coxon outlined the specific threat models driving the internal dissent. He argued that the race to improve models is "gambling with our lives," pointing to the risk of autonomous systems going rogue if capabilities continue to scale faster than alignment techniques. The timeline cited by departing researchers is aggressively short, with some warning of severe global disruptions by 2030.[4][7]
Anthropic itself provided evidence of these friction points, publishing a 154-page threat intelligence report on September 10. The document detailed the exact vectors of misuse currently observed in the wild. It confirmed that state-sponsored groups, spyware vendors, and propagandists have actively attempted to use Anthropic's models to design biological pathogens and surveil dissidents.[5]
Anthropic itself provided evidence of these friction points, publishing a 154-page threat intelligence report on September 10.
It is necessary to distinguish between what the models can actually do today and what the doom-oriented equivalent of marketing suggests they can do. The Anthropic report documents attempts by criminals to generate missile designs and pathogen blueprints. It does not state that the models successfully provided actionable, novel scientific data that the actors could not have found via traditional search engines. The capability overhang is real, but the immediate existential threat is often amplified by the very researchers building the tools.[5]
Platformer described this cultural pivot on September 11 as the "AI safety vibe shift." Existential risk, once a fringe obsession relegated to Bay Area rationalist message boards, has become the dominant conversational currency among the engineers actually writing the code. The shift in tone is forcing executive leadership to publicly address scenarios that were considered science fiction just 24 months ago.[3]
The mainstreaming of this timeline forces policymakers to react, even if the underlying science remains highly speculative. The New York Times dedicated a segment of its technology coverage on September 10 to dissecting whether artificial intelligence could genuinely cause human extinction by 2030. When the nation's paper of record treats a four-year extinction timeline as a valid premise for debate, the regulatory apparatus has no choice but to engage.[7]
While federal coordination remains legally ambiguous, state-level regulation has already shipped. On September 10, 2026, California's governor signed a suite of laws targeting the deployment of artificial intelligence chatbots and social media algorithms. The legislation bypasses the existential debates entirely, focusing instead on immediate, measurable harms.[6]
The California legislation is specifically aimed at protecting minors from algorithmic exploitation. The rules mandate strict guardrails on how generative systems interact with users under 18, forcing companies to implement age-verification and behavioral constraints at the application programming interface level. The laws face significant industry pushback, but they establish a compliance baseline that companies must now engineer around.[6]
Slowing down is not simply a matter of unplugging graphics processing units. The development of a frontier model requires orchestrating tens of thousands of chips over several months. A deliberate deceleration means altering the capital expenditure cycle. If OpenAI or Anthropic actually pauses a training run, they must still pay for the leased compute capacity, which runs into the hundreds of millions of dollars.
The next verifiable checkpoint is whether OpenAI formally petitions the Department of Justice for an antitrust exemption to discuss safety caps with Anthropic and Google. Until that legal pathway is cleared, any talk of an industry slowdown remains purely aspirational. The models will continue to scale, because the alternative is illegal.[1][2]
What to know
- OpenAI CEO Sam Altman informed staff the company is open to slowing frontier model development if competitors follow suit.
- Antitrust laws currently prevent AI laboratories from legally coordinating an industry-wide pause in model training.
- Former Anthropic researcher Jacob Coxon resigned in protest, citing the risk of human extinction by 2030 if scaling continues unchecked.
- Anthropic published a 154-page report detailing attempts by state-sponsored actors to use its models for pathogen and weapon design.
- California enacted new legislation imposing strict guardrails on how AI chatbots interact with minors.
Key terms
- Frontier Model
- The most capable, state-of-the-art artificial intelligence systems developed by leading laboratories, requiring massive computational resources to train.
- Alignment
- The field of research dedicated to ensuring artificial intelligence systems act in accordance with human values and do not pursue unintended or harmful goals.
- Capability Overhang
- The phenomenon where an AI model possesses latent abilities or knowledge that its creators are unaware of until discovered by users after deployment.
- Antitrust Law
- Statutes designed to promote market competition and prevent monopolies or collusion, which currently prohibit competing companies from agreeing to halt product development.
Reader questions
Why can't AI companies just agree to slow down?
Under current U.S. antitrust laws, competing companies that agree to halt or throttle product development can be prosecuted for collusion and anti-competitive behavior.
What did the Anthropic threat report reveal?
The 154-page report documented attempts by criminals and state actors to use Anthropic's models to design weapons and pathogens, though it noted these were attempts rather than successful novel discoveries.
What do the new California AI laws do?
Signed on September 10, 2026, the laws mandate strict behavioral constraints and age-verification requirements for AI chatbots to protect minors from algorithmic exploitation.
Sources
[1]BloombergCommercial AI CompaniesOpenAI Is Open to Slowing Cutting-Edge AI, CEO Sam Altman Tells Staff
Read on Bloomberg →
[2]WiredCommercial AI CompaniesOpenAI Wants to Know if an AI Industry Slowdown Would Even Be Legal
Read on Wired →
[3]PlatformerAI Safety ResearchersThe AI safety vibe shift
Read on Platformer →
[4]NPRAI Safety ResearchersFormer Anthropic researcher outlines threat of AI going rogue
Read on NPR →
[5]The GuardianCommercial AI CompaniesAnthropic details how criminals and scientists misuse its AI for cyberattacks and surveillance
Read on The Guardian →
[6]EngadgetState PolicymakersCA governor signs 'landmark' laws on youth use of social media and AI chatbots
Read on Engadget →
[7]NYTAI Safety ResearchersCould A.I. Really Kill All Humans?
Read on NYT →
Comments
More in Technology
See all →Platform Algorithms
The Short-Form Video Era Ends: TikTok and Instagram Algorithms Now Prioritize Long-Form Content and Watch Time
5 sources
Semiconductor Supply
The Evidence Pack: How the AI Memory Crisis is Reshaping the Consumer GPU Market
3 sources
Magnetoelectrics
Warwick Researchers Synthesize Magnetoelectric Material That Operates Near Room Temperature
3 sources
Propellant Physics
The Engineering Trade-off: Why the Aerospace Industry is Abandoning Hydrogen for Methane
3 sources
Every angle. Every day.
Get Technology stories with full source coverage and perspective breakdowns delivered to your inbox.




