OpenAI and Anthropic Signal a Shift Toward Deliberate AI Scaling as Safety Concerns Mount
Leading artificial intelligence laboratories are exploring mechanisms to coordinate a slowdown in cutting-edge model development. The shift follows internal resignations over existential risks and new regulatory frameworks targeting autonomous systems.
By Wei Zhang
Share this story
The binding constraint for a coordinated pause in artificial intelligence development is antitrust law. For any single laboratory to intentionally delay the training of a frontier model, it must be mathematically certain that its competitors are doing the same; otherwise, it simply cedes the market. Currently, that condition does not hold. If the chief executives of OpenAI, Anthropic, and Google DeepMind were to agree on a synchronized development cap today, they would immediately face federal collusion charges.[2]
On September 11, 2026, OpenAI Chief Executive Officer Sam Altman addressed his staff, stating that the company is open to slowing the development of its most advanced systems. He explicitly noted his hope that rival laboratories would adopt a similar pacing. The internal communication marks a sharp pivot from the company's historical posture of aggressive scaling, reflecting a growing unease within the engineering ranks about the trajectory of the technology.[1]
The immediate hurdle to that hope is legal. OpenAI leadership is actively investigating whether an industry-wide agreement to throttle development would violate antitrust statutes. The paradox of the current regulatory environment is that laws designed to prevent price-fixing also prevent safety-driven development caps. Without a formal exemption from the Department of Justice, the major laboratories cannot legally coordinate a ceasefire in the compute arms race.[2]
The internal pressure to decelerate is no longer confined to theoretical white papers. It is manifesting in high-profile departures across the sector. Jacob Coxon, a researcher at Anthropic, resigned in early September 2026 in direct protest of the company's development velocity. His departure underscores a fracturing consensus within the laboratories building the models.[3][4]
Speaking to NPR on September 10, Coxon outlined the specific threat models driving the internal dissent. He argued that the race to improve models is "gambling with our lives," pointing to the risk of autonomous systems going rogue if capabilities continue to scale faster than alignment techniques. The timeline cited by departing researchers is aggressively short, with some warning of severe global disruptions by 2030.[4][7]
Anthropic itself provided evidence of these friction points, publishing a 154-page threat intelligence report on September 10. The document detailed the exact vectors of misuse currently observed in the wild. It confirmed that state-sponsored groups, spyware vendors, and propagandists have actively attempted to use Anthropic's models to design biological pathogens and surveil dissidents.[5]
It is necessary to distinguish between what the models can actually do today and what the doom-oriented equivalent of marketing suggests they can do. The Anthropic report documents attempts by criminals to generate missile designs and pathogen blueprints. It does not state that the models successfully provided actionable, novel scientific data that the actors could not have found via traditional search engines. The capability overhang is real, but the immediate existential threat is often amplified by the very researchers building the tools.[5]
Platformer described this cultural pivot on September 11 as the "AI safety vibe shift." Existential risk, once a fringe obsession relegated to Bay Area rationalist message boards, has become the dominant conversational currency among the engineers actually writing the code. The shift in tone is forcing executive leadership to publicly address scenarios that were considered science fiction just 24 months ago.[3]
The mainstreaming of this timeline forces policymakers to react, even if the underlying science remains highly speculative. The New York Times dedicated a segment of its technology coverage on September 10 to dissecting whether artificial intelligence could genuinely cause human extinction by 2030. When the nation's paper of record treats a four-year extinction timeline as a valid premise for debate, the regulatory apparatus has no choice but to engage.[7]
While federal coordination remains legally ambiguous, state-level regulation has already shipped. On September 10, 2026, California's governor signed a suite of laws targeting the deployment of artificial intelligence chatbots and social media algorithms. The legislation bypasses the existential debates entirely, focusing instead on immediate, measurable harms.[6]
The California legislation is specifically aimed at protecting minors from algorithmic exploitation. The rules mandate strict guardrails on how generative systems interact with users under 18, forcing companies to implement age-verification and behavioral constraints at the application programming interface level. The laws face significant industry pushback, but they establish a compliance baseline that companies must now engineer around.[6]
Slowing down is not simply a matter of unplugging graphics processing units. The development of a frontier model requires orchestrating tens of thousands of chips over several months. A deliberate deceleration means altering the capital expenditure cycle. If OpenAI or Anthropic actually pauses a training run, they must still pay for the leased compute capacity, which runs into the hundreds of millions of dollars.
The next verifiable checkpoint is whether OpenAI formally petitions the Department of Justice for an antitrust exemption to discuss safety caps with Anthropic and Google. Until that legal pathway is cleared, any talk of an industry slowdown remains purely aspirational. The models will continue to scale, because the alternative is illegal.[1][2]
Key points
- OpenAI CEO Sam Altman informed staff the company is open to slowing frontier model development if competitors follow suit.
- Antitrust laws currently prevent AI laboratories from legally coordinating an industry-wide pause in model training.
- Former Anthropic researcher Jacob Coxon resigned in protest, citing the risk of human extinction by 2030 if scaling continues unchecked.
- Anthropic published a 154-page report detailing attempts by state-sponsored actors to use its models for pathogen and weapon design.
What we don’t know
- Whether the Department of Justice or the Federal Trade Commission would consider granting an antitrust exemption for AI safety coordination.
- How much of the current slowdown rhetoric is genuine policy versus strategic signaling to preempt harsher federal regulation.
- The exact threshold of capability that would trigger a mandatory, government-enforced pause in model training.
How we got here
Early Sept 2026
Jacob Coxon resigns from Anthropic in protest of the company's development velocity and safety posture.
Sept 10, 2026
Anthropic releases a 154-page threat intelligence report detailing misuse attempts by state-sponsored actors.
Sept 10, 2026
California signs landmark legislation regulating AI chatbot interactions with minors.
Sept 11, 2026
OpenAI CEO Sam Altman informs staff the company is open to slowing frontier model development if competitors follow suit.
- Commercial AI Companies
- Express a desire for coordinated safety slowdowns but remain bound by market competition and antitrust laws that force continued scaling.
- AI Safety Researchers
- Argue that the current scaling velocity poses an unacceptable existential risk and that development must be paused until alignment techniques catch up.
- State Policymakers
- Focus on immediate, tangible harms like youth exploitation and algorithmic bias, deploying legislative guardrails while federal coordination lags.
Perspectives this story doesn't cover
- Open-Source AI Developers
- Enterprise AI Adopters
Sources
[1]BloombergCommercial AI CompaniesOpenAI Is Open to Slowing Cutting-Edge AI, CEO Sam Altman Tells Staff
Read on Bloomberg →
[2]WiredCommercial AI CompaniesOpenAI Wants to Know if an AI Industry Slowdown Would Even Be Legal
Read on Wired →
[3]PlatformerAI Safety ResearchersThe AI safety vibe shift
Read on Platformer →
[4]NPRAI Safety ResearchersFormer Anthropic researcher outlines threat of AI going rogue
Read on NPR →
[5]The GuardianCommercial AI CompaniesAnthropic details how criminals and scientists misuse its AI for cyberattacks and surveillance
Read on The Guardian →
[6]EngadgetState PolicymakersCA governor signs 'landmark' laws on youth use of social media and AI chatbots
Read on Engadget →
[7]NYTAI Safety ResearchersCould A.I. Really Kill All Humans?
Read on NYT →
More in Technology
See all →Open-Weight AI
Moonshot AI Restricts Hacking Capabilities in Kimi K3.1 Pro Ahead of Open-Weight Release
3 sources
Database Architecture
How Sequential Timestamps in UUIDv7 Prevent B-Tree Page Splits and Buffer Bloat
6 sources
Network Protocols
Two Maximum Segment Lifetimes: Why TCP Holds Closing Sockets in TIME_WAIT to Drain Wandering Packets
9 sources
Edge Security
Fortinet Warns of Unpatched FortiMail Zero-Day Under Active Exploitation
7 sources
Comments
Every angle. Every day.
Get Technology stories with full source coverage and perspective breakdowns, free every day.




