A resignation that became a public alarm
Jacob Coxon, a former Anthropic researcher who said he spent three years working on pre-training at OpenAI and Anthropic, has put a stark version of the artificial intelligence safety debate into public view: the people closest to frontier AI systems, he claimed, privately fear the technology could become deadly before the end of the decade . His resignation post, published on X and reported by multiple outlets on September 9, 2026, accused the two leading labs of “racing” toward self-improving superintelligence without acting responsibly .
The headline phrase is deliberately alarming, but the underlying dispute is narrower and more technical than a simple doomsday claim. Coxon is not saying a current chatbot is about to attack humanity. He is warning that the trajectory of frontier AI research could produce systems with capabilities beyond human control, while the methods used to monitor, align and govern those systems remain immature .
That distinction matters. The current controversy is about timelines, incentives and confidence. Coxon’s claim is that insiders do not merely worry about misuse, fraud or job displacement; they worry about existential risk from future systems that could exceed human ability to understand or restrain them .
What Coxon said
Coxon’s central allegation is that Anthropic and OpenAI are moving toward “self-improving superintelligence” while “gambling” with public safety . He described superintelligent systems as potentially able to outperform humans across domains, including cybersecurity and scientific research, and suggested that progress is not slowing .
The most widely quoted line from Coxon’s post was that people building AI “earnestly believe” it could “kill us all” by the end of the decade . According to reporting from The Independent, Coxon also argued that senior researchers and executives often soften their language in public while expressing sharper fears in private .
Coxon’s role is important to the story because he is not an outside critic with no connection to the labs. ABC reported that he worked on pre-training, the stage of model development that helps determine a system’s broad capabilities, and that he had worked at both OpenAI and Anthropic over the previous three years . That does not make his forecast correct, but it gives his warning weight inside a debate often divided between industry optimism and outside skepticism.
Other Anthropic researchers amplified the concern
The story escalated because Coxon’s warning was not isolated. Evan Hubinger, described in reports as Anthropic’s alignment-science lead, publicly backed the broad concern and said he personally put the risk of AI killing all humans within the next decade above 10 percent . Hubinger also said he believed Anthropic was trying its best, while adding that the company did not yet have a plan to solve alignment for superintelligence and was not clearly on track .
That is the most significant part of the episode. Anthropic is often presented as one of the frontier labs most focused on safety, having been founded by former OpenAI employees who were concerned about responsible development. When a safety-focused researcher at such a company says the lab lacks a clear plan for superintelligence alignment, the warning lands differently from a generic anti-AI critique .
Samuel Marks, another Anthropic researcher, also weighed in in a personal capacity, according to The New Republic and The Independent . Marks reportedly argued that AI developers believe their technology could cause human extinction or similarly severe outcomes, and that the risk could arrive within the next few years . He also connected the problem to incentives: companies keep building because competitors may build first, and because commercial and strategic pressures reward speed .
What “superintelligence” means here
In this debate, “superintelligence” refers to a system that surpasses human performance across a wide range of intellectual tasks, not just one benchmark or one professional exam . The fear is not that a model becomes dangerous because it has emotions or consciousness. The fear is that a highly capable system, given goals, tools, autonomy and access to real-world systems, could pursue objectives in ways humans did not intend and cannot stop.
The alignment problem is the technical and governance challenge of ensuring that advanced AI systems reliably act in accordance with human interests. ABC described it as the risk that a superintelligent system may not share human values . In practical terms, that can include deception, power-seeking behavior, cyber misuse, biological misuse, uncontrolled replication or strategic manipulation of human institutions.
One reason the Coxon warning is attracting attention is that it combines a near-term timeline with claims about insider belief. “By 2030” is not a distant science-fiction horizon. It is less than four years from the September 2026 reporting window. That compressed timeline turns abstract safety concerns into an immediate policy and engineering question: what should labs, governments and the public demand before more capable systems are trained and deployed?
The counterpoint: present systems are not superintelligence
The reporting also contains an important caveat. KPTV, citing CNN Newsource, noted that Hubinger later clarified his comments by pointing to Anthropic’s own risk report: the report acknowledges the concern but also says present AI has little chance of gaining that level of power . That means the dispute is not about whether today’s consumer AI systems are already existentially dangerous in the strong sense. It is about whether the industry is approaching a threshold faster than safety work can handle.
This caveat should shape how the public reads the warning. Coxon and the Anthropic researchers are describing a possible failure mode of future frontier AI, not documenting an ongoing extinction-level incident. Their claim is probabilistic, not a prediction of certain catastrophe. Hubinger’s reported figure of above 10 percent is still a minority probability, but in ordinary risk management a double-digit chance of human extinction would be treated as intolerable .
The challenge is that AI risk is not easily tested in the way a bridge, aircraft or medical device can be tested before use. If a system’s most dangerous capabilities emerge only at the frontier, and if labs cannot fully predict those capabilities in advance, then waiting for proof may mean waiting too long. That is the logic behind calls for stronger monitoring, pacing and government oversight.
The industry-wide context
The resignation comes amid a broader pattern of AI workers going public with safety concerns. KPTV reported that AI safety worries among workers in the field are becoming more common, and said OpenAI’s chief scientist had warned two days earlier that AI capabilities are advancing faster than researchers’ ability to reliably monitor and control them .
ABC connected Coxon’s resignation to a July open letter signed by more than 1,300 employees of frontier AI companies, who warned of a real risk that capability development could accelerate beyond the ability to understand or control the resulting systems . The letter called for the U.S. government to support an international effort to develop tools that could deliberately pace automated AI development .
That “pacing” language is important. It is not simply a call to ban AI. It reflects a view that governments and labs may need mechanisms to slow or pause frontier development when monitoring, evaluation or containment methods lag behind model capabilities. Supporters see that as a safety brake. Critics may see it as vague, unenforceable or vulnerable to geopolitical competition.
Why Anthropic is central to the story
Anthropic’s presence at the center of this controversy is notable because the company has built much of its public identity around safety, alignment and constitutional AI. Coxon’s criticism does not accuse Anthropic of ignoring safety entirely. Instead, he and the researchers responding to him appear to argue that even a safety-conscious lab can become trapped by the race dynamics of the industry .
That is a sharper critique than simply saying “AI companies are reckless.” It suggests a structural problem: if each lab believes another company or country will build more powerful systems anyway, then each has an incentive to continue, even while believing the collective race increases danger. Marks’ reported comments point directly at that dynamic, saying developers are driven forward by commercial incentives and fear of being beaten by less responsible actors .
This is the familiar “race to the bottom” problem in safety-critical technology. The faster the competition, the harder it becomes to impose voluntary restraint. If companies police themselves, safety commitments depend on leadership judgment, investor tolerance, employee pressure and public scrutiny. If governments intervene, they face the problem of defining what counts as dangerous capability without freezing beneficial research.
What governments are being asked to do
The immediate policy demand emerging from this episode is not a single bill or rule, but a stronger public role in AI governance. The July employee letter cited by ABC asked for an international effort to manage the pace of advanced AI, while KPTV reported that nearly 1,400 AI company employees had urged U.S. regulation to slow the pace and rein in Big Tech .
The difficulty is that frontier AI is both commercially valuable and strategically sensitive. Governments want domestic companies to lead, companies want to ship products, and researchers want access to the compute and data required to test safety ideas. A hard moratorium may be politically unrealistic; purely voluntary standards may be too weak. The contested middle ground includes mandatory evaluations, incident reporting, licensing for the largest training runs, shared safety thresholds, compute governance and international coordination.
Coxon’s resignation adds urgency to that debate because it comes from inside the very labs expected to manage the risk. The public question is no longer whether outside critics can imagine catastrophic scenarios. It is whether the industry’s own safety-minded employees believe the control problem is being solved quickly enough. According to Coxon, Hubinger and Marks, the answer is no .
What to watch next
The next phase of the story will turn on responses from the companies and from policymakers. The Independent reported that it contacted Anthropic and Coxon for comment, and that OpenAI declined to comment on Coxon’s post while pointing to recent updates on its safety and alignment work . That leaves a gap between general assurances and the specific insider claim that no adequate plan exists for superintelligence.
For readers, the useful takeaway is neither panic nor dismissal. Coxon’s warning is an insider alarm about a potential future system, not proof that catastrophe is inevitable. But the severity of the claimed risk, the short timeline and the corroborating concern from other Anthropic researchers make it difficult to treat the episode as ordinary workplace dissent.
The most important question is now measurable: can frontier labs demonstrate, before building more powerful systems, that they can monitor, evaluate and control those systems under adversarial conditions? If they cannot, the “kill us all by 2030” warning will remain more than a provocative quote. It will be a test of whether the AI industry can slow down long enough to make its own safety case credible.