Tech • AI • Robotics • Game
Anthropic’s New Model 4 Beats Opus 5.5 (ASI in 2 Years)
9.4/10TL;DR
A high-access evaluation inside Anthropic found evidence that progress in the company’s frontier AI models accelerated to about 1.9 times its earlier rate after October 2025, intensifying concerns that AI could soon speed up AI research itself.
KEY POINTS
Internal review found a sharp capability acceleration
From August 12 to October 29, researchers at METR conducted an 11-week evaluation inside Anthropic, examining whether AI is beginning to accelerate AI development. Using Anthropic’s internal capability index, AECI, they found model progress rose from about 15 points per year before October 2025 to roughly 29 points per year afterward, a 1.9x increase with a 95% confidence interval of 1.5 to 2.8.
An unreleased model outperformed current public systems
The strongest internal model tested, identified only as Model 4, scored 178.6 on AECI. That was above Opus 5.5 at 174.5 and Mythos 5.1 at 172.5. The evaluation said Model 4 scored above 50% on all 15 benchmarks it faced, raising concerns that existing tests may be nearing saturation for top-end models.
Researchers concluded the curve likely bent, not merely jumped
Anthropic had previously suggested that a late-2025 leap might be a one-time jump rather than a lasting change in trend. METR tested both explanations using earlier data, then checked which better predicted later systems including Model 2, Model 3, Opus 5.5, and Model 4. The lasting slope-change model missed by 1.9 AECI points, versus 3.9 for the one-time-jump model, and won in 81% of bootstrap draws.
The result held under most robustness checks
Swapping benchmarks, removing vision-heavy tests, or dropping tasks once models exceeded 85% performance still produced acceleration estimates around 1.9 to 2.2. The largest variation came from the statistical link function used to map benchmark results to capability growth. Depending on that assumption, the estimate ranged from 1.1 to 2.9, although the standard logistic specification remained the central estimate because alternative fits performed similarly.
Anthropic may be moving faster than the broader frontier
When researchers plotted frontier models across multiple labs, the overall slope rose from 17 to 24 AECI points per year, implying a smaller 1.4x acceleration. That leaves two possibilities: Anthropic was lagging in 2025, making the internal acceleration look larger, or competitors were more benchmark-optimized earlier, making Anthropic’s 1.9x figure closer to the real underlying trend.
Much of the recent acceleration may come from AI itself
The evaluation judged that faster progress was probably not mainly due to increased training compute. Its best estimate was that by October 2026, roughly 80% of the acceleration came from AI assistance, with the rest attributable to staffing growth and more inference compute. The late-2025 bump, however, was thought to be driven more by non-AI factors, with AI’s share rising afterward.
Internal staff reports described major productivity gains
Anthropic staff polling suggested perceived uplift from internal models rose from about 1.2x in early 2025 to roughly 5x by June, then 12x by October. Researchers treated that as loose evidence because self-reports can exaggerate impact, but interviews were strikingly consistent: some staff said coding and experiment execution had shifted from a bottleneck to something close to abundant, with agent systems able to run many parallel research attempts.
AI is not yet fully autonomous, but warning signs are growing
The review did not conclude that AI can already make fully autonomous, high-significance contributions to the most important parts of model training. Still, internal interviews and task-allocation data pointed to meaningful early signs of that direction. Researchers also reported finding no obvious algorithmic, compute, or data bottlenecks likely to slow progress over the next 12 months, increasing concern that acceleration could continue.
The public evidence was heavily redacted
Anthropic retained unilateral redaction rights over the report and removed what METR described as much of the most informative evidence, citing concerns that disclosure could aid rival AI developers. Redacted material reportedly included algorithmic progress estimates, bottleneck analysis, future-model forecasts, and evidence suggesting the acceleration may exceed 2x and still be rising. A planned check of how Anthropic divides compute between capabilities and safety work was also dropped mid-process.
CONCLUSION
The findings suggest Anthropic may be entering a phase where AI systems materially accelerate the creation of stronger AI systems. If that trend continues, the main policy and safety challenge will be the shrinking time available to evaluate each capability jump before the next one arrives.
Ask a question