Why a new report on Anthropic automating AI development is alarming researchers

Key Points:

• A report by AI safety group METR reveals that Anthropic's AI models are improving at roughly twice the speed compared to a year ago, largely due to AI-assisted research rather than just increased computing power, indicating rapid acceleration in AI development.
• Anthropic has developed several advanced internal AI models that remain unreleased, contributing to significant productivity gains, but the company has heavily redacted much of the METR report, limiting public insight into the specifics of their progress and associated risks.
• The findings raise concerns about how quickly AI could reach a stage of recursive self-improvement, where AI systems autonomously enhance themselves, potentially outpacing existing safety measures and prompting calls for greater transparency and independent oversight.
• Despite Anthropic's public advocacy for cautious AI development, the report highlights a perceived contradiction as the company accelerates AI progress while restricting information, fueling frustration among safety researchers and calls for stricter regulation.
• Policymakers and experts are increasingly urging for stronger governance, including potential bans on recursive self-improvement technologies, as AI development accelerates beyond current regulatory and societal readiness.