Anthropic has introduced three new metrics designed to assist artificial intelligence companies in monitoring the pace of their technological development. The announcement comes just days after CEO Dario Amodei sparked industry discussion by advocating for a coordinated slowdown in AI progress.
In a blog post published on Thursday, the company detailed its methodology for measuring AI-led research and development, oversight of AI agents, and compute allocation. Anthropic emphasized that sharing these methods is intended to encourage other organizations in the sector to adopt similar transparency measures.
The move builds upon a three-step slowdown plan Amodei released over the weekend. While that plan outlined a vision for tempering the speed of model capability improvements without sacrificing commercial advantage or U.S. leadership in AI, it lacked specific details on practical implementation.
“As the world considers pacing the frontier, we should do everything possible to minimize the gap between what frontier labs know and what the public knows,” Anthropic stated. “This means better measuring the development of AI, reporting on it publicly, and giving society an opportunity to decide how to use this information.”
Amodei’s proposal has garnered support from prominent industry figures, including OpenAI CEO Sam Altman, SpaceX CEO Elon Musk, and Google DeepMind Chair Demis Hassabis. This backing follows urgent warnings from researchers regarding the potential for AI to cause significant harm.
Anthropic outlined the results of its initial measurements using the new metrics:
- First, the company determined that its Claude models are not operating fully autonomously within any subset of the measured research and development work.
- Second, regarding oversight, Anthropic found that approximately 30,000 agents were conducting research and engineering tasks on its most-used internal platform at any given time, with systems in place to intervene in their actions.
- Third, analyzing a compute snapshot from July 13 to July 20, Anthropic reported that roughly 6% of total compute dedicated to AI research and development was allocated toward safety. When focusing specifically on “AI-driven” research and development, that figure rose to approximately 12%.
The company noted that these metrics are designed to illustrate how models are constructed, serving as a complement to capability evaluations that demonstrate what models can achieve. Anthropic believes that together, these data points provide third parties with a starting point for assessing the pace of AI development.
“We hope to model that transparency by releasing these measurements, and we’ll continue to do so,” Anthropic added.
Does anyone else think this is just clever marketing? Anthropic looks good while pushing the narrative.
Finally, concrete numbers instead of just scary predictions. This gives the public something tangible to evaluate.
Wait, only 6 percent of compute for safety? That feels dangerously low given how fast these models are evolving.
Transparency is good, but will competitors actually adopt these metrics? Skeptical they’ll slow down without regulation.