Anthropic Proposes Three Internal Metrics to Track Frontier AI Progress

Anthropic is sharing three measurements to help the public track artificial intelligence development inside frontier labs. [1] These metrics are designed to illuminate the extent to which AI systems are building the next version of themselves rather than being built by humans. [1] The second metric tracks the ability to oversee and intervene in the actions that AI agents take on Anthropic’s systems. [1] Finally, the company will measure the compute resources allocated to power the development of more capable models. [1]

The company states that AI systems are becoming exponentially more powerful and have begun automating more of the process of building themselves. [1] By better understanding this production process, Anthropic aims to correlate model inputs, such as compute, with model capabilities. [1]

Anthropic plans to embed independent third-party evaluators from multiple organizations directly into the company. [1] These third parties will be given access to internal systems, processes, and data that is comparable to the access held by internal risk assessment teams. [1] The external evaluators will verify safety practices, report incidents, and monitor the newly established metrics. [1] Anthropic notes that these numbers are expected to shift if there is coordination on pacing the frontier, a step called for by CEO Dario Amodei. [1]

By opening up its internal production metrics, Anthropic is attempting to set a new standard for transparency among frontier AI labs. The focus on measuring how much AI is being used to build the next generation of AI highlights the rapid approach of recursive self-improvement, a key concern in safety circles. While capabilities evaluations show what models can currently do, tracking the underlying inputs like compute allocation provides a leading indicator of future model power. This move aligns with CEO Dario Amodei's broader call for coordinated pacing, suggesting that internal metrics could form the basis for future regulatory thresholds or international agreements. What the sources don't address: How often these metrics will be publicly reported and whether competing labs will adopt the same measurement framework.