METR, a Berkeley-based nonprofit founded by former OpenAI researcher Beth Barnes, has become a prominent independent evaluator of artificial intelligence models, working with major AI labs and governments to assess the risks of cutting-edge technology. The organization now finds itself at the center of an intensifying public debate over whether AI development is moving too fast for safety measures to keep pace.
This week, Anthropic researcher Joe Benton announced he had left the company to join METR, writing that he worries about “extinction-level risks” from AI technology. His announcement followed a viral post by another researcher, Jacob Coxon, who said he was departing Anthropic and accused leading AI labs of “gambling with our lives.”
METR, formally known as Model Evaluation and Threat Research, was formed in 2022. It has worked closely with OpenAI, Anthropic, Google, and Meta to study the rapidly changing capabilities of AI systems. This summer, researchers from the nonprofit investigated a security incident at OpenAI and now plan to examine security issues at Anthropic, relying on internal access granted by each company.
Mission and Independence
“The public should know whether AI development is headed down a dangerous path,” said Jasmine Dhaliwal, a member of METR’s policy staff, on Friday. “That is core to our mission: to provide independent, scientific assessment of AI capabilities, alignment, and control measures.”
Barnes left OpenAI to start the nonprofit in 2022. She has described an AI safety field that is severely constrained by its size, noting that even METR’s high salaries, which reach $503,000 according to current job postings, do not resolve the organization’s talent shortage.
“Ideally, we’d like to scale really large, but in practice, we’ve been able to fundraise as much as we need, and the bottleneck is much more talent,” Barnes said.
Benton is not the only researcher to move from a major technology company to the Berkeley organization. Josh Engels, an AI safety researcher, recently joined METR from Google DeepMind.
Growing Demand for AI Evaluation
The security incident announced by OpenAI and Hugging Face in July prompted new questions about how to evaluate and control AI systems. More than 1,300 frontier lab employees signed a letter in July warning that AI development could outpace control measures. In the last month, both OpenAI and Anthropic said they had temporarily paused training to better understand their new AI models.
While the Hugging Face incident surprised much of the corporate world, METR’s team was far less taken aback. Governments, companies, and even religious groups routinely ask the nonprofit for help understanding and testing the technology’s progress.
Chris Painter, METR’s president, describes the lab as “humanity’s preparedness team,” a reference to the preparedness teams at OpenAI and Anthropic that report safety threats to their chief executives.
“We aren’t really accountable to anyone other than the public and the public’s well-being,” Painter said.
As AI labs release new models, METR measures the likelihood that they can complete tasks of increasing duration. These measurements form the nonprofit’s most widely cited offering: a chart showing that over the last six years, AI capabilities have doubled approximately every seven months.
Barnes emphasized the importance of METR’s independence so that it can produce research not tied to any company’s goals. While METR does not accept money from frontier AI labs or their employees, it accepts compute grants and works with the companies to analyze unreleased models.
Staff members have analyzed labs’ safety practices, written about AI’s effect on software engineers’ productivity, and studied the merits of using AI models to monitor other AI models. Barnes said she would like to expand into predicting the next levels of AI capabilities and how those changes could accelerate AI development.
Neev Parikh, a METR researcher, said talent constraints limit the number of questions researchers can address about AI models’ internal reasoning.
“There’s a dearth of people,” Parikh said. “I would happily see the field expand 10x.”
Looking Ahead
METR plans to continue its evaluation work with Anthropic and other AI laboratories, according to the organization. The nonprofit’s findings are expected to inform ongoing policy discussions about AI safety and oversight. As AI companies release more advanced models, demand for independent assessment is likely to grow, though METR has said its ability to respond depends on recruiting additional research talent.
