Embedded AI evaluators won’t be an easy solution, Hugging Face’s policy chief says

Model benchmarking faces a reproducibility crisis, Irene Solaiman, Hugging Face’s chief policy officer, told TechCrunch Disrupt. Labs stake reputations on evaluations supposed to measure whether one AI product outperforms another, but the methodology remains unsettled.

Trust and reproducibility in doubt

"Trust is such a weighted term," Solaiman said. "In terms of whether we can trust results, one big part is can you reproduce the results of an evaluation? Do you have the level of access that you need to the model itself — this is where open weights come in in a big way."

Reproducibility has long been a stickier aspect of benchmarking. Beyond individual test validity, there's the question of how one evaluation compares to another. "They are an evolving science — where we don't know how to meaningfully compare them to each other," Solaiman said. "They start to get ineffective relatively quickly, and we don't have as much of a thriving ecosystem as we need."

Solaiman also highlighted the emerging market for embedded evaluators — supposedly neutral third-party observers placed inside AI organizations. The idea gained traction after safety incidents, including one involving Hugging Face. Anthropic's decision to hire Accenture as its first evaluator drew backlash, with critics questioning the consulting firm's qualifications.

From model to system evaluation

Evaluations are moving beyond testing the model itself. They now test the entire system around the model, including the harness — the tool a user employs to access the model. Solaiman noted a weaker model with a better harness could outperform a frontier model.

"Agentic evals are really hard and really expensive," she said. "We have just more evaluations for models. We don't have as many evaluations for harnesses, and we are developing better evaluations for the holistic agent — because that's what a lot of folks are getting value out of."

In other words, the evaluation space remains a free-for-all. If benchmarking is to remain a useful tool for industry leaders, a more uniform, industry-wide approach — where everyone is on the same page — may be necessary.