AI labs have staked their reputations on model benchmarking — the supposedly neutral and accurate system of evaluations that are designed to measure whether one AI product is better than another. However, the methods behind these evaluations are complicated and still maturing, said Irene Solaiman, Hugging Face’s chief policy officer, during a discussion at TechCrunch Disrupt last week. Testing needs to come together into a more consistent science if the industry is to rely on them in the future.
With that in mind, can we trust the current tests that are being released? Do you have the level of access that you need to the model itself — this is where open weights come in in a big way,” she added. Reproducibility in model evaluations has long remained one of the stickier aspects of benchmarking — and remains a topic of significant debate in the industry.
Outside of an individual test’s validity, there’s also the added question of how that test compares to other evaluations. Hugging Face was famously the target of one of those incidents. Instead of active government regulation to identify and mitigate model safety concerns, labs have pushed for the third-party solution.
However, doubts remain over how neutral or effective those evaluators will actually be. Anthropic’s decision to hire Accenture as its first evaluator spurred a good amount of backlash, with some critics mocking the idea that the consulting firm was qualified to do the task at hand. Evaluations are now moving from merely a test of the model itself to a test of the entire system around the model — including the harness and agent.
The harness — or the tool the user is using to access the model — specifically, has a lot to do with a model’s performance, and a weaker model with a better harness could easily out perform a frontier model, Soleiman shared. In other words: the evaluation space is a bit of a free-for-all at the moment, and if benchmarking is going to continue to be seen as a useful tool for industry leaders, a more uniform, industry-wide approach — where everyone is on the same page — may be necessary.
Extract — continue reading at the source.
