What are benchmark scores actually measuring?
Ai2's BenchMIRT method examines benchmarks at the level of individual questions, and shows that a single score often mixes together more than one capability.
Research
models
Fable 5.1's “45 percent cheaper” claim is disputed
Anthropic announced its new models with a cache discount. Artificial Analysis, which took part in pre-release testing, says the saving does not hold in every scenario.
Models