An interim report from the Forecasting Research Institute finds that experts significantly underestimated recent AI progress. But there are areas where their forecasts ran too high, not too low.
Model launches arrive weekly, each with its own chart. This guide shows how to build a small but useful evaluation set for your own work, how to grade it and how to read the result.
Anthropic released Claude Opus 5.5, the first model in the Claude 5.5 family. The company says it performs at Fable 5.1 level on most work and costs 40% less to run than Opus 5 at default settings.
OpenAI announced GPT-6 Sol and Luna. Both run at half the price of their predecessors, with no large jump in capability. The company's pitch is price-performance.
xAI released Grok 4.7, its new flagship for coding and agentic work, at the same price as Grok 4.6. The company's chart shows a big jump; an independent index places it mid-pack.
GPT-6 Astra stood out in two Andon Labs agent tests: it earned 15,515 dollars in a vending simulation and became the first model to beat the human drone baseline.
They can neither predict how long a task takes nor tell how long they have worked. They also overrate their own output by 20 points on average.
Benchmark questions leaking into training data make scores meaningless. DeepMind is using cryptography to close both sides off from each other.
Methods borrowed from psychological testing show that reducing AI safety benchmarks to a single number is misleading.
Artificial Analysis has launched Optima, a platform letting users build custom benchmarks from their own data and workflows, comparing models on cost and time as well as quality.
Microsoft Research's MindTopo benchmark measures whether multimodal models understand relations like connectivity and knottedness. They do well on static images, not on action.
Moonshot AI's PerceptionBench isolates the visual perception of multimodal models from reasoning. None of the 16 frontier models tested reached 60 percent accuracy.