A new study finds that popular coding assistants cannot predict how long a task will take, and cannot reliably tell how long they have already been working. For long-running jobs, that is a problem.

The work was done by two independent AI researchers as part of the MATS research program. The pair tested Anthropic's Claude Code and OpenAI's Codex on their sense of time.

The method is simple: before each coding task the agents had to estimate how long they would need. Then they solved the task and, looking back, reported how much time had passed. The test material came from 200 tasks in a collection called ProgramBench, plus the researchers' own suite of 18 benchmarks.

The results

  • Consistent overestimation. On ProgramBench both models mostly guessed around 90 minutes, no matter the difficulty.
  • Large error. In the second round Claude was off by three times on average, Codex by six to ten times.
  • Worst on short tasks. Only in the multi-hour range did some predictions come close to reality.

Same model, different harness, different behaviour

The results shift with the software setup the models run in. Claude Code keeps working until it thinks the task is done — a median of about 90 minutes. Codex stops after roughly half an hour, almost regardless of the task.

According to the study, the same language model takes 2.5 times more steps in Claude Code than in Codex on average. Runtime depends not on the model but heavily on the surrounding software, the harness.

That finding matters on its own: how long an agent works is not a property of the agent but of the system it is placed in.

It misjudges its own work too

The agents are just as unreliable at judging the quality of their own output. The older models, Opus 4.8 and GPT-5.5, overrated their results by 20 points on average and handed themselves high marks even on failed tasks.

In one case both figured their work was about 70 percent successful. The actual scores were 7 and 14.5 percent.

In practice this means calling an agent "fast" or "slow" is the wrong frame. The same model works two and a half times longer, and tries more, inside a different tool. What matters when choosing a tool is not the model's name but how much effort that model is allowed to spend.

Why it matters, and the fix

The researchers say the ability to self-assess is critical. For an agent to work reliably on tasks that run for hours, it has to follow instructions like "iterate on this task for two hours." An agent that constantly misjudges the time is hard to control.

The fix turned out to be surprisingly simple: when the agents were given a tool that reports elapsed time, they got it right almost every time.

The lesson is about design more than model capability. The model does not feel time because it has no sense with which to perceive it; asked, it guesses. Give it a clock and the problem disappears. For anyone building long-running agents, that is a correction available today without waiting for a better model.