New research published by Nvidia argues that when you ask an AI to carry out long-horizon tasks, the harness matters far more than the underlying model. A harness is the software wrapper around a model — the tools, memory management and rules that turn a raw model into something able to act on its own.

The short version: using a custom harness tuned for memory handling and including a "supervisor" component, researchers got Claude Opus 5 to a perfect score on ARC-AGI-3. That benchmark is a set of 2D games with no instructions, where the model has to work out how to play and win much as a human would. Without the harness, the same model scored 30 percent — which was still the top result among all models tested.

An agent is not just a model

Adel El Hallak, vice president of product in Nvidia's AI unit, puts the distinction this way: "Generally speaking, the world interprets an agent almost as an API of the model." An agent is more than that. In his account it is the model, plus the scaffolding around it, plus the set of tools it uses, plus the runtime and the skills and libraries it is given access to.

Long-horizon tasks are those requiring many decisions strung together, sometimes over days. That is a different thing from a model producing a response to a prompt, and getting an AI to hold course through such work is one of the hardest open problems in agentic research.

Several results show the scale of the problem:

  • Microsoft research from April — tested 19 language models on long-horizon document editing; every one of them, frontier models included, filled the documents with errors.
  • Models chaining decisions on their own — have been caught deleting users' files and even entire databases.
  • In pursuit of an objective — they have turned to behavior ranging from collusion to hacking.

The choice of benchmark carries a message

Picking this particular benchmark says something on its own. A perfect score means the model can beat the games as well as a human can. Rival lab OpenAI was sufficiently bothered by its models' results here — below 10 percent — that it ran its own study. Like Nvidia, OpenAI found that adjusting two harness settings tripled its scores. But none of its models came close to a perfect result.

What Nvidia's team added was a "supervisor" component. As El Hallak describes it: "The more interesting part was introducing a supervising agent in addition to your main agent that's doing the work." That component prods the agent when it drifts off direction, starts exploring a path likely to dead-end, or re-explores a path it has already tried — behaving, in his words, almost like a CEO.

A finding, not a product

The supervising agent is not a new idea. But most agent users today rely on a single layer for their harness. Nvidia's researchers named their souped-up version Agentic Variation Operators. It is worth stressing this is not a new Nvidia product; the company produces the building blocks for harnesses under the Nemo brand, and much of that is openly available.

The result strengthens a growing body of evidence that model choice is far from the only factor in agentic performance. Databricks research in July showed that the harness, more than the model, dramatically affects cost. Picking the same model with a different harness changes both the outcome and the bill.