Picking up from the previous question

IBM Research's previous work compared ALTK-Evolve with ACE and showed that how an agent's self-distilled guidelines are delivered drives both accuracy and cost: a few retrieved per task, or the whole set injected?

The new work steps back to the question that comes before it: how much should you give it?

The expectation and the result

Equipping an agent with agentic memory sounds simple: distil lessons from its past work, put them back in context, and more experience should mean better performance. The finding is that it does not always work that way.

When the evaluation was scaled to eight models — from a 30B dense model to frontier proprietary systems — one result stood out. In the authors' own words: agentic memory is not a feature you switch on; it's a dose you calibrate to the model.

The dose differs by tier

The picture the work summarises:

  • Strong models with headroom want the full guideline set; they can take everything and filter it themselves.
  • Weaker models do best with a compact core plus per-task retrieval; more than that distracts them.
  • Saturated models show no measurable gain — adding memory does not change performance.

How ALTK-Evolve works is unchanged: reusable guidelines are distilled from the agent's past trajectories and injected back at inference time. No weight updates, no human annotation.

Why it matters

The finding breaks a common assumption in agent design. Memory is usually thought of as a switch: turn it on and things improve. The work proposes a dial instead, and the right position of that dial differs by model.

The practical consequence: measuring an agent setup on one model and carrying the same settings to another is misleading. The same amount of memory can produce a gain on one model and a loss on another. That the evaluation spans eight models is exactly why it matters — a measurement on a single model hides the finding itself.

The absence of gain on saturated models is also notable. It suggests memory has a ceiling: if the model can already do the task, past lessons have nothing to add. Memory, in other words, does not add a capability; it compensates for a missing one.

As a caveat, the findings rest on an in-house evaluation and have not been confirmed by independent replication. Still, being measured across eight models gives firmer ground than claims made on a single setup.