One problem, two answers

Give an LLM agent a realistic multi-step task - split a bill, find a song, reconcile an order across nine simulated apps - and when it fails, it usually is not for lack of knowledge. It mis-paginates an API, resolves the wrong person, or returns a value when none was asked for. The model knows the APIs; what it has not internalised is how to use them reliably. That is learnable from the agent's own history.

A comparison published by IBM Research on 11 August 2026 takes two systems that do exactly this on the same kind of agent: ACE (Agentic Context Engineering) and IBM's own ALTK-Evolve. Both are a form of agentic memory, turning past trajectories into reusable lessons and feeding them back at inference time. No weight updates, no human labels.

What they agree on

The striking part is that both teams agree on the hard part: compressing an agent's hard-won lessons into a tidy summary is the wrong move.

ACE names the trap precisely. The first failure mode is brevity bias, optimisation collapsing toward short, generic instructions. The second is context collapse, where a model asked to rewrite its whole context summarises the detail away at each step. ACE's answer is to keep a rich, itemised playbook with a helpful/harmful counter on every bullet, and let the model distil relevance at read time.

ALTK-Evolve reaches the same conclusion from the other direction: every distinct guideline keeps a support count recording how many independent episodes produced it, and the store is never summarised down to a handful of rules. A lesson five different tasks discovered is a different object from one that appeared once. In the authors' phrasing: count them, do not collapse them.

Where they diverge

The divergence falls into two headings, and it is the second that shows up in the token bill:

  • Building the store: ACE grows one playbook through a Generator, Reflector and Curator loop, applying incremental delta updates and de-duplicating by embedding. ALTK-Evolve clusters near-duplicate lessons and merges within a cluster. The merge is support-conserving: the survivor inherits the combined count, so the store shrinks without losing the record of how much experience backs each guideline.
  • Delivery: ACE injects the comprehensive playbook on every step, the same way regardless of model or task. ALTK-Evolve treats delivery as a dial rather than a constant: a small fixed core of high-support guidelines, extended per task with a handful selected for the task at hand, or the full consolidated set when a model has the headroom to use it.

What the numbers say

On the AppWorld benchmark, with the same base ReAct agent and both systems run in-house, DeepSeek-V3.2 with ACE reaches 80.4 task and 73.2 subtask goal completion at 634K tokens per task, while ALTK-Evolve reaches 89.3 and 80.4 at 263K tokens: better on both metrics at roughly 40 percent of the inference cost. With gpt-oss-120b the picture changes. ALTK-Evolve edges ACE 56.0 to 54.8, but the authors treat that as a tie within the benchmark's run-to-run noise. The gain there is not accuracy but cost: 116K tokens against 777K, about one-seventh.

The authors add an honest caveat on cost. ACE's own efficiency story is about building its context cheaply; ALTK-Evolve's is about serving it. Retrieving a few guidelines per task instead of injecting the whole playbook on every step is directly where the tokens go. The by-difficulty breakdown supports this: on gpt-oss-120b the comprehensive context leads on easy and medium tasks, but on hard tasks, where the model has to pick the right lesson rather than wade through all of them, curated retrieval pulls ahead, and that is the tier deciding the aggregate.