Two layers that cut an LLM bill: prefix caching and semantic caching
In a production LLM application the cost piles up in two separate places. This guide separates prefix caching from semantic caching, and says what to measure.
Models
LLM cost
AI's biggest customer is now AI: agentic token usage jumped 14x
According to one analyst, February 6, 2026 may have been the last day humans consumed more tokens than agents. Agent usage has grown 14x since.
Research
In agent memory the real difference is delivery, not learning: ALTK-Evolve versus ACE
IBM Research compared its agent memory system ALTK-Evolve against ACE on the same agent. Both refuse to compress the lessons learned; they part ways on how those lessons reach the model, and that is where the bill comes from.
Research