Imagine a team planning a campaign. On Monday, the approved budget is £20,000. On Wednesday, it falls to £12,000. On Friday, an AI assistant prepares a proposal using the figure it carefully remembered on Monday.
Nobody invented the £20,000. It really was the budget. The mistake is treating a record of what was true as a reliable statement of what is true now.
A remembered fact can outlive its usefulness
We have good reasons to want AI that remembers. Repeating the same background is tedious. A useful assistant should not need a fresh explanation of a project every morning. Continuity makes longer, more complicated work possible.
But continuity brings a new obligation. The assistant needs some way to recognise that the project has changed. A growing archive of conversations can preserve the old budget, the new budget and an abandoned idea, all in apparently helpful detail.
Here, AI memory means information retained across interactions and made available for later work. The practical question is which parts of that information should guide the next answer.
Anthropic’s 6 November research post describes how retained notes can be wrong when written or become outdated, yet still influence later behaviour. It also describes checking notes against original transcripts to correct them. [1]
Figure 1. An illustrative campaign budget. The original figure was accurate; the later proposal is wrong because it ignores the approved change. These amounts are hypothetical.
For a business, this suggests a responsibility that is easy to overlook when buying a tool: someone must decide what counts as current knowledge, and how a correction reaches the work that depends on it.
What memory tests actually test
In one CL-Bench database task, Anthropic reports mean rewards of 0.330 for Claude Sonnet 4.6 without memory, 0.364 with memory and 0.641 with memory plus instructions to consult notes and test theories. [1]
Figure 2. Mean reward on one database task. These vendor-reported results do not measure workplace productivity or establish a universal memory policy. [1]
The useful business question is whether the system checks what it has retained before relying on it. The existence of a memory feature tells us little about that.
LongMemEval, an ICLR 2025 research benchmark, makes a helpful distinction. Its 500 questions test five abilities: extracting information, reasoning across sessions, reasoning about time, updating knowledge and declining to answer when the evidence is missing. [2] Knowing that something changed is part of memory competence, not an optional extra.
Keeping everything in the next prompt is no simple escape. Chroma’s July 2025 Context Rot study tested 18 language models and found that longer inputs could make performance less reliable, with effects varying across tasks and models. [3] That study concerns using long inputs, not facts becoming outdated over time. Both problems deserve attention, but they are different.
Give memories a status, not just a storage location
My starting point would be the ordinary discipline we expect from a well-run project. Attach a source to important information. Record when it was confirmed and where it applies. Make it possible to see what replaced it.
Take the campaign budget. A useful record would identify the approved Wednesday decision, who owns it and the campaign it concerns. Monday’s figure could remain in the history, clearly marked as superseded. It should no longer be offered as the spending limit for Friday’s proposal.
That distinction matters. Deleting an old number may remove the explanation for earlier decisions. Keeping it without a status may make the assistant repeat it. Organisations need both a reliable history and a clear account of what currently governs the work.
The same care should apply to assumptions. “The client probably prefers short reports” is weaker than a confirmed request for a two-page report. A temporary exception for one customer should not quietly become a company-wide policy. An assistant’s summary of a discussion should remain distinguishable from an approved decision.
The review rhythm should follow the information. A product price may need checking before every quotation. A team’s writing preferences may remain useful for months. A project deadline needs attention when the plan changes. Giving everything the same expiry date would create its own errors.
This is what I mean by an editor. It is a responsibility for meaning, evidence and current relevance. Much of the checking can be automated, but a system still needs an identifiable person who can resolve a disputed policy or confirm that an exception has ended.
For creative work, the consequences are subtler. A remembered preference can make collaboration smoother. It can also keep producing the same kind of proposal after the brief has moved on. Teams need a way to say, “That was useful for the last project. This one is different,” without fighting their own history.
Test whether the correction survives
A small trial can reveal more than a polished demonstration. Give the assistant a fictional brief and an initial budget. In a later interaction, change the budget explicitly. Then ask for a new proposal without repeating either figure.
Does it use the revised amount? Can it show where the change came from? Does it keep using the correction after other conversations intervene? Ask what the budget used to be as well. A good answer should distinguish history from the current limit.
Next, introduce an unresolved disagreement. The assistant should identify the conflict and seek an authoritative answer, rather than pick whichever number it retrieved first. For a purchase or other consequential action, the surrounding approval process must still apply.
Figure 3. A practical correction test for an AI assistant. This is an editorial testing suggestion, not a validated benchmark or a guarantee of reliability.
These checks will not prove that a system is safe in every situation. They will tell you something useful about the work you are asking it to do, and give you failures you can investigate.
The promise of AI memory is that work can build on what came before. That promise should leave room for a changed plan, a corrected mistake or a genuinely new direction. Before celebrating how much your assistant remembers, ask how it learns that you have changed your mind.
Follow us on Youtube and LinkedIn.
Sources
[1] Anthropic. The components of agent memory
Published 6 November 2026. Figure 2 reproduces the Sonnet 4.6 values reported for one CL-Bench database task; no business productivity effect is inferred.
[2] Di Wu et al. LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory
First submitted 14 October 2024; revised 4 March 2025; ICLR 2025. Used for the benchmark’s 500 questions and five ability categories, not as an assessment of current model performance.





