Retention Is Allocation
Isolating what makes a writable page memory persist. Neil Gilani, independent.
Writable memory is usually motivated as a cure for catastrophic forgetting, and the mechanism credited with the cure is usually the write — a learned, gradient-computed update that places new information into a state without disturbing what is already there. On a continual-learning stream where every ingredient can be held fixed but one, the write is not what produces retention. Allocation is.
- A gradient write into a single shared state retains 0.072 of what it learned.
- A private slot per memory with no write at all retains 0.988.
- The combination retains 1.000.
Varying only the number of pages available, across a 128× range, leaves immediate accuracy at exactly 1.000 at every setting while retained accuracy moves from 0.068 to 1.000 — and retained accuracy is predicted, with no free parameters, by the fraction of memories that still hold a page of their own, to within 0.0039 across all 6 settings.
Retention here is not a graded property of a learned update rule. It is a bookkeeping property of whether a memory was given somewhere to live.
Because retention is allocation, the whole problem moves to addressing. With an oracle over page identity the system is exact; with learned content addressing it sits on a false-merge / missed-revisit frontier that meta-training improves but does not clear. In a multi-tenant language model routing is not the only remaining source of error either: supplying the correct page still leaks, and the residue is the frozen model answering from its own prior having never read the neighbour's page.
The failures are reported in the same detail as the successes — including two benchmarks where a plain episodic store beats us outright, and one conclusion of our own that a later measurement overturned. All measured numbers.