Retention Is Allocation
Isolating what makes a writable neural memory persist. Neil Gilani, independent.
Writable memory is usually motivated as a cure for catastrophic forgetting, and the mechanism credited with the cure is usually the write — a learned, gradient-computed update that places new information into a state without disturbing what is already there. On a continual-learning stream where every ingredient can be held fixed but one, the write is not what produces retention. Allocation is.
- A gradient write into a single shared state retains 0.072 of what it learned.
- A private slot per memory with no write at all retains 0.988.
- The combination retains 1.000.
Varying only the number of pages available, across a 128× range, leaves immediate accuracy at exactly 1.000 at every setting while retained accuracy moves from 0.068 to 1.000 — and retained accuracy is predicted, with no free parameters, by the fraction of memories that still hold a page of their own, to within 0.0039 across all 6 settings.
Retention here is not a graded property of a learned update rule. It is a bookkeeping property of whether a memory was given somewhere to live.
Because retention is allocation, the whole problem moves to addressing. With an oracle over page identity the system is exact; with learned content addressing it sits on a false-merge / missed-revisit frontier that meta-training improves but does not clear. In a multi-tenant language model, the gap between oracle and learned routing is the entire remaining error.
The failures are reported in the same detail as the successes — including two benchmarks where a plain episodic store beats us outright, and one conclusion of our own that a later measurement overturned. All measured numbers.