Desmond: Nineteen eighty-nine. Michael McCloskey and Neal Cohen train a small neural network on the addition facts for one — one plus anything.
Desmond: It learns them perfectly.
Nora: And then?
Desmond: Then they teach it the twos. And it can no longer do the ones. Not degraded. Gone.
Nora: Catastrophic interference. That's the famous headline.
Desmond: The headline is wrong about why. And getting the why right is what this whole episode turns on.
Nora: Welcome back to The Seams Are The Point. I'm Nora.
Desmond: I'm Desmond. Episode three of five.
Nora: Last time, we found out human memory has no delete key. Only accessibility, competition, and correction.
Desmond: This time — why there are two separate memory systems at all, and what actually earns a fact permanent residence in one of them.
Nora: Spoiler. It isn't how many times you saw it, and it isn't how sure you feel.
Nora: Back to that network. Ones, then twos, and it forgets the ones. Everyone reads that as 'neural networks are forgetful by nature.'
Desmond: That's the pop version. Here's the precise claim, and it's a different animal.
Nora: Go.
Desmond: The same network learns both fact sets fine — perfectly fine — when they're interleaved. Ones and twos mixed together in training.
Nora: So it's not that the network can't hold both.
Desmond: It can hold both. The failure only shows up when you update shared parameters on a non-representative sample — hammer one thing, then hammer a totally different thing, nothing mixing them.
Nora: So it's not 'networks forget.' It's 'this specific training regimen breaks them.'
Desmond: Right. And that's the first correction of the episode. The kind of deflation this show runs on.
Nora: A famous result, quietly reattributed from a property of neurons to a property of a training schedule.
Desmond: Which matters, because six years later somebody took that result and built an entire theory of the brain on top of it.
Nora: Nineteen ninety-five. McClelland, McNaughton, and O'Reilly.
Desmond: They turn McCloskey and Cohen's result into an argument for why brains need two memory systems, not one.
Nora: This is the hippocampus-and-cortex story. Short-term and long-term. RAM and disk.
Desmond: That's the gloss everyone remembers, and it has the axis wrong.
Nora: Wrong how?
Desmond: It was never about how long something is retained. It's about representational format.
Nora: Meaning?
Desmond: One system is sparse and pattern-separated — it can grab a single new event without disturbing anything else already stored.
Desmond: The other is dense and overlapping — it extracts structure across many experiences, and that requires small adjustments to shared parameters, constantly.
Nora: So the same problem that broke the ones-and-twos network —
Desmond: — is exactly the problem a structure-extracting system runs into by design. A system built for that cannot also be the system that has to grab one new fact right now, safely, without touching anything else.
Nora: You need both. And they genuinely cannot be the same machinery.
The Record: Two corrections on the historical record, since this act gets quoted more loosely than most.
The Record: The lion-at-the-watering-hole example that usually illustrates this literature belongs to Kumaran and colleagues, two thousand sixteen. Not to McClelland and colleagues, nineteen ninety-five.
The Record: And the original slow-learning-cortex claim was amended by its own authors as evidence accumulated. Amended. Not retracted.
Nora: So if format is the reason you need two systems, what decides when something moves from the fast one into the slow one?
Desmond: Two thousand seven. Dorothy Tse and colleagues, in Richard Morris's lab. This is the experiment that answers that.
Nora: Set it up.
Desmond: Rats learn six flavor-place associations in an arena full of sand wells. About a month of training to get there.
Nora: A month.
Desmond: Then two brand-new pairs. One rewarded trial each. Just one. And the animals have them twenty-four hours later.
Nora: One trial, retained a full day. That's fast.
Desmond: Here's the scalpel. Remove the hippocampus three hours after that single trial, and the memory is gone at test. Remove it forty-eight hours after, and the memory is intact.
Nora: Consolidation that normally takes weeks had closed inside two days.
Desmond: Right. But the headline isn't the interesting part. The controls are.
Nora: Go on.
Desmond: Take those same experienced rats — a month of training behind them — and move them to a different arena. New landmarks, new geometry.
Nora: And?
Desmond: They learn six new pairs at exactly the rate naive animals took the very first time. All that expertise, gone as a speed advantage.
Nora: So familiarity with the task didn't transfer.
Desmond: None of it transferred. Only a matching schema helped. Task familiarity is not sufficient — consistency with existing structure is the whole game.
Nora: Translate that for us.
Desmond: Consistency with the existing general-fact store is the strongest available predictor of whether one observation can safely become a durable fact.
Desmond: Stronger than recency. Stronger than frequency. Stronger than how confident the source sounds.
Nora: That's the single most actionable rule in this entire act.
Desmond: And it has a corollary with teeth. If the store is internally contradictory, promotion does not work — and throwing more replay at it will not rescue it.
Nora: So a contradictory store isn't just messy. It's a store that's stopped being able to learn anything new.
Desmond: That's the failure mode where the pipeline looks busy and produces nothing durable. Audit for internal contradiction like a standing health metric — it's load-bearing for everything downstream.
The Record: One number missing from that account, on purpose.
The Record: The Tse experiment includes an inconsistent-schema control, and specific figures from it could not be verified during fact-checking. They are omitted here and reported only qualitatively, not numerically.
Nora: So consistency gates promotion. What does the offline pass actually do with that, mechanically?
Desmond: Three more results shape it. First — interleave against a retrieved neighborhood, not against everything.
Nora: Not the whole corpus.
Desmond: Kumaran, Hassabis and McClelland's two thousand sixteen review is explicit that which memories get interleaved is still an open problem.
Desmond: And that replaying the entire corpus is neither required nor plausible.
Nora: So the move is —
Desmond: Retrieve the k related existing facts first. Re-derive the general statement over the union of those and the new episode. Never over the new episode alone.
Nora: Second result.
Desmond: The writer cannot compute promotion priority. Only the consolidator can.
Nora: Why not the writer, in the moment?
Desmond: Lisman and Grace's hippocampal-VTA loop proposal, plus work on retrospective reweighting, points at something structural — encoding priority gets revised by events that happen later.
Nora: The information you'd need to score an episode arrives after the episode.
Desmond: Exactly. So the fast writer just captures cheaply and uniformly, with a stable id.
Desmond: Priority assignment belongs to the offline pass, with hindsight over the whole window. That's an architectural constraint, not a metaphor.
Nora: Third result — and it's the one in the section title.
Desmond: Memory as a function of expectedness is U-shaped. Both strongly schema-consistent items and strongly schema-violating items get retained better than the stuff in the middle.
Nora: So the two obvious promotion rules —
Desmond: 'Promote what's surprising' and 'promote what fits' — are each half-right, and this refutes both of them at once.
Nora: What's the actual rule?
Desmond: Promote strong confirmations, because they make a schema statistically reliable and they're cheap to absorb. Promote strong violations, because they signal the schema is wrong. Drop the mushy middle.
The Record: A scope note on that U-shape.
The Record: The behavioral effect is well supported and has replicated. The neural mechanism originally proposed to explain it has not held up nearly as cleanly. Take the behavior. Leave the mechanism.
Nora: One more thread before we close the act, because this is the best one.
Desmond: Experience replay, in machine learning — that wasn't just convenient naming. It was an explicit borrow from hippocampal replay. Lin, nineteen ninety-two. Mnih and colleagues, twenty fifteen.
Nora: So the field borrowed the biology on purpose.
Desmond: And then, independently, rediscovered a second piece of it — reweighting replay toward significant events. Same thing the brain seems to do.
Nora: That's the part that should worry us a little.
Desmond: It's the next part that should reassure you. The field also discovered the correction the biology apparently doesn't have.
Nora: What correction?
Desmond: Schaul and colleagues' prioritized replay needs importance-sampling weights, because biasing what you replay biases what you learn.
Nora: And Kumaran's review —
Desmond: — lists exactly that as an open problem. For the brain. Not solved. Just noticed.
Nora: Walk me through why it matters for us specifically.
Desmond: If we prioritize consolidation by surprise and failure, the general-fact store over-represents failure modes. It misstates base rates.
Nora: Give me the failure in one line.
Desmond: 'Tool X is unreliable,' derived from a queue that only ever replayed the times it broke.
Nora: So the fix.
Desmond: A uniform sampling stratum, sitting alongside the priority-weighted one. We can build that in an afternoon. Biology apparently can't.
Nora: Let's land the design rules. Catastrophic interference was never networks being forgetful — it's shared parameters updated on a non-representative sample.
Desmond: Two systems exist because of representational format, not retention duration. Sparse and separated versus dense and overlapping. You need both.
Nora: Promotion is gated by consistency with what the store already believes, more than by recency, frequency, or confidence. And a contradictory store cannot be rescued by more replay.
Desmond: Interleave against a retrieved neighborhood, not the whole corpus. Let the offline pass assign priority, with hindsight — the writer can't know it yet.
Desmond: And promote the ends: strong confirmations, strong violations, drop the middle.
Nora: And correct for what biology apparently can't. Sample uniformly, or your fact store learns mostly about your own failures.
Desmond: Which raises the obvious next question. If two systems and a gate get memory right for one agent — what happens once you put more than one agent in the loop?
Nora: Episode four. The Committee. And the honest answer, going in, is that more agents mostly don't help — the literature is fairly blunt about it.
Desmond: See you there.