In a nutshell Property #3 of four: what the model is looking at right now. Everything it holds in front of it at this moment — your prompt plus the whole chat history — is the context window. It's finite. When a conversation gets too long, the earliest part falls off the edge, and the model forgets it.
What the context window is
The model doesn't remember you "in general". Before each answer it's fed all the visible text: the system instruction, your question and the entire prior exchange. That fed-in amount is its working memory — what it can actually lean on right now. The window's size is limited and counted in tokens (see next-token prediction): it differs by model, but it's always finite.
The key difference from knowledge. Knowledge is what's "baked in" for good during training (long-term memory). Context is what's given here and now (short-term, working memory). They're different things, and confusing them is a source of errors.
Analogy — a desk. Knowledge is everything a person once learned and holds in their head. Context is the papers spread on the desk right now. The desk isn't infinite: lay down too many new sheets and old ones must go — and then you can't discuss them, even if they mattered.
What "falls off the edge"
When the total text (history + prompt) exceeds the window, the oldest is trimmed:
- the model "forgets" the start of a long conversation — you agreed early to "keep it short", and many messages later it's writing long again;
- details set long ago are lost — a name given at the start; a rule set an hour ago;
- it looks like it "got dumber" or "isn't listening", when really the needed part just left the window.
"Lost in the middle"
Even when everything fits, there's a subtlety: the model pays more attention to the start and end of the context, while information in the middle of a long text is processed more weakly. An important fact buried mid-way through a huge document can be "overlooked", though it was formally in view.
How to work with it
- Keep the important near the edges. Key instructions and data go at the start and/or repeat at the end, not drowned in the middle of a wall of text.
- Repeat / remind. In a long chat, periodically restate briefly what matters.
- Summarise and start fresh. The conversation ballooned — ask for a summary of conclusions and continue in a new, clean chat with it. The important is back in the window, the clutter is gone.
- Don't dump extra. The less irrelevant text, the more attention on what matters.
- Split big tasks. A huge document — in parts, not one lump where the middle "sags".
Takeaways
- "Forgot" ≠ "dumb": most likely the needed part left the window. The fix is to bring it back into context, not scold the model.
- Context is short-term memory (the desk); knowledge is long-term (the head). Different things, different problems (see knowledge & hallucinations).
Source
Anthropic Academy course AI Capabilities and Limitations, section "Working Memory".