The situation
Two hours in, the session was going well. Now it is rewriting a function you already fixed, the constraint you set at the start has quietly stopped applying, and it is describing a file that does not exist. You have not changed how you ask, and nothing on screen says anything is wrong: the whole conversation is still there, scrolled up.
The idea in one paragraph
Everything the model appears to know about your project is text put in front of it for this one answer. That space is the context window: your messages, its replies, the files it opened, the output of commands it ran, and product instructions you never see. It is not memory. It is a workbench, cleared when the session ends and more crowded with every turn until then. A crowded bench is harder to work on before it is full, so the window's published size tells you what can be submitted, not what will be used well.
What is on the bench when you send one message
| What is in there | Who put it there | Do you see it | Across the session |
|---|---|---|---|
| The product's instructions and tool list | The product | No | Constant |
| Your project's instruction file | You, once | Rarely | Constant |
| Every message so far | Both of you | Yes | Grows every turn |
| Files it opened and command output | The tool | Partly | Grows fastest, mostly spent |
| The message you just typed | You | Yes | Small |
| Room to think and to answer | The product | The answer only | Whatever is left |
Only the bottom two rows are about the thing you just asked for.
How it actually works
Give the model five relevant facts and it mainly has to reason. Bury the same five in a hundred thousand tokens of transcript and log, and it has two jobs: find them, then reason. The finding can fail while the answer is technically present, which is how a session gets worse while being better informed. Anthropic describes this space as a finite attention budget with diminishing returns rather than a shelf that is either full or not (checked 10 August 2026).
Published evaluations agree without promising a curve. The 2024 "Lost in the Middle" paper found tested models did better when the relevant passage sat near the start or end of a long input than in the middle. The 2025 NoLiMa benchmark found accuracy falling sharply as inputs grew when the question did not reuse the source's wording. A 2025 Chroma report across eighteen models found reliability declining with length, and near-miss distractors mattering as much as length itself. Those are particular models on particular tasks: there is no percentage at which quality falls over, and anyone quoting one is guessing.
What you see is also not what it gets. To stay under the limit, products drop older turns or summarize them into something shorter, and the conversation still reads as continuous.
What this changes for you
- Give one session one finishable job, and start a new one for the next. The reset is the feature.
- Control what lands on the bench. Ask for the failing test rather than the whole run, one directory rather than the repository, the conclusion rather than the log that produced it.
- Restate a load-bearing constraint next to the request it constrains, rather than trusting a sentence you typed an hour ago.
Where it breaks
A crowded window is a mechanism; a bad answer is an outcome, and the link between them is real but loose. Plenty of wrong answers owe nothing to context: stale training knowledge, an ambiguous request, a check nobody ran. Diagnosing every one as "the window is full" costs you the actual cause.
The opposite failure is just as common. Too little context makes the model guess, and a guess costs more than the tokens you saved. The target is the smallest complete set of current material the job needs, not the shortest prompt you can write.
Terms used on this page
- Context window — the text a model can see while producing its next answer, cleared when the session ends.
- Token — the unit text is split into before a model reads it. One vendor puts it at about four characters of English, so 100 tokens run to 60 or 80 words (checked 10 August 2026). Window sizes are quoted in tokens.
- Effective context — the part of what you sent that the model actually uses well. Always smaller than the part it accepted.
- Compaction — a product summarizing older turns into a shorter form to stay under the limit. It is selective compression, not lossless memory.
- Distractor — material that looks relevant but is not, competing with the fact you needed. Harder to ignore the closer it sits to the truth.
Read next
Tokens and false economies — what fills that space is counted in tokens, and the cuts that look like a saving are usually the ones that cost you another attempt.