The situation
You took the point about a crowded window seriously, so now you write short. No file attached, no background, no example: the request, stripped to a line. The first answer misses what you meant, the second fixes half of it, and by the fourth you have typed more than you withheld. The prompt was economical. Nothing else was.
The idea in one paragraph
A token is the unit all of this is counted in: what you send, what comes back, and what your bill or usage limit measures. The two directions are not the same good. What goes in is bulk, cheap per unit, and reusable across turns if you leave it where it was. What comes back is made one token at a time, costs several times more, and is reusable never. A retry buys both again, so the saving worth chasing is one fewer exchange, and almost nothing you can cut from a prompt is worth one.
Where a session's tokens actually go
| What it pays for | Who sizes it | What it costs | What cutting it buys |
|---|---|---|---|
| Your instruction | You | The smallest line on the bill | A guess where a constraint belonged |
| Files and command output | The tool, on request | Little each, huge in bulk | A real saving, if it was not needed |
| The answer | The model | Several times the input rate | A short answer you ask about again |
| Every earlier turn, re-read | The session's length | Paid again on every message | The other real saving: start fresh |
| The retry | Whether the rows above went right | All of them, twice | The row to protect |
Two of these can be cut safely. The one most people cut first is not one of them.
How it actually works
Tokens are not words. One vendor puts English at roughly four characters per token, with long words broken across several. Nor is a token fixed inside one vendor: Anthropic documents its newer models on a newer tokenizer producing about 30 percent more tokens for identical text, and calls its own count an estimate (both checked 10 August 2026).
Reuse is the part worth understanding, and the part you lose by accident. What gets reused is a prefix, not a set of facts: the request is matched from the front, so a hit means everything up to some point was identical to a recent one. Anthropic's cache runs tools, then instructions, then messages, and a change at any level discards that level and everything after it. A hit reads at about a tenth of the input rate, and the entry lives about five minutes.
So reuse rewards a stable opening and a varying end. Put a timestamp or today's shifting notes near the front and you pay full price every time while feeling thrifty. Below a minimum length nothing is cached at all, and no error tells you.
What this changes for you
- Judge input by exchanges, not by length. A paragraph of constraint that prevents one retry has paid for itself before you send it.
- Order a session stable-first: the instruction file, then the reference material, then today's request. That is the ordering a discount can land on.
- When you want a short answer, constrain the artifact rather than the thinking: the three lines that change, one file, a table with named columns. "Be brief" cuts the part that told you whether to trust the rest.
Where it breaks
The economy runs the other way too. Attaching the whole repository is not spending input well: it is the crowded context window with a bill attached, and it makes the answer worse as it makes the session expensive.
The discount is not a plan: it expires in minutes, you cannot see whether you got it, and most products decide the details for you. A workflow that only works when the cache hits does not work.
On a flat subscription none of this reaches you as money. The units are your usage limit and your afternoon, and the shape is identical.
Terms used on this page
- Tokenizer — the step that splits text into tokens before a model reads it. The same text costs a different number of tokens on a newer model, and long words break into several.
- Cached prefix — the opening stretch of a request a vendor recognises from a recent one and re-reads at a fraction of the input price. Change anything inside it and it is gone.
- Round trip — one full exchange: everything so far read in, one answer generated. The unit a retry costs.
- False economy — a saving that is real on the line you measured and larger somewhere you did not look.
Read next
Models and effort — the two settings that decide how much capability and how much thinking you are buying, and why the expensive one is not always the one that fixes the problem.