Prompt caching, from the causal mask outwardAfter the parts
Recap
Every number, rule and trap from the post on one surface, laid out to be recalled at a glance rather than read.
ContentsAfter the parts
The whole thing, on one page
For the second reading, and for the moment six months from now when the hit rate drops and you need the table rather than the argument.
Prompt caching on one page
Every rule below is a consequence of the first panel. Figures are Claude Opus 5 unless noted.
The invariant
Caching is a prefix match. Attention is causal, so a token’s K/V vectors are a function of everything before it and nothing after. Any byte change anywhere in the prefix invalidates everything from that point on. There is no fuzzy match and there cannot be one.
Syntax
// automatic — one slot, last block
cache_control: { type: "ephemeral" }
// explicit — up to 4, per block
{ type: "text", text: DOC,
cache_control: {
type: "ephemeral", ttl: "1h" } }- Max 4 breakpoints; automatic uses one.
- 1h entries must precede 5m entries.
- Declaring breakpoints is free — you are billed on writes and reads.
Resolution
- Write happens only at your breakpoint: one entry, hashing the whole prefix above it. Nothing is written at earlier positions.
- Read walks backward from the breakpoint looking for entries prior requests wrote — not for content that happens to be stable.
- ≤ 20 positions. A run of consecutive tool_use blocks counts as one, and so does a run of tool_result blocks.
- An entry is readable only once the first response begins streaming.
Price (Opus 5, $5/MTok in)
| base input | 1× | $5.00 |
| write, 5m ttl | 1.25× | $6.25 |
| write, 1h ttl | 2× | $10.00 |
| read | 0.1× | $0.50 |
- Break-even: 2 requests at 5m (1.25 + 0.1 < 2), 3 at 1h (2 + 0.2 < 3).
- Reads are 0.025× on Claude Fable 5.1 ($0.25/MTok); unconfirmed for Mythos 5.1.
- Stacks with the Batch API discount.
Minimum cacheable prefix
| Opus 5, Fable 5 / 5.1, Mythos 5 / 5.1 | 512 |
| Opus 4.8, Sonnet 5 / 4.6 / 4.5, Opus 4.1 / 4, Sonnet 4 | 1,024 |
| Opus 4.7, Mythos Preview, Haiku 3.5 | 2,048 |
| Opus 4.6, Opus 4.5, Haiku 4.5 | 4,096 |
Below the minimum, caching is skipped with no error. Not monotonic across generations: a 3K prompt caches on Opus 5 and silently will not on Opus 4.6.
TTL and the clock
| start-to-start gap | ttl |
|---|---|
| under 5 min | 5m — every request refreshes it free |
| 5–60 min | 1h — the only window the 2× write repays |
| over an hour | neither; re-warm or accept the miss |
The clock starts when the request starts, not when the response ends: a 4-minute generation leaves about 1 minute of a 5-minute entry. Reads do not count toward input-token rate limits on most models (Haiku 3.5 excepted).
Invalidation — what survives
| change | tools | system | messages |
|---|---|---|---|
| message content edited | kept | kept | lost |
| tool_choice, images | kept | kept | lost |
| system prompt content | kept | lost | lost |
| speed, web search, citations | kept | lost | lost |
| thinking / effort | model-specific | model-specific | lost |
| tool definitions add/remove/reorder | lost | lost | lost |
| model switch | lost | lost | lost |
Escape hatches
- System prompt change → a {role:"system"} message in messages[]. Opus 5, Opus 4.8, Fable 5/5.1, Mythos 5/5.1. No beta header. Not Sonnet 5.
- Tool add/remove → tool_addition / tool_removal blocks. Opus 5 onward, beta mid-conversation-tool-changes-2026-07-01.
- Effort change → system message with content: [] and output_config. Fable 5.1, Mythos 5.1, Opus 5, beta mid-conversation-output-config-2026-07-01.
- Per-turn reminder → clear_at: "next_user_message", left in the transcript forever.
- Model switch → none.
Reading usage
cache_read_input_tokens 0.1×
cache_creation_input_tokens 1.25×
input_tokens 1× ← tail only
total prompt = sum of all threeHealthy loop: read grows turn over turn, creation ≈ the last turn’s delta, input_tokens is just the tail. Creation near full conversation size every request means the prefix is being rewritten upstream — or the turn exceeded the 20-position lookback.
usage.cache_creation splits by TTL into ephemeral_5m_input_tokens / ephemeral_1h_input_tokens.
Pre-warming
max_tokens: 0
→ content: [], stop_reason:
"max_tokens", cache written- Breakpoint goes on the shared block, never the placeholder message — and never via automatic caching, which would key the entry to the placeholder.
- Rejected with stream: true, thinking.type:"enabled", output_config.format, forced tool_choice, or inside Batches.
- Skip it when traffic is continuous, the prefix is small, or the prefix varies per user.
Silent invalidators
- Date.now() / uuid() anywhere in the prefix.
- JSON serialized without sorted keys — Go and Swift randomize map order, which silently changes tool_use hashes.
- Conditional system sections: every flag combination is its own prefix.
- Per-user tool sets or session IDs in the system prompt: no cross-user sharing.
- A fork (summarizer, sub-agent) that rebuilds system/tools instead of copying the parent’s verbatim.
- Workspace split: caches are per workspace on the Claude API, Claude Platform on AWS and Foundry; per organization on Bedrock and Vertex.
Limits and non-goals
- No manual eviction. Wait out the TTL or change the prefix — plan system-prompt rollouts around it.
- Sub-content blocks (citations) are not cacheable; mark the top-level document instead.
- Empty text blocks are not cacheable. Neither is a clear_at system message (a marker on one is a 400).
- Thinking blocks cannot carry cache_control, but are cached as part of prior assistant turns.
- N parallel requests with the same prefix all miss. Send one, await its first token, then fan out.
- Server tools such as web search insert their own 5-minute write after tool results. Expected, not a bug.
When it breaks, in order
- Check cache_read_input_tokens across two identical requests. Zero means nothing hit.
- Check the prefix clears the model’s minimum.
- Log consecutive request bodies and diff the overlap. Strip cache_control first — the moving marker always differs and is not the invalidator.
- Or use cache diagnostics (beta cache-diagnosis-2026-04-07), which names the divergence server-side. Send the header on every request: a one-shot retrofit fails with previous_message_not_found.
- Assert it in CI. The expensive failure is a regression six months after it worked.