Introduction

The bill you are already paying

The Messages API is stateless. There is no session, no conversation ID, no handle to something the server is holding for you. Every request carries the entire conversation, and on turn twelve of an agent loop that looks like this:

await client.messages.create({
  model: 'claude-opus-5',
  max_tokens: 16_000,
  tools,              //    600 tokens — unchanged since the last deploy
  system: houseRules, //  2,400 tokens — unchanged since the last deploy
  messages: history,  // 11 turns you have now sent eleven times
})

The server reads all of it. For every token it computes a key and a value vector at every layer, runs the forward pass, streams the answer, and throws the whole intermediate state away. On turn thirteen you send the same bytes and it does the same arithmetic again.

This is not a rounding error. Fifty questions about one 20,000-token document, each with a hundred-token question attached, is 1,005,000 input tokens — of which 980,000 are the same document read forty-nine more times. At Opus 5’s $5 per million that is $5.03 to ask fifty questions about a document you sent once. Cached, the same fifty requests bill one write, forty-nine reads and the questions: $0.64. And you pay the uncached version twice over — in money, and in time-to-first-token, because prefill is the latency floor before generation can begin at all.

So why is the API stateless in the first place? Because the alternative is worse. A server-side session pins your conversation to one machine, makes retries stateful, and takes away the thing that makes the Messages API tractable: you own the transcript. You can rewind it, branch it, edit a turn, replay it against a different model. Prompt caching exists to keep that property and stop paying for it twice — the client stays the sole owner of the conversation, and the server keeps a disposable copy of the arithmetic.

What this post does

It derives prompt caching from that one fact instead of listing its rules, in four parts. Each part ends where the next one needs to begin, so they read best in order.

1. Part 1 — Why the cache is a prefix. The causal mask, what is actually stored, how a breakpoint cuts a request, and why a miss walks backward looking for earlier writes rather than for unchanged blocks.
2. Part 2 — Where the breakpoint goes. The placement that bills more than not caching at all, why “the last block” is right for a growing conversation, and how four breakpoints map to four rates of change.
3. Part 3 — What it costs, and for how long. The multipliers and their break-even, a TTL measured from the start of the request, the per-model floor, and how to read the usage fields.
4. Part 4 — Running it. Pre-warming without the fan-out trap, the sharp edges that fail silently, and the mechanism rebuilt from scratch in eighty lines of tested TypeScript.

After the parts: a one-page recap for recall, cards to check yourself, a glossary and the references. If you would rather not page through it, the whole post is on one page.