inqizitblog

Theme

Linen — The house face. Warm paper, rust ink, and nothing shouting.

ai · llm ·

Prompt caching, from the causal mask outwardPart 3 of 4

What it costs, and for how long

Break-even in two requests, a TTL that starts with the request, a per-model floor below which nothing caches, and the usage fields that show it.

IndexFull pageRecap

ContentsPart 3
  1. Start hereIntroduction
  2. Part 1Why the cache is a prefix
  3. Part 2Where the breakpoint goes
  4. Part 3What it costs, and for how long
    1. 11The arithmetic
    2. 12The clock starts earlier than you think
    3. 13The floor
    4. 14What the response tells you
  5. Part 4Running it
  6. After the partsRecap
  7. After the partsPractice
  8. AppendixGlossary
  9. AppendixReferences

IndexFull page

The arithmetic

Writes cost more than ordinary input; reads cost far less. Everything else is division.

TokenMultiplierOpus 5
base input1×$5.00 / MTok
cache write, 5-minute TTL1.25×$6.25 / MTok
cache write, 1-hour TTL2×$10.00 / MTok
cache read0.1×$0.50 / MTok

Break-even is two requests on the 5-minute TTL (1.25 + 0.1 = 1.35, against 2.0 for sending it twice uncached) and three on the 1-hour TTL (2 + 0.2 = 2.2, against 3.0). One hit repays the write. These multipliers stack with other modifiers, including the Batch API discount.

Claude Fable 5.1 is the exception worth knowing: reads there are 0.025× ($0.25/MTok), which moves every break-even below proportionally and changes a TTL decision we will get to in a moment. Whether Mythos 5.1 shares that rate was open at launch.

The clock starts earlier than you think

Default TTL is five minutes, and a read refreshes it for free. The part that bites: the lifetime is measured from the start of the request that writes or reads the entry, not from the end of the response. A generation that streams for four minutes leaves you about one minute to get the next request in.

So choose by the start-to-start gap between requests that share the prefix, not by how long a session feels:

GapTTLWhy
< 5 min5-minuteEvery request refreshes it. Strictly cheaper — the 1-hour TTL buys nothing here except the doubled write.
5–60 min1-hourThe only window where the 2× write repays itself.
> 1 hourneitherRe-warm on a schedule, or accept the cold miss.

Set it per block, and remember the ordering constraint: 1-hour entries must appear before 5-minute ones in the prompt.

system: [
  { type: 'text', text: HOUSE_RULES,
cache_control: { type: 'ephemeral', ttl: '1h' } },   // long-lived first
  { type: 'text', text: retrievedDoc,
cache_control: { type: 'ephemeral' } },              // then 5m
]

Is the 1-hour TTL ever the wrong tool in that 5–60 minute window? On Fable 5.1, usually. Because reads there are 0.025× rather than 0.1×, a read is nearly free and a miss is expensive relative to one — so instead of paying 2× on the write, stay on the 5-minute TTL and, while idle, resend the previous request with max_tokens: 0 shortly before the entry expires. That refreshes the timer and bills only a cheap read. It beats the 1-hour TTL unless pauses regularly approach an hour.

One more thing about reads that matters for capacity, not cost: on the Claude API they do not count toward input-token rate limits on most models (Haiku 3.5 is the documented exception). Keeping entries warm raises effective throughput as well as cutting the bill.

The floor

Below a per-model minimum, caching is skipped silently — no error, no warning, both usage counters at zero.

ModelMinimum cacheable prefix
Opus 5, Fable 5 and 5.1, Mythos 5 and 5.1512 tokens
Opus 4.8, Sonnet 5 / 4.6 / 4.5, Opus 4.1, Opus 4, Sonnet 41,024
Opus 4.7, Mythos Preview, Haiku 3.52,048
Opus 4.6, Opus 4.5, Haiku 4.54,096

Note that it is not monotonic across generations. A 3,000-token prompt caches on Opus 5, Opus 4.8 and Sonnet 4.5, and silently will not on Opus 4.6 or Haiku 4.5. Opus 5 halved the Opus 4.8 minimum, so prompts that were previously too short to cache now create entries with no code change on your side.

What the response tells you

Three fields, and one of them is routinely misread:

console.log(response.usage.cache_read_input_tokens)      // served from cache, 0.1×
console.log(response.usage.cache_creation_input_tokens)  // written this request, 1.25×
console.log(response.usage.input_tokens)                 // NOT your total input

input_tokens is only the tokens after your last breakpoint. The total prompt is the sum of all three. With a 200,000-token cached document and a fifty-token question you will see a read of 200,000, a creation of 0, and input_tokens: 50 — and if you are metering credits or attributing cost per user, that three-way decomposition is exactly what you want to bill against, since the three components have different prices.

With 1-hour caching in play, usage.cache_creation breaks down further into ephemeral_5m_input_tokens and ephemeral_1h_input_tokens.

When reads collapse to zero, the usage fields tell you that the prefix broke but not where. Two ways to localise it. Log consecutive request bodies and diff the overlapping region — adjacent payloads in a growing conversation legitimately differ at the end, but the previous request’s prompt must reappear unchanged as a prefix of the next one. Strip the cache_control markers before diffing: the moving marker always differs between adjacent requests and is not the invalidator. Or opt into cache diagnostics (beta cache-diagnosis-2026-04-07), which does the comparison server-side and names the tier that diverged:

const res = await client.beta.messages.create({
  betas: ['cache-diagnosis-2026-04-07'],   // send on EVERY request —
  diagnostics: { previous_message_id: prevId },  // fingerprints are only kept
  ...request,                                     // for requests that carried it
})
console.log(res.diagnostics)   // model | system | tools | message history

A one-shot retrofit fails with previous_message_not_found, because nothing fingerprinted the earlier request.

And verify after every change, not once at setup. The costliest caching failure in production is a regression, not a bad first implementation: it worked when written, then someone added a dynamic field to the system prompt, or a feature that rewrites history, or a tool list that stopped being sorted — and nothing announced it. Requests keep succeeding. The bill is just higher. An integration test asserting that a second identical request shows cache_read_input_tokens > 0 is the cheapest standing check there is.