LLMs
Inference as a system with a cost model, not an oracle — caching, context, tokens and latency.
Filter by tag — they cut across categories.
- Prompt caching, from the causal mask outward
Prompt caching explained from the one fact it follows from: attention is causal, so the cacheable unit is a prefix, never a block.
- prompt-caching
- kv-cache
- inference
- claude-api
- cost
- latency
- agents