Caching
Prompt caching shipped in 1.5.1. Most of every Spec4 request is text the model has already seen — an agent's system prompt, and for an agent that holds a running exchange with the model, the turns so far. Caching lets the model reuse the processed prefix instead of processing it again.
Spec4 turns it on automatically for the providers where it is safe. There is nothing to configure.
What it does
On Anthropic models a cached read costs one tenth of a normal input token, and a cache write costs 1.25×. So a prefix that is written once and read on every later call pays 1.25 once and 0.1 thereafter, instead of 1.0 every time.
Nothing about the plan changes. The model sees the same prompt; only the price of the part it has already processed changes.
The two breakpoints
- The system prompt. Static for the life of a run: written once, read on every later call that agent makes. It also covers the web-search tool definition, because Anthropic caches tool definitions ahead of the system prompt.
- The last message. A moving breakpoint. Each turn reads the whole previous prompt from cache and writes only what that turn added.
Which agents get which
Exactly six agents get the second breakpoint: Brainstormer, Agentifier, CodeScanner, StackAdvisor, Phaser, and Deployer. These are the multi-turn agents — the ones whose exchange with the model accumulates, so the last message of one call sits inside the prefix of the next.
Multi-turn here is about the exchange with the model, not the one with you. CodeScanner is in this set and asks you nothing; Designer is outside it and refines a mock against your feedback.
Everything else gets the first breakpoint alone. Its last message is a user turn that differs on every call, so a second breakpoint would be a guaranteed write with no read. That covers the one-shot sub-agents — Scout, TierAnalyst, Linker, Composer, Prioritizer, SpecDrafter, and Feature Speccer — and Designer, which builds one system message and one user message per draw and carries no conversation history.
Designer is worth stating rather than leaving to inference. It is one-shot, and its system prompt sits below every current model's minimum cacheable prefix, so Designer draws are not cached at all. A prompt edit that pushes it over the floor would change that.
Providers
| Provider | Caching |
|---|---|
| Anthropic | On. |
| AWS Bedrock (Claude) | On. |
| OpenAI | Provider-managed. Spec4 sends no markers. |
| Google Gemini | Off, deliberately: a message-level marker makes LiteLLM switch to Vertex explicit context caching, which adds live HTTP calls per request. Gemini's own implicit caching still applies. |
| Mistral | Off. Markers are stripped. |
| Nebius Token Factory | Off, deliberately: a strict OpenAI-compatible endpoint may reject the markers. |
| OpenRouter | Off. Spec4 routes no Claude traffic through it. |
The provider is resolved from the configured model string, so a model picked from any provider's list gets the right behaviour without configuration. Settings
Minimum cacheable prefix
Anthropic will not create a cache entry below a per-model floor. The floor is not monotonic across a model line, so it is a table and not a sentence. This is Anthropic's published table as of September 2026; check theirs if this one looks stale.
| Floor | Models |
|---|---|
| 512 | Fable 5.1, Mythos 5.1, Opus 5, Fable 5, Mythos 5 |
| 1,024 | Sonnet 5, Sonnet 4.6, Sonnet 4.5, Sonnet 4, Opus 4.8, Opus 4.1, Opus 4 |
| 2,048 | Opus 4.7, Mythos Preview, Haiku 3.5 |
| 4,096 | Haiku 4.5, Opus 4.6, Opus 4.5 |
A request below the floor is processed normally, at full price: no write, no penalty.
Spec4's agent system prompts run from about 150 tokens (Composer) to about 10,500 (StackAdvisor). On a 1,024-floor model the system prompt caches from the first call for every agent except the smallest one-shots, and a conversation caches turn over turn from call 2. On a 4,096-floor model, Brainstormer and Agentifier show no cache activity until the conversation itself passes 4,096 tokens — 7–8 turns for Brainstormer and 2–3 for Agentifier in measured runs — while the larger agents' system prompts clear the floor on their own and the small one-shot sub-agents never cache at all.
What it saves
Measured on one vision, Brainstormer and Agentifier only, against the same run with caching off:
- Sonnet 4.5. 16% of run cost with the system-prompt breakpoint alone, 35% with both. Brainstormer read 91% of its input tokens from cache and saved 61% of its cost.
- Haiku 4.5. About 0% with the system prompt alone, 14% with both.
Those two agents are the front of the pipeline, not a whole round. StackAdvisor, Phaser, and Deployer have the largest system prompts, so their savings should land at the upper end of that range; they were not in the measured slice. Designer saves nothing, for the reason above.
Cache lifetime
An entry lives five minutes from its last use. Pause a Brainstormer conversation for longer than that and the next turn misses and rewrites the whole prefix — about a cent on Sonnet at current prompt sizes — with reads resuming on the turn after. In a measured run, two pauses of 400–518 seconds produced exactly one miss.
Two other things reset a conversation's cache: changing the web-search key mid-run, and a validation retry that drops the tools array.
Turning it off
SPEC4_PROMPT_CACHING=0 in the environment — or false, or off. It is read on every call, so it takes effect without a restart.
It exists for A/B measurement. There is no reason to use it in normal operation.
Reading the numbers
The usage report, spec4-usage, carries three cache columns per agent: cached is tokens read from cache, cache_write is tokens written, and hit% is cached over input.
Both cache figures are already inside input. Do not add them to it.
A blank means no call reported the field — a round planned before caching, or a provider that does not report writes. The healthy shape is a write on the first call and reads on every call after it. In the app, the cost card shows (N cached) beside the token counts.
In usage.json, every call record carries cache_read_input_tokens and cache_creation_input_tokens, and the per-agent rollups carry cached_input_tokens and cache_creation_input_tokens. Artifacts