The Anthropic API caches prompt prefixes (cache_control breakpoints). A cache hit prices cached input at about 10%. More important here, it cuts time to first token a lot, because the provider skips re-prefilling the transcript. Our sessions fit this well: a stable system prompt, then tool contracts, then an append-only transcript.

Check what the Pi SDK already does per provider: whether it sets cache breakpoints for Anthropic, and whether usage's cacheRead (already recorded in RunUsage) shows hits in prod. If cacheRead is about 0 on Anthropic providers, this is an easy saving.

Breakpoint placement: end of system prompt, end of tool definitions, and a moving breakpoint at the second-to-last turn. Keep the prefix byte-stable (see lever 4).

The Responses API supports store: true + previous_response_id. The client sends only the new turn and the provider keeps the context. Per-turn upload drops from O(session) to O(delta). That is the biggest single win for the large dev sessions that produced 91 GB of egress in 33h. Pi SDK support still needs checking. Fall back to full replay where it is not supported.

Old tool results are the dead weight. A 64 KB exec output that stopped mattering three turns ago is re-serialized, re-uploaded, and re-prefilled every turn, and on a cache miss it breaks the prefix. Elide or summarize tool results older than N turns, and leave a model-visible [output elided] note. This cuts bytes and event-loop serialization time. It also makes prompt-cache prefixes shorter to rebuild after a miss. The append-time spill of oversized outputs (a follow-up from the finished perf-squeeze plan) does the same job on the storage side. Session continuation is a separate approach that starts a fresh session.

Prompt caching only pays when the prefix is identical across turns. Hazards to audit:

The plan-state block and the <background_work_update> / <concurrent_user_messages> synthetic messages are appended with timestamp: Date.now(). If timestamps end up in the payload, every turn breaks the cache. Pin or strip them.

System-prompt resolution embeds remote hm:// docs (cached 5 min). A re-fetch that changes bytes mid-session breaks the whole cache. Consider pinning the resolved prompt for the session's lifetime.

Tool promotion (touch-expand) changes the tool list mid-session. That can't be avoided when it happens. Tool definitions should sit in their own cache segment, so a promotion only invalidates from that point on.

provider.request_gap measures everything before the request leaves. Known costs in that window: #piMessages decoding the full CBOR transcript every turn, building replay messages, and Pi session assembly. The decode grows with session length. The wire-truncation work capped what is stored, and did not cap what is replayed. If prod shows this gap growing with session length, the fix is an in-memory cache of the decoded transcript for each live session.

Each tool batch costs a full provider round trip on a growing context. Two behavioural levers need no protocol work:

Tool prompts should keep pushing agents to batch independent tool calls in one turn (parallel calls in one response), instead of one call per turn.

Long tool outputs where the model only needs a slice (big file reads, listings) should come back already bounded. The smaller each result, the cheaper every later turn. Output caps mostly do this already. Audit the stragglers.