🚀 pi-deepseek-cache

Squeeze the most out of DeepSeek's context caching inside the Pi coding agent.

Stable prompt prefixes · higher cache-hit rates · live cache stats — so long sessions pay the cache-hit rate on most of their input.

English | 中文

✨ Why this exists

DeepSeek's API has Context Caching on Disk built in: any request whose prompt prefix fully matches a previously persisted cache prefix unit is billed at the much cheaper cache-hit rate. The catch — your prompt prefixes must stay stable across turns, token-for-token.

In long agent sessions that's surprisingly hard:

• the system prompt or tool list changes subtly between turns
• conversation history grows and shifts
• large / non-deterministic tool outputs break the prefix
• repeated metadata blocks aren't aligned

pi-deepseek-cache keeps your DeepSeek prompts cache-friendly and shows you exactly how well it's working — on the official API and on any third-party deployment that serves DeepSeek behind an OpenAI-compatible endpoint.

🎯 Features

• Prefix Guard — strips volatile-scratch messages from the context to keep the prompt prefix stable across turns
• Stable Tool Ordering — lexicographically sorts tools in before_provider_request (Harness orderTools parity) so tool-list order jitter never breaks the cache
• Cache Break Diagnostics — fingerprints the full system + tools + messages prefix (stable-JSON + SHA-256) and warns only when an existing prefix is rewritten, not on normal append; returning to any still-live earlier prefix counts as a valid hit; empty/unknown payloads are silently skipped
• Hit Rate Telemetry — accumulates cacheRead / input / cacheWrite / turns per model from every response and persists to disk, so switching models never blends two different price points into one number
• Live Status Bar — see the current model's cumulative hit rate and turn count in the Pi footer after every message
• ASCII Trend Chart — visualize the all-models-cumulative hit rate over time with /cache-graph (flat-rate handling + fixed chart labels)
• Cost Savings Estimation — savings priced from the active model's cost in the model catalog (models.json / Pi's built-in catalog), which is the correct source for both official and third-party deployments. Only for the official deepseek provider does the official Models & Pricing table and its peak/off-peak windows still apply; a model the catalog prices at zero shows — instead of an invented number. /cache-stats shows cumulative and per-request hit rates, the model, where the price came from, the unit prices used, and a per-model breakdown
• Cache-Friendly Compaction — summarizes at temperature 0 with official deepseek-flash (legacy deepseek-v4-flash fallback) when that model exists, otherwise with the current model when it is already a flash-tier model, otherwise with a flash-tier model of the current provider, otherwise the current model; results are SHA-256–cached and persisted across sessions (LRU capped at 64). Note: this is a standalone cold call, not a verbatim replay of the routed request — see §7.4
• Atomic Persistence — tmp + rename writes + debounced async flush + session_start/session_shutdown lifecycle guarantees, no torn JSON
• /cache-reset — clear all stats, history, and summary cache with one command

📦 Installation

Requires Pi and Node.js ≥ 18.

pi install npm:pi-deepseek-cache

Or install from git:

pi install git:github.com/ruanbw/pi-deepseek-cache

🚦 Quick start

1. Make sure a DeepSeek provider is configured (DEEPSEEK_API_KEY set).
2. Select a DeepSeek model such as deepseek/deepseek-v4-flash.
3. Start coding — the extension activates automatically and reports cache stats in the footer.

export DEEPSEEK_API_KEY=sk-...
pi --model deepseek/deepseek-v4-flash

Using a third-party DeepSeek deployment (OpenRouter, a self-hosted endpoint, a local router)? No DeepSeek API key or deepseek provider is required — see the next section.

🔌 Third-party DeepSeek deployments

Every hook is provider-agnostic: telemetry reads usage.cacheRead / usage.input from whatever the endpoint reports, and the prefix guard plus the break diagnostics operate on the outgoing wire payload. So a DeepSeek model served by a self-hosted vLLM/SGLang endpoint, a hosted router, or a local gateway works too — the extension does not care which provider id you put in front of it.

What differs from the official API:

Two caveats specific to third-party hosts:

• Instance affinity. Self-hosted engines usually keep the prefix cache per instance, so a cache miss can mean "the router sent this turn to a different replica", not "the prefix was rewritten". The break diagnostics cannot tell those apart. If your router can pin a session to one backend (magpie does this through compat.sendSessionAffinityHeaders), keep that on.
• Billing follows the host, not DeepSeek. Hosts differ on how cached input is billed — flat discount, free, or not discounted at all. Savings are estimated from the catalog price you configured, so keep cost.input / cost.cacheRead in models.json accurate: for non-official providers that is the only pricing input the extension trusts.

// ~/.pi/agent/models.json — a third-party DeepSeek deployment
{
  "providers": {
    "my-router": {
      "api": "openai-completions",
      "baseUrl": "https://my-router.example/v1",
      "apiKey": "...",
      "models": [
        {
          "id": "deepseek-v4.1-flash",
          "name": "DeepSeek V4.1 Flash (my-router)",
          "contextWindow": 1000000,
          "maxTokens": 384000,
          "cost": { "input": 0.15, "output": 0.28, "cacheRead": 0.003, "cacheWrite": 0 },
        },
      ],
    },
  },
}

pi --model my-router/deepseek-v4.1-flash

🧩 Commands

🔍 How it works

📖 Deep dive: Prefix Cache Principle — official Context Caching on Disk (SWA-based independent cache prefix units, 3 persistence timings, best-effort) + Harness 5-layer enforcement + Pi mapping

🛠️ Troubleshooting

• Cache hit rate is low → usually a changing static prefix. Avoid injecting timestamps, random IDs, or volatile tool output near the start of the prompt. Note that adding or removing an extension changes the tools block, which invalidates the prefix just as a history rewrite does.
• "Cache prefix change" warning → something in the earlier conversation history, the tool set, or a tool's schema was modified. Check if a tool or extension is mutating past messages or registering/unregistering tools between turns. On a third-party host this can also mean the router moved you to another replica (see instance affinity).
• Footer shows nothing → confirm a DeepSeek-serving model is selected and its provider has working credentials.
• Hit rate stays at 0% on a third-party endpoint → that endpoint reports no cached-token count in usage; the extension has nothing to measure. Check a raw response, not the extension.
• Estimated savings shows — → the model has no usable cost.input in the catalog (free / unpriced model). Add cost to the model entry in models.json to get a number.

🧪 Test

npm test              # 80 tests (70 unit + 10 integration)

🤝 Contributing

Issues and PRs welcome! Please run npm test before submitting.

npm run lint          # ESLint + Prettier check
npm test              # Unit + integration tests

📄 License

MIT © ruanbw