pi-for-k3

Kimi K3 prefix-cache optimizer for the pi coding agent

Byte-stable request prefixes · per-turn miss attribution · K3-tuned system-prompt constraints

~43% cheaper than bare pi and ~56% cheaper than Kimi Code at equal resolve (SWE-bench Verified Mini)

---

Kimi K3 bills cached prompt tokens at a tenth of the normal price — but only if every request keeps the exact same byte prefix. One shuffled tool, one rewritten message, and you silently pay full price again.

pi-for-k3 is a single-file pi extension (k3-cache.ts) that keeps you on the cheap side. It watches every request, keeps the prefix byte-stable, tells you exactly why any cache miss happened, and makes cost-aware calls on things like compaction and context-window upgrades. It works on both K3 channels (Moonshot pay-per-use API and the Kimi Code subscription), and if you're on any other model it simply stays out of the way.

Contents — Install · System-prompt constraints · Commands · Configuration · Guarantees

Install

pi install npm:pi-for-k3   # or: pi install git:github.com/WuP1ao0/pi-for-k3@v0.2.0
pi -e git:github.com/WuP1ao0/pi-for-k3               # try without installing

System-prompt constraints (K3-specific)

K3 is a strong model with one documented bad habit: it does too much when it isn't sure. Kimi's own docs recommend reining it in with explicit behavioral constraints, so on K3 sessions this extension appends a short <k3-cache-constraints> block to pi's system prompt. It never touches pi's own prompt text, and the block is byte-identical every turn, so the cache stays safe.

The block draws four simple lines:

• Scope — fix what the issue asks for; don't refactor, rename, or "improve" things nearby.
• Completeness — before finishing, check whether the same behavior lives elsewhere (a subclass, a parallel handler) and fix all of it.
• Verification — once something passes, leave it alone unless you can name a concrete failing case.
• Economy — don't dig through the whole git history; the issue text and current code are enough.

These aren't just vibes — in an A/B test on a task that baits the model into git archaeology, the full block solved it in 14 turns for $0.25, while dropping the Economy line ballooned the same task to 40 turns and $0.53.

Commands

Command	What it does
/cache	Shows how your session is doing: hit rate, actual cost vs what you'd pay with no cache, any drift alerts, and the reason behind every miss
/effort <level>	Changes the reasoning effort. Asks you to confirm first, because Kimi's docs say the server cache rebuilds after a switch
/k3upgrade	On the subscription: jumps from k3-256k to k3 for the 1M window. Your cache survives the jump (an official exception), but quota burns about twice as fast

Configuration

Everything is tunable. Drop overrides into ~/.pi/agent/k3-cache.json — you only need the keys you want to change:

Key	Default	What it controls
enabled	true	The master switch. Set false and the extension does nothing at all
hitRateAlertThreshold	0.7	If a turn's hit rate falls below this, you get a warning with the reason
segmentSizeChars	16384	How finely the prefix is hashed for drift detection (~4K tokens per segment)
idleTtlMs	300000	If you've been idle longer than this, the next miss is blamed on server-cache expiry rather than something you did
defaultEffort	high	Reasoning effort to lock in when pi doesn't have one set
subscriptionWarnFraction	0.8	Warns you when context grows past this share of the 256K window
overflowUpgrade / overflowAutoContinue	true	When k3-256k overflows, switch to k3 (1M) instead of compacting — your cache survives an upgrade but not a compaction. Also auto-resumes the interrupted turn
highWaterFraction	0.85	Holds off pi's automatic compaction until context is really full, since every compaction throws away the cached prefix
summaryChainEnabled / MaxLayers / MaxTokens	true/5/8000	When compaction does happen, add a new summary layer instead of rewriting old ones (and when to merge layers)
prewarmEnabled / prewarmIdleMs	true/300000	After a long break, quietly re-warm the server cache with one tiny request so your first real turn doesn't pay the miss
externalizerEnabled	false	Park oversized tool results on disk and let the model fetch them back with a read_result tool. Off by default; the gentler head/tail truncation always runs
externalizeMinChars / onDemandTools	8192/["bash"]	Only touch a tool result if it's bigger than this AND comes from one of these low-reuse tools
toolChangeAllowlist	[]	Tools that are allowed to appear and disappear mid-session without a warning. Globs work: "goal_*"
compactionOwnership	"k3cache"	Set "external" if another extension manages compaction and this one should stay out of it
k3Models	catalog map	Which provider/model ids count as K3. Everything else is ignored

Guarantees (cache discipline)

A few promises about how this extension behaves:

• It never rewrites your history. Messages and payloads are left untouched; the only in-place change is putting tools in a stable order. Everything else is watching and warning.
• Tool-set changes don't spam you. A removed or reshaped tool gets flagged once per distinct change. If the same change keeps recurring (some extensions toggle tools by design), you get a ready-to-paste toolChangeAllowlist line instead of endless alerts.
• Every expensive turn comes with a reason. Misses are labeled: first turn, prefix drift, model switch, idle expiry, compaction, branch navigation, system-prompt rebuild, tool change, or a genuine server-side miss.
• Drift alerts point at the culprit. Not just "something changed" — you're told which message, tool, or prompt section diverged.
• Sessions don't leak into each other. State is kept per session, so subagent runners sharing one process can't trigger false alarms. /cache also shows a combined total when several sessions are active.