pi-cache-stats

English · 中文文档

A Pi Agent extension that observes prompt-cache hit rate and diagnoses prefix changes. It does not freeze, reorder, or strip the system prompt or provider payloads.

Pi Agent has a solid event system and extension API. Provider prompt caching still depends on a stable rendered prefix, provider routing, TTL, and workload. This extension records usage with Pi's net-input formula and can show a compact TUI footer. It does not claim to create caching where the provider already has none, and it does not silently drop new system rules, skills, or project-instruction boundaries.

Development conventions

Detailed development and contribution guidelines are maintained across:

• AGENTS.md: Agent collaboration rules, real build/test commands, and repo index pointers.
• .agents/notes/README.md: Decision notes triggering conditions, exemption rules, and maintenance guide.

How it works

1. Safe observation (default)

Each factory instance keeps its own stats, enable flag, footer handle, and unclassified-400 list. There is no module-global golden prompt and no rollback of systemPrompt.

On every turn the incoming systemPrompt is kept unchanged. New tools, resource discovery, prior-extension rules, and later-extension edits all remain visible to the model.

The extension does not insert or overwrite prompt_cache_key, does not rewrite tools/messages/system TTL, and does not change PI_CACHE_RETENTION on import, enable, or disable.

session_start (new / resume / fork / switch / reload) resets this instance's current-run counters. Historical session custom entries are not replayed.

2. Unclassified 400s

A 400 with only status/headers is recorded as unknown. Subsequent cache strategy is unchanged. The extension API does not expose the response body, so the rejected parameter is not guessed.

3. Cache statistics

Per-turn cacheRead / cacheWrite / input is recorded while enabled. /cache-stats shows totals. Pi's usage.input is net input tokens across all APIs, so hit rate is cacheRead / (input + cacheRead + cacheWrite). Multi-turn aggregation is sum(cacheRead) / sum(denom). Per-turn detail is bounded; cumulative totals are not discarded.

/cache-stats prints two scopes so the numbers can never contradict the footer:

• Aggregate hit ... (current run; ...) — this process since session_start (the state.snapshot window used for per-turn reports).
• Session aggregate ... (all session, footer scope / post-reset window; ...) — the full session entries (or the post-/cache-stats reset window), the same algorithm the footer's ◆ cumulative rate uses. After /cache-stats reset, the footer only counts entries appended after the reset, and the label reflects that.

Per-turn detail shows the measured hit rate for each round. When the scanned session window never showed a cache interaction, Per-turn shows n/a (never a fabricated 0%). Once cache was seen in the session, a round with denom > 0 but no cache interaction shows 0%. A round with no usage or zero denominator still shows n/a.

String length in diagnostics is character count, not UTF-8 bytes or tokens.

4. Prefix-change diagnostics (default on, read-only)

At this extension's before_provider_request hook, a snapshot of the current payload is compared with the previous snapshot for the same instance + session + provider + API + model + endpoint. Endpoint isolation fingerprints the full baseUrl string in memory with a per-instance salt (protocol, host/port, path, query, and userinfo differences are kept, never printed). That fingerprint is not evidence of real backend routing. It reports only fixed categories (system / developer / instructions / tools) plus status (baseline / stable / changed / unknown / skipped / disabled) and length or count. Empty sections are distinct from missing or oversize sections; missing or oversize is never labeled stable. Tool-array order is compared as sent (not sorted). Untrusted payload fields are read only as own data properties: accessors are not evaluated, Proxy objects are skipped via Node util.types.isProxy without running traps, and payload Symbol.iterator / toJSON are not executed. History user / assistant / tool bodies are not read when diagnosing system / developer (role/type/position metadata only). Host ctx / model / sessionManager.getSessionId are called by contract; a host throw skips that observation, does not change the request, and does not leak the exception text. Variable strings that enter walking, fingerprinting, concatenation, or scope formation (including field names) are charged against explicit length and total-character budgets before expensive work; oversize or unsafe observations are never labeled stable. These are JSON-like observation limits, not a sandbox against arbitrary JavaScript.

Required identity must be present as non-empty strings within the length caps (ctx, model provider/api/id, session id, baseUrl). If any of those cannot be trusted, the observation is skipped, no scope summary is stored, and the comparison chain (max 8) is cleared so the next valid context starts at baseline. Missing/empty/non-string values are not coerced into a shared blank key or endpoint:none. Oversize or unsupported payload inside an already-known scope still breaks that scope only.

Section fingerprints keep request shape, string vs block-array representation, target-message count/order/original index/role, and content-block type/boundary/order. Each chunk mixes a fixed-width 32-bit UTF-16 length (low 16 bits, then high 16 bits, including a zero high half) before the code units; the separator stays mix(31). That digest is internal and non-authenticating: fixing length aliasing is not a proof that a bounded hash has no other collisions, and values are not persisted or used as a cache key. The recognized shape (openai-chat / openai-responses / anthropic) is stored as a bounded primitive on the comparison snapshot: two comparable snapshots in the same scope with different shapes are changed. Shape is not folded into scopeKey and is not inferred by rewriting ctx.model.api. Joined text is not the comparison key. Appending history after the last target message, or changing non-target bodies, does not by itself mark instructions changed. Mixed exclusive own fields (messages together with input or instructions, including input as string / empty string / null / own undefined) are unknown rather than dropping a field for a partial stable. Anthropic system+messages and Responses instructions+input (array or string) stay legal. unknown / skipped are not compared as a shape change; the next valid context is baseline. Vendor or model names are not used to guess the protocol.

This is not a cache-invalidation verdict, a hit-rate, or a token-savings number. It is not the final HTTP body (later extensions can still change the payload) and not the provider's token prefix or cache state. Character counts are not tokens. Stable comparable fields do not mean the full conversation was frozen. Offline CPU improvements are not cache hits or bill savings.

Known shapes are detected from payload structure used by this machine's Pi SDK (OpenAI chat messages system/developer, OpenAI Responses instructions / input system/developer, Anthropic system + tools). History, reasoning, and tool results are not collected. If exactly one well-formed unique Pi tagged block can be recognized inside already-extracted system text, an extra length summary is shown. Supported tag-like forms are only: exact <project_instructions> / </project_instructions>, or <project_instructions then ASCII whitespace (space/tab/LF/CR) plus attributes then >; and exact <available_skills> / </available_skills> (no attributes). The scan requires a single open-then-close pair in that order: a leading, between-block, or trailing isolated close, a second open, nesting, truncation, or an unsupported name suffix (. / : / non-ASCII / other) stays unidentified and is not claimed as a complete target block. This is not general XML. The extras scanner is a linear forward scan, not a backtracking regex, and does not rewrite the prompt.

Turn off with PI_CACHE_STATS_PREFIX_DIAGNOSTICS=0. /cache-stats disable and reset / session_start also stop updates and clear comparison state. Unknown or unsafe payloads are passed through unchanged. Observation never inserts cache keys, TTL, or retention, and never mutates the request object. Unclassified-400 and stats lines that embed provider/model fragments are control-character-sanitized and length-bounded for display only; cleaned text is not used as an identity key.

Pi-native provider behavior stays first. Unknown endpoints keep host/user configuration. This tree does not hard-code a vendor capability table, and offline tests are not evidence of server-side caching or bill savings.

5. Live TUI footer (default on)

When running in TUI mode, the footer renders stats calculated in real time at render-time (mirroring Pi's official footer behavior) rather than displaying static event snapshots:

• Render-time live calculation: Cache hit rate is computed dynamically during render() across all sessionManager.getEntries(), while context window usage is fetched via getContextUsage().
• Dual hit-rate display: Cache hit rate shows both cumulative and latest rates: ◆ <cumulative>% ◇ <latest>% (e.g. ◆ 87% ◇ 95%). The ◆ cumulative rate uses the default text color, while the ◇ latest round rate uses the warning color. When the scanned window never showed a cache interaction (cacheRead=0 and cacheWrite=0 for every entry, e.g. a provider without prompt caching), both items show n/a — never a fabricated 0%. Once cache was seen in the window, a later round that itself used no cache shows ◇ 0% (that round was measured, it just had no cache interaction), while a round with no usage field at all (interrupt / error / injected message) shows n/a instead of a stale previous value. The ◆ aggregate keeps its own stricter rule and additionally requires a computable denominator.
• Context occupancy colors: The ▲ context segment mirrors the official footer thresholds — >90% uses the error color, >70% uses the warning color, otherwise the default text color. Other footer segments are unaffected.
• Last-turn generation speed (▸ t/s): Between ▲ context and ■ model the footer shows an estimated speed for the most recent provider generation, ▸ <n> t/s (e.g. ▸ 45 t/s). TPS is measured over the streaming decode window (message_update first delta to done / message_end, excluding TTFT prefill and tool-execution latency, splitting out reasoning tokens when both thinking/text deltas and usage.reasoning are present, and guarded by four sanity checks) and falls back to the end-to-end before_provider_request -> turn_end window when no streaming delta events occur. It shows a dim n/a when timing stamps or assistant usage are missing, output is not a finite positive number, any sanity guard fails, or no turn has completed yet; reset / session_start clear all stamps so no value leaks across sessions.
• Refresh triggers: High-frequency events (turn_end, tool_execution_start, tool_execution_end) trigger throttled 16ms footer repaints, while low-frequency events (agent_end, model_select, thinking_level_select, /cache-stats reset) trigger immediate force redraws.
• Dirty-check caching: The live hit-rate scan is cached and only recomputed when the entry count changes; /cache-stats reset and session changes invalidate the cache (a reset watermark makes the footer count only entries appended after the reset, so it shows n/a until new usage arrives; if the entry list is ever shorter than the watermark a safe n/a fallback is used). Context usage is always fetched fresh.
• Single-entry accumulation: Token usage is accumulated at turn_end into a run-level liveRun buffer (settled directly at agent_end without re-scanning messages), preventing double-counting in command statistics.

Install

npm (recommended)

pi install npm:pi-cache-stats

Pi auto-installs and loads it. No settings changes required.

Direct copy

git clone https://github.com/dorokuma/pi-cache-stats.git
cd pi-cache-stats
bash integrations/pi/install.sh   # or: pnpm sync

install.sh is idempotent: when the deployed copy already matches the repo source (md5), it reports unchanged and does not rewrite the target (mtime untouched). It fails loudly instead of silently: a missing md5sum, an unset HOME (only when PI_EXT_DEST is not given), or a non-absolute / directory destination all print install.sh FAILED: ... to stderr and exit non-zero. Override the destination with PI_EXT_DEST=/abs/path/to/cache-stats.ts. Pi loads extensions at session start — run /reload or start a new session afterwards. This script only syncs the repo source into the local extension directory; the npm route above (pi install npm:pi-cache-stats) installs the published package and does not go through this script.

Environment variables

Note: The compactionCacheLoss field was removed. The Pi extension API has no reliable compaction event to accumulate it.

Commands

/cache-stats          # Show cache statistics (current run + all-session scopes, incl. a short prefix-status line)
/cache-stats prefix   # Show the latest prefix-change snapshot (categories/status/lengths only)
/cache-stats disable  # Disable this instance (no stats writes, no footer, no prefix updates)
/cache-stats enable   # Re-enable (footer only if TUI and FOOTER is on)
/cache-stats reset    # Reset current-run statistics, unclassified 400 list, and prefix comparison state; reset the footer live view to post-reset entries only; refresh footer

Tests

npm test

npm test is offline: it loads the extension, drives Pi events, runs an isolated AgentSession tool-set path, and self-checks comparison helpers. It does not read ~/.pi/agent, user credentials, or call a model API.

To exercise the host SDK instead of the project pin (0.84.2):

PI_CACHE_STATS_TEST_SDK=/path/to/@earendil-works/pi-coding-agent npm test

A real network A/B is opt-in only (never part of npm test):

PI_CACHE_STATS_BENCH=1 \
PI_CACHE_STATS_BENCH_PROVIDER=... \
PI_CACHE_STATS_BENCH_MODEL=... \
PI_CACHE_STATS_BENCH_AGENT_DIR=/explicit/isolated/agent-dir \
node test/comparison.mjs

See docs/testing-methodology.md and docs/audit-remediation.md.

References

• OpenAI Prompt caching: prefix matching, routing keys, retention, and cost
• Anthropic Prompt caching: cache blocks, TTL, cost, and order constraints
• DeepSeek Context caching: automatic prefix caching (best-effort)

Commit conventions

Commit messages must follow Conventional Commits (TYPE: subject or TYPE(scope): subject):

• Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore, revert
• Scope: Optional ([a-z0-9._-]+)
• Subject: Non-empty, ≤ 72 characters (supports mixed English and Chinese)
• Exemptions: Merge, revert, fixup!, and squash! commits
• Validation: Enforced locally by the commit-msg hook with secret scanning and noise-word checks.