pi-jev-prune

This extension is not ready for general usage and don't recommend adding it to your main Pi workflow yet. Rather than just hype, we did some actual experiments to figure out if this thing actually works.

pi extension that prunes stale tool results from context using TypeSafe's Jev. Before each LLM call it asks Jev which completed tool outputs the agent won't need again and replaces them with a recoverable stub:

[pruned: read src/resolve.rs — 734 lines/~8k tok. Use recall("toolu_01…") to restore.]

Published as a reference implementation. Works, tested, rough edges. Run in dry mode first.

• Ephemeral — session file untouched, stubs applied in the context hook only.
• Tool results only — never touches user/assistant messages or tool calls.
• Cache-aware — prefix changes only at windows (user turn, budget pressure, cold cache); mid-run prunes are batched into one rewrite.
• Recoverable — recall(toolCallId) returns the original verbatim.
• Fails open — any error, timeout, or missing key → context unchanged.

Results

Visual report (plain-language, with the real Jev request and every chart): fsmiamoto.github.io/pi-jev-prune. Full write-up: experiments/REPORT.md. Offline replay of 14 sessions + 13 live runs on a small Rust repo (Sonnet 4.5).

• Jev's signal is in the low tail: below p 0.15, 1 of 28 outputs was used later; 0.15–0.30 ≈ 1 in 4, the same as random (24 % base rate). A cutoff of 0.20 is half as wrong as random, 0.35 ≈ random. Default threshold 0.25.
• Live: 0 recalls, 0 hallucinated identifiers in 13 runs. Peak context −15 % on a 3-task session; the benefit compounds across turns.
• Every applied prune is a prompt-cache rewrite. Unbatched it was net-negative in cost; batched it's ≈ break-even. The win is headroom, not money.
• A no-model rule (read later superseded by edit/read of the same file) did much of the useful pruning.
• Jev latency: p50 300 ms, max 1.1 s per 10–30 candidates, 0 timeouts at 2 s.

Install

pi install npm:pi-jev-prune

The installer registers the package in ~/.pi/agent/settings.json. Keep dry mode for initial use:

{
  "packages": ["npm:pi-jev-prune"],
  "jev-prune": { "mode": "dry" }
}

Export TYPESAFE_API_KEY. Requires pi ≥ 0.86.

Config

Usage

• /prune status · /prune show · /prune dry|on|off|every-call · /prune now (force a window)
• recall(toolCallId) — tool the model calls to restore a pruned output
• Footer: prune: 12k saved · 3k staged · 4k pending · 56k/100k
• Decision log: ~/.pi/agent/jev-prune/log.jsonl

How it decides

1. Candidates: tool results older than exemptSteps, ≥ minTokens, no sticky decision yet.
2. Code rule: a read followed by a later read/edit/write of the same path → auto-prune.
3. Window check: only consult Jev (and change stubs) at user turn / over budget / pressure / cold cache / big pending backlog. Otherwise re-apply previous stubs → byte-identical prefix.
4. Jev: one Noul per candidate over { task, now, candidates[{tool, arg, turnsAgo, sizeTokens, head, tail, agentReaction}] }: "Will the agent need the full output again to finish task, given what it is doing now?" → prune if p < threshold.
5. Mid-run prunes are staged until they save ≥ minApplyPct × budget, then applied together. Decisions are sticky and persisted as session entries keyed by toolCallId.

Limitations

• Template stubs, no summaries. If the agent needs it, it calls recall (one extra step).
• Jev sees head/tail/reaction only (~600 chars) — good at "obviously irrelevant", not "subtly still needed".
• Small repos with "survey everything, then act" tasks → Jev is conservative and little gets pruned.
• Not integrated with pi's compaction or pi-context (they see un-stubbed messages).

Development

npm test            # typecheck + unit tests
npm run test:live   # one real Jev request (needs TYPESAFE_API_KEY)

Experiments: experiments/replay.ts (replay your own sessions; JEV_PRUNE_SESSIONS='--Users-me-Code-repo--,…', optional JEV_PRUNE_EXCLUDE='pattern'), experiments/live.ts (headless A/B on a repo clone), experiments/analyze.ts.

Not yet tried: ingest-time chunk filtering of large outputs (no cache cost), apply-only-when-cache-is-cold, a "delegate to subagent" nudge.

Maintainer releases

CI runs typechecking and unit tests on every push and PR. Release publishing uses Node 24 and npm ≥ 11.5.1; both workflows install current npm 11 explicitly (early Node 24 releases bundled older npm).

One-time setup:

1. If the npm package does not exist yet, bootstrap it locally with Node 24 and npm ≥ 11.5.1:

   npm login
   npm ci
   npm test
   npm publish --access public

2. In npm's package settings → Trusted Publisher, select GitHub Actions and enter:
   • Organization/user: fsmiamoto
   • Repository: pi-jev-prune
   • Workflow filename: publish.yml (not the full path)
   • Environment: leave blank
   • If shown, allow direct npm publish in allowed actions.
3. No NPM_TOKEN or other npm secret is needed. Trusted publishing uses OIDC on GitHub-hosted runners (ubuntu-latest here); self-hosted runners are unsupported. Keep package.json's repository.url matching this repository. Public repository/package publishing gets automatic provenance.
4. After verifying OIDC publishing works, npm recommends Require two-factor authentication and disallow tokens in Publishing access.

Each release: bump package.json and package-lock.json to a new, unpublished version (including after bootstrap), commit reviewed changes, and push them with a tag exactly matching v<package.json version>, e.g. v0.1.1. Publish a GitHub Release for that tag. The workflow checks the tag, installs dependencies, and runs npm test before npm publish --access public.

Tag pushes and draft releases do not publish. Published prereleases also trigger this workflow and use npm's default latest dist-tag; this workflow does not select a separate prerelease channel.

MIT