pi-fast-jev-compaction

Fast, verbatim, jev-guided compaction for pi.

This is a pi version inspired by fast-jev-compaction, the Claude Code plugin by tamaratran. The algorithm, the jev prompts and the core implementation are theirs (MIT); this package adapts them to pi's compaction pipeline, adds chained re-pruning, a debugging mode and benchmarks. See Credits.

Open the MP4 demo

Why

pi's built-in /compact asks an LLM to rewrite old turns into a summary. That takes 15 to 50 seconds and is lossy: a file path, an exact error, a constraint the user stated once or a command line can disappear even when it matters later.

/compact-jev never rewrites anything. It shows TypeSafe's jev model the whole conversation and asks it, for every tool call in the part pi is about to discard, two probability questions:

1. does knowing this call was made, with its input, still matter?
2. does its full output still need to stay verbatim?

Calls jev judges stale are removed with their output; calls whose output is stale keep their first 300 characters plus a note; everything else, including all user and assistant text, stays verbatim and in order. That pruned transcript becomes pi's compaction "summary". pi's recent window (compaction.keepRecentTokens) is untouched as usual.

Full numbers: references/benchmarks.md.

Install

pi install npm:@rossz/pi-fast-jev-compaction

Or straight from GitHub, or try it without installing:

pi install git:github.com/ross-jill-ws/pi-fast-jev-compaction
pi -e npm:@rossz/pi-fast-jev-compaction

For local development, point pi at the checkout:

pi -e /path/to/pi-fast-jev-compaction/extensions/index.ts
# or in ~/.pi/agent/settings.json:  { "packages": ["/path/to/pi-fast-jev-compaction"] }

Requirements

• pi 0.84 or newer.
• A TypeSafe API key in TYPESAFE_API_KEY. Without it /compact-jev refuses to run and pi's default compaction is unaffected.

export TYPESAFE_API_KEY="$(cat ~/.typesafe/<your-key-file>)"

Usage

Instructions are passed to jev as part of its goal and to pi's summarizer as custom instructions.

Plain /compact and automatic compaction keep using pi's default unless PI_JEV_COMPACT_AUTO=1.

Every run tells you where the time went:

jev compaction: 13 calls removed, 1 results truncated, 0 kept; frees 97.1% of the replaced context;
  took 0.7 s (jev 0.7 s in 1 request). jev verbatim summary: ~1,850 tokens (97.1% freed)
compaction done in 0.7 s: 101,686 → ~24,890 tokens

Configuration

Everything is an environment variable.

Benchmarks

Measured against real pi processes (0.84.2 and 0.85.1) over its RPC protocol, with the real TypeSafe API. Compaction time is what the user waits for, from compaction_start to compaction_end. Method, fixtures and every table: references/benchmarks.md.

Speed

The jev round trip is the whole cost: preparing candidates and applying decisions took 0 to 2 ms in every run. The vendored core is as fast as the Claude Code plugin's on identical input (0.26 to 0.67 s versus 0.28 to 0.64 s over three runs each, one request).

Size and content

Keep-threshold sweep on the real session: at 0.5 jev keeps 0 of 172 calls and frees 90%; at 0.15 it keeps 84, truncates 79, removes 9 and frees 28.5%.

Debug mode

PI_JEV_COMPACT_DEBUG=1 writes a bundle per compaction under <cwd>/.pi/pi-fast-jev-compaction/ with the report, the exact jev state and answers, per-call decisions with both probabilities, the transcript, and pi's default summary of the same input. The default summary is computed in the background after /compact-jev has returned, so debugging never slows the compaction; only /compact-jev-compare waits for it. Plain /compact results are captured too, so the two can be compared on identical input.

Details, the bundle layout and how to read a report: references/debug-mode.md.

How it works

1. pi fires session_before_compact with the messages it wants gone (plus a split-turn prefix when the cut lands inside a turn). The extension converts them with pi's convertToLlm, replaces images with [image omitted] and drops thinking blocks.
2. If the previous compaction on the branch was also jev's, its stored pruned messages are prepended and re-evaluated, so chained compactions never stack summaries. A previous pi-generated summary is carried along verbatim instead.
3. jev's goal is the original task plus the last substantive prompts and any instructions. The state is fitted into 25k estimated tokens the way the plugin does it, and the questions go out in as many concurrent requests as needed.
4. Decisions are applied to pi's messages; the transcript is rendered without truncating kept results, with pi-style <read-files> / <modified-files> lists covering everything replaced.
5. The compaction is returned to pi, or nothing is returned and pi's default runs when jev failed or the reduction is below the minimum.

The full walk-through and a comparison with the Claude Code plugin: references/how-it-works.md.

Good to know

• At the default 0.5 threshold jev dropped almost every old tool call in every session tested, real or synthetic (probabilities were mostly 0.1 to 0.4 for calls and under 0.2 for results). In practice the transcript is the user and assistant text of the old history. Lower PI_JEV_COMPACT_KEEP_THRESHOLD if you want tool output retained, and expect a much smaller reduction.
• Verbatim text is larger than an LLM summary. For text-heavy histories the 25% minimum correctly hands over to pi's default; force overrides it.
• pi wraps the transcript in its usual "compacted into the following summary" framing, so the model sees a transcript inside <summary> tags rather than real messages.
• Compaction entries are tagged details.kind = "pi-jev-compact", the extension's original name, so sessions compacted before the rename keep re-pruning correctly.

Development

extensions/index.ts     commands, session_before_compact / session_compact hooks, fallback, debug bundles
extensions/adapter.ts   pi messages <-> jev messages, normalization, applying decisions, file lists
extensions/render.ts    the verbatim transcript and summary text
extensions/goal.ts      jev's goal from the user's prompts
extensions/debug.ts     bundle writer and report
extensions/config.ts    environment variables
extensions/jev/         the fast-jev-compaction core, vendored (see Credits)
references/             how-it-works.md, debug-mode.md, benchmarks.md
tests/                  unit tests with a fake jev, no network

node --test tests/*.test.ts

Type-check against the installed pi runtime rather than a pi-mono checkout, whose dist may be older than its source: map @earendil-works/* in paths to the runtime's dist folders.

Credits

• fast-jev-compaction by tamaratran: the idea of verbatim, jev-guided compaction, the two-question formulation, the state fitting and the core implementation. extensions/jev/ is that project's src/ with small changes (abort signal, usage accounting, paired calls returned with the result, a pi-specific truncation note). MIT, notice reproduced in LICENSE.
• TypeSafe for the jev model.
• pi by Mario Zechner for an extension API that lets a compaction be replaced in a few dozen lines.

License

MIT. See LICENSE, which also carries the fast-jev-compaction notice for the vendored code.