@tallshort/pi-context-prune-manual-plus

Fork of championswimmer/pi-context-prune, maintained by tallshort.

A Pi coding-agent extension that summarizes completed tool-call batches, prunes raw tool outputs from future LLM context, and preserves the originals for on-demand recovery.

See CHANGELOG.md for release notes.

What this fork adds

The primary enhancement is safe, observable manual on-demand pruning through /pruner now, so you can explicitly initiate pruning when using on-demand mode.

Manual /pruner now enhancements

• Preview first: /pruner dry-run reports pending batches, threshold skips, and an estimated saving without provider calls or session changes.
• See each batch: a centered live overlay reports per-batch progress, received summary characters, retries, and a sanitized terminal failure reason.

Example manual progress overlay

┌──────────────────────────────────────────────────────────────────────────────┐
│ Pruner Now (4/47 complete · 8 running)                                       │
├──────────────────────────────────────────────────────────────────────────────┤
│ ⚠ Batch 1/47 · 1 tool call · skipped                                         │
│ ⠋ Batch 2/47 · 4 tool calls · 133 summary chars / 31.2k raw chars            │
│ ⚠ Batch 3/47 · 1 tool call · skipped                                         │
│ ⠋ Batch 4/47 · 1 tool call · 133 summary chars / 3.3k raw chars              │
│ ⠋ Batch 5/47 · 1 tool call · 140 summary chars / 1.0k raw chars              │
│ ⠋ Batch 6/47 · 1 tool call · 68 summary chars / 930 raw chars                │
│ ⠋ Batch 7/47 · 1 tool call · 22 summary chars / 5.1k raw chars               │
│ ⠋ Batch 8/47 · 1 tool call                                                   │
│ ⚠ Batch 9/47 · 1 tool call · skipped                                         │
│ ⚠ Batch 10/47 · 1 tool call · skipped                                        │
│ ⠋ Batch 11/47 · 1 tool call                                                  │
│ ⠋ Batch 12/47 · 1 tool call                                                  │
│ ○ Batch 13/47 · 1 tool call · pending                                        │
│ ○ Batch 14/47 · 1 tool call · pending                                        │
│ ○ Batch 15/47 · 1 tool call · pending                                        │
│ ○ Batch 16/47 · 1 tool call · pending                                        │
├──────────────────────────────────────────────────────────────────────────────┤
│ Esc: stop scheduling new batches; in-flight batches will finish              │
└──────────────────────────────────────────────────────────────────────────────┘

• Control throughput: configure 1–16 simultaneous summary calls instead of using an unbounded automatic fan-out.
• Stop safely: Esc stops scheduling new batches while in-flight work finishes; completed results remain available for settlement.
• Handle transient failures: eligible provider failures retry once. When an earlier batch fails, later successful summaries are reused in the same process instead of calling the provider again.
• Respect provider limits: a shared 5–60-second rate-limit cooldown blocks new provider requests across every trigger while preserving pending batches.

This fork also adds:

• minimum raw-character thresholding, cumulative statistics, and detailed normalized failure reporting;
• stable prune-frontier handling across compaction and recovery.

The upstream project remains the foundation; PRUNING.md is retained as its original algorithm and research-background document. See the original repository for its history and general pruning design.

---

Why

📖 For a deep dive into how pruning works, how prefix caching interacts with it, and the research behind summarization-based context management, see PRUNING.md.

As long agent sessions grow, every tool call adds token-heavy output to the context window. Most of it is not needed verbatim after the first use. This extension:

1. Captures completed tool-result batches from turn_end, and re-scans the current session branch when a flush runs so unsummarized results can still be picked up
2. Summarizes those batches using your configured model when the selected trigger fires
3. Stores a compact hidden summary message either as a runtime steer or a session custom message, depending on the flush path
4. Prunes the original verbose tool outputs from future context (context event)
5. Preserves every original output in a session-backed index — retrievable at any time via context_tree_query

The extension does append its own custom summary/index/frontier/stats entries to the session, but it does not rewrite or delete the original tool-result messages. Pruning only changes how future request context is assembled.

Installation

Install from npm (after the first release)

The package has not been published to npmjs.org yet. After the first release, install the stable, versioned package with:

# Install globally (all projects)
pi install npm:@tallshort/pi-context-prune-manual-plus

# Or install for the current project only
pi install -l npm:@tallshort/pi-context-prune-manual-plus

Once installed, the extension is auto-loaded every time you run pi. Re-run the command to upgrade to a newer npm release.

Install from GitHub (cutting-edge / main branch)

If you want the latest unreleased changes from main, install directly from the git repository:

# Install globally (all projects)
pi install git:github.com/tallshort/pi-context-prune-manual-plus

# Or install for the current project only
pi install -l git:github.com/tallshort/pi-context-prune-manual-plus

Note: Until the first npm release, install from GitHub. The main branch may contain unreleased or experimental changes.

Try without installing

# Load the latest from git for this session only (no install)
pi -e git:github.com/tallshort/pi-context-prune-manual-plus

From source (development)

git clone https://github.com/tallshort/pi-context-prune-manual-plus
cd pi-context-prune-manual-plus
pi -e .

Manage installed extensions

pi list                      # show installed packages
pi remove @tallshort/pi-context-prune-manual-plus

Prune-On Modes

The extension supports five trigger modes controlling when summarization and pruning happen.

Cache-aware guidance

This extension rewrites the future request context by replacing old raw toolResult messages with a compact summary. That saves tokens, but it also changes the prompt prefix seen by the model.

On providers with prefix / prompt caching (for example Anthropic-style prompt caching), cache hits require the earlier prompt prefix to stay identical. If you keep changing earlier context, the provider has to recompute from the point of change onward, which means higher latency, higher input cost, and fewer cache hits. In other words: pruning too often can save tokens in-context while still hurting overall performance by repeatedly busting the provider cache.

That is why agent-message is the default: it batches a whole stretch of tool work, prunes once when the agent is done and sends a final text reply, and then leaves the new shorter context stable again. You usually pay one cache bust per meaningful work batch instead of one cache bust per tool turn.

References:

• Anthropic prompt caching docs: https://docs.claude.com/en/docs/build-with-claude/prompt-caching
• AWS Bedrock prompt caching overview: https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html
• pi-context extension (context_checkpoint, context_timeline, context_compact; legacy names context_tag, context_log, context_checkout): https://github.com/ttttmr/pi-context

Mode trade-offs

Mode	Trigger	Pros	Cons / cache impact	Recommendation
every-turn	Immediately after each tool-calling turn	Smallest raw context as fast as possible; easiest to reason about	Busts prompt cache the most often because earlier context is rewritten after almost every tool turn; adds summarizer latency every turn; can cost more overall despite saving context tokens	Debugging only. Useful to test the extension, inspect summaries, or study behavior — not recommended for normal day-to-day use
on-context-tag	When context_checkpoint is called	Lets you align pruning with explicit milestones / save-points; fewer cache busts than every-turn if you tag sparingly	Only auto-triggers if you have the pi-context extension installed, because that extension provides the context_checkpoint tool (legacy name context_tag is still recognized); if you tag too often, you still churn cache; if you forget to tag, pending batches keep growing	Good if you already use pi-context and think in checkpoints / milestones
on-demand	Only when you run /pruner now	Maximum manual control; easiest mode for preserving cache because nothing changes until you decide; good for long investigations where you want to delay pruning	Easy to forget; pending batches can grow large; you must manage timing yourself	Good for advanced users who want explicit control over when the cache is intentionally invalidated
agent-message	When the agent sends a final text-only response	Best balance of automation, context savings, and cache friendliness; batches many tool turns into one prune; after the prune, future requests become highly cacheable again until the next batch finishes	You do not reclaim space mid-batch; if a run goes extremely long before the final reply, context can grow more than in aggressive modes	Recommended default. Safest general-purpose mode for normal coding-agent workflows
agentic-auto	The model decides by calling context_prune	Lets the agent compact context before it gets too large; can work well for long autonomous runs when the model is disciplined	Depends on model judgment; if the model calls context_prune too often, it can churn cache similarly to every-turn; behavior is less predictable than agent-message	Good for longer autonomous sessions after prompt-tuning and observation

How each mode works

every-turn — Every tool-calling turn is summarized and pruned immediately. This is intentionally aggressive. It is useful for debugging the extension or validating summaries, but in real work it usually rewrites the prompt prefix too frequently and hurts provider-side prompt caching.

on-context-tag — Tool-call turns are queued until context_checkpoint is called, then all pending batches are summarized in one LLM call and pruned together. This mode is meant to pair with the pi-context extension; without that extension, context_checkpoint is not available, so this mode will not auto-trigger unless you switch modes or flush manually with /pruner now. The legacy tool name context_tag from older pi-context versions is still recognized.

on-demand — Tool-call turns are batched but never summarized automatically. You decide when to flush with /pruner now. This is the most manual mode and also the easiest to keep cache-friendly, because you can wait until a large chunk of work is complete before changing earlier context.

agent-message — Tool-call turns are batched. When the agent finally replies with a normal text answer (a turn with no tool calls), all pending batches are summarized and pruned together from message_end. If the session ends before that happens, the extension does not start a last-second summarizer call from agent_end; it simply leaves the batches pending so you can flush them later (for example with /pruner now). This mode is the default because it usually causes just one context rewrite per meaningful task batch.

agentic-auto — The context_prune tool is activated and exposed to the LLM. The system prompt tells the model to use it only after a meaningful batch of related tool calls, not after every small step. Used well, this gives the agent flexibility; used badly, it can over-prune and reduce cache effectiveness.

Commands

The extension registers the /pruner command:

Command	Effect
/pruner	Interactive picker over all subcommands
/pruner settings	Opens an interactive settings overlay
/pruner on	Enable pruning
/pruner off	Disable pruning
/pruner status	Show enabled state, summarizer model, thinking level, prune trigger, batching mode, notices, and cumulative stats
/pruner model	Show current summarizer model
/pruner model <id>	Set summarizer model (e.g. anthropic/claude-haiku-3-5)
/pruner model <id>:<thinking>	Set summarizer model and thinking together (e.g. openai/gpt-5-mini:low)
/pruner thinking	Show current summarizer thinking level
/pruner thinking <level>	Set summarizer thinking (default, off, minimal, low, medium, high, xhigh)
/pruner prune-on	Interactive picker over all trigger modes
/pruner prune-on <mode>	Set trigger mode directly
/pruner batching	Interactive picker over batching modes
/pruner batching <mode>	Set batching mode directly (turn or agent-message)
/pruner min-raw-chars	Show the minimum raw-character threshold for summarization
/pruner min-raw-chars <n>	Skip batches with n or fewer raw characters (0 disables)
/pruner manual-concurrency	Show /pruner now simultaneous summary-call limit (default 8)
/pruner manual-concurrency <1-16>	Set /pruner now simultaneous summary-call limit
/pruner stats	Show cumulative summarizer token/cost stats, automatic retry total, terminal failed-batch total, and normalized failure-kind counts
/pruner tree	Browse pruned tool calls in a foldable tree browser; press Ctrl-O on a summary to open it in a bordered overlay
/pruner dry-run	Preview pending batches, threshold skips, and a historical summary/savings estimate without calling the summarizer or changing session/index/frontier state.
/pruner now	Flush pending tool calls immediately (works in all modes) in a focusable centered progress overlay showing up to 16 batch rows. It automatically retries a rate-limit, network, or temporary provider failure once per batch; the row then shows either the retry or a themed terminal failure. A rate limit applies a shared 5s-to-60s process-local cooldown to all summarizer triggers; requests during cooldown remain pending. Press Esc to stop scheduling new batches; already-running batches finish and are retained. The overlay closes automatically when processing ends.
/pruner help	Show full help text

Settings overlay

/pruner settings opens a TUI overlay with ten interactive items:

1. Enabled — toggle pruning on/off
2. Prune status line — show or hide the footer status widget and queued turn notifications
3. Startup notice — show or hide the passive pruner loaded — ... info notice on session start
4. Prune trigger — cycle through all five pruneOn modes
5. Summarizer model — press Enter to open a searchable submenu listing "default" plus all available models
6. Summarizer thinking — cycle through the thinking/reasoning level used for summarizer calls
7. Remind unpruned count — toggle the agentic-auto <pruner-note> reminder
8. Min raw chars — skip summary calls at or below the selected raw-character threshold; 0 disables it
9. Batching mode — switch between per-turn and per-agent-message summaries
10. Manual concurrency — set /pruner now simultaneous summary calls from 1 to 16 (default 8)

All changes are saved immediately to ~/.pi/agent/context-prune/settings.json and reflected in the footer status widget when it is enabled.

Tools

context_tree_query

When pruning is on, the LLM sees compact summary messages instead of raw tool outputs. Each summary ends with short aliases such as:

Summarized tool refs: `t1`, `t2`
Use `context_tree_query` with these refs to retrieve the original full outputs.

Those short refs are generated by the extension and mapped back to the real toolCallIds in the summary message metadata. The LLM only sees the short refs in future context; the full IDs stay in the stored details used by context_tree_query and internal tree/browser recovery. The tool is always available when the extension is loaded.

context_prune (agentic-auto mode only)

When pruneOn is set to agentic-auto, the context_prune tool is activated and made available to the LLM. It is removed from the active tool list in all other modes.

When the model calls context_prune:

• Pending batches with 600 or fewer raw result characters bypass the summarizer by default; set minRawCharsThreshold to customize or 0 to disable this behavior. Larger batches are summarized together, with automatic paths using one call per batch in parallel and /pruner now using its configured concurrency (default 8; range 1–16) with live per-batch updates. /pruner now retries only rate-limit, network, and temporary provider failures once; cancellation, stale context, persistence errors, threshold skips, and oversized-summary skips are never retried. Press Esc to stop scheduling more manual batches; already-running calls finish and completed results are retained.
• /pruner dry-run uses the same pending-batch capture and threshold rules as /pruner now, but never calls the provider or changes session, index, statistics, or frontier state. When prior accepted summaries exist, its estimate extrapolates their observed raw-to-summary character ratio; otherwise it explicitly reports that no estimate is available.
• While the tool is running, compact live progress is streamed into the tool output box above the input (for example Context prune running… batch 2/4 · 1.2k chars received)
• If the summary is smaller than the raw tool-result text it would replace, the original outputs are pruned from future context and a summary message is injected as a steer
• If the summary is larger than the raw tool-result text, pruning is skipped for that attempted range: the original tool results remain in context, but the prune frontier still advances so the next prune attempt starts after that range instead of retrying it forever

The tool is guided by a system prompt that instructs the model to use it after completing a meaningful batch of work (not after every trivial call).

Configuration

Config is stored in ~/.pi/agent/context-prune/settings.json (global, project-independent):

{
  "enabled": false,
  "showPruneStatusLine": true,
  "showStartupNotice": true,
  "summarizerModel": "default",
  "summarizerThinking": "default",
  "pruneOn": "agent-message",
  "remindUnprunedCount": true,
  "notifySkipped": true,
  "batchingMode": "turn",
  "minRawCharsThreshold": 600,
  "manualPruneConcurrency": 8
}

Key	Values	Default
enabled	true / false	false
showPruneStatusLine	true / false	true
showStartupNotice	true / false	true
summarizerModel	"default" or "provider/model-id"	"default"
summarizerThinking	"default", "off", "minimal", "low", "medium", "high", "xhigh"	"default"
pruneOn	"every-turn", "on-context-tag", "on-demand", "agent-message", "agentic-auto"	"agent-message"
remindUnprunedCount	true / false	true
notifySkipped	true / false	true
batchingMode	"turn" / "agent-message"	"turn"
minRawCharsThreshold	Non-negative integer raw-character count; 0 disables skipping	600
manualPruneConcurrency	Integer simultaneous /pruner now summary calls, from 1 to 16	8

• showPruneStatusLine: true keeps the prune footer widget and the automatic queued-turn notice visible. Turn it off if you want pruning to stay active without that extra status noise.

• showStartupNotice: true shows the passive pruner loaded — pruning ON/OFF | model: ... info notice when a session starts. Turn it off if you want startup to stay quiet; manual command output and real errors still appear.

• remindUnprunedCount: true appends a small ephemeral <pruner-note> to the last tool result before each LLM call to remind the model of the number of unpruned tool calls in context. This only has an effect when pruneOn is set to "agentic-auto".

• notifySkipped: false silences the "skipped pruning" warning shown when a summary would be larger than the raw tool output it replaces (pruning is skipped in that case; only the notification is suppressed).

• batchingMode: "turn" keeps one summary per assistant tool-using turn. Set it to "agent-message" to merge all assistant turns between two user messages into one summary.

• minRawCharsThreshold: 600 skips batches at or below 600 raw characters (about 150 tokens). Set it to 0 to summarize every batch.

• summarizerModel: "default" means the current active Pi model. An explicit value like "anthropic/claude-haiku-3-5" uses that model for summarization (must be registered in Pi and have an API key).

• summarizerThinking: "default" preserves old behavior: no explicit thinking/reasoning option is added to summarizer calls.

• summarizerThinking: "off" requests no summarizer reasoning where the provider adapter supports an explicit disable path. Some providers may still fall back to their own default behavior.

• "minimal", "low", "medium", "high", and "xhigh" request that thinking level for summarizer calls where supported. For cheap background summarization, prefer "minimal" or "low" with a small/fast model.

• Settings are persisted on every change via the /pruner command or the settings overlay.

Choosing a Summarizer Model

The default ("default") reuses whatever model you have active in Pi. This is convenient but wasteful — you don't need a powerful coding model to write a bullet-point summary of tool outputs. Using a cheaper, faster model here reduces both latency and cost without any quality trade-off.

Rule of thumb: pick the smallest/fastest model available on your current subscription or API plan.

Subscription / API plan	Recommended summarizer model
GitHub Copilot / Codex	openai/gpt-4.1-mini or google/gemini-2.5-flash or xai/grok-3-fast
OpenRouter	openrouter/qwen/qwen3-30b-a3b (fast MoE, very cheap)
Anthropic direct	anthropic/claude-haiku-3-5
Google AI direct	google/gemini-2.5-flash

Set it with:

/pruner model openai/gpt-4.1-mini
/pruner thinking low

# Or set both at once:
/pruner model openai/gpt-4.1-mini:low

# Or via the interactive settings overlay
/pruner settings

Or directly in ~/.pi/agent/context-prune/settings.json:

{
  "summarizerModel": "openrouter/qwen/qwen3-30b-a3b",
  "summarizerThinking": "low"
}

Architecture

For stable internal lifecycle contracts and invariants, see SPEC.md.

index.ts                    — TypeScript source entry point, wires events + modules
dist/index.js               — generated ESM bundle shipped in the npm package
src/
  types.ts                  — shared types, constants, PruneOn modes
  config.ts                 — load/save ~/.pi/agent/context-prune/settings.json
  batch-capture.ts          — capture turn_end/session-branch tool results → CapturedBatch
  summarizer.ts             — resolve model, stream LLM summaries, return usage
  indexer.ts                — Map<toolCallId, ToolCallRecord> + session persistence
  pruner.ts                 — filter context event messages
  reminder.ts               — append <pruner-note> count hints in agentic-auto mode
  summary-refs.ts           — short ref generation + summary wrapper/details helpers
  progress-text.ts          — shared live progress text formatter
  query-tool.ts             — context_tree_query tool registration
  context-prune-tool.ts     — context_prune tool registration (agentic-auto)
  frontier.ts               — persisted prune-frontier tracker for last attempted prune boundary
  stats.ts                  — StatsAccumulator for cumulative token/cost tracking
  tree-browser.ts           — foldable tree browser for /pruner tree
  commands.ts               — /pruner command, settings overlay, widgets, and message renderer

Event flow

session_start
  └─► loadConfig()              read ~/.pi/agent/context-prune/settings.json
  └─► indexer.reconstruct()     rebuild Map from session branch entries
  └─► statsAccum.reconstruct()  rebuild stats from session branch entries
  └─► frontier.reconstruct()    rebuild last prune-attempt boundary from session entries
  └─► syncToolActivation()      activate/deactivate context_prune tool

session_tree
  └─► indexer.reconstruct()     rebuild Map (branch may have different history)
  └─► statsAccum.reconstruct()  rebuild stats (branch may have different history)
  └─► frontier.reconstruct()    rebuild last prune-attempt boundary for the branch
  └─► clear pendingBatches      discard queued batches from old branch

turn_end (tool calls present + enabled)
  └─► captureBatch()            serialize the just-finished tool call batch
  └─► trim against index/frontier so same-turn later tool calls survive an earlier mid-turn prune
  └─► drop context_prune housekeeping results
  └─► push remaining tool calls to pendingBatches
  └─► if every-turn: flushPending() immediately (session delivery)
  └─► otherwise: notify user of pending count + trigger

tool_execution_end (context_checkpoint / legacy context_tag, on-context-tag mode)
  └─► flushPending()            runtime delivery

message_end (final text-only assistant message, agent-message mode)
  └─► flushPending()            session delivery

agent_end
  └─► update footer status only if batches remain pending

context_prune tool call (agentic-auto mode)
  └─► flushPending()            runtime delivery

flushPending()
  └─► scan the current session branch for completed unpruned tool results, including mid-turn subsets
  └─► trim against index/frontier so already-attempted prefixes are ignored
  └─► summarize batches         parallel by default; `/pruner now` uses configured bounded concurrency (default 8, range 1–16) with per-row progress
  └─► compare summary chars vs raw tool-result chars
  └─► if smaller: persist index + hidden summary, then advance frontier
  └─► if larger: keep original tool results, skip summary/index writes, still advance frontier
  └─► statsAccum.add()/persist() accumulate token/cost stats for the summarizer call

context
  └─► pruneMessages()            remove summarized toolResult messages from future context
  └─► optionally append <pruner-note> with the unpruned-count reminder in agentic-auto mode

before_agent_start (agentic-auto mode)
  └─► append AGENTIC_AUTO_SYSTEM_PROMPT to system prompt

Session persistence

• Config lives in ~/.pi/agent/context-prune/settings.json — the extension's own file, independent of Pi's project settings
• Index is persisted via pi.appendEntry("context-prune-index", { toolCalls }) — one entry per summarized batch, NOT in LLM context
• Prune frontier is persisted via pi.appendEntry("context-prune-frontier", ...) — it records the last attempted prune boundary even when an oversized summary is rejected
• Summaries are injected as hidden custom_message entries with customType: "context-prune-summary" — these ARE in LLM context (replacing the raw outputs only when pruning is accepted) but are not rendered into Pi's main message window. Their text uses short refs, while the details.toolCallRefs metadata keeps the full toolCallId mapping for later recovery.
• The underlying session JSONL file always retains the original ToolResultMessage entries unchanged

Footer status widget

The extension registers a status widget in the Pi footer that shows the current state:

• prune: OFF (On agent message) — pruning disabled, showing what mode it would use
• prune: ON (On agent message) — pruning active with the current trigger mode
• prune: ON (Every turn) │ ↑1.2k ↓340 $0.003 — pruning active with cumulative stats (input/output tokens, cost)
• prune: 3 pending — batches queued, waiting for the trigger
• prune: summarizing… — currently running the summarizer LLM call
• Live progress details are shown in richer surfaces instead: /pruner now uses a focusable centered overlay (up to 16 rows), and agentic-auto context_prune streams updates in the tool output box above the input
• When showPruneStatusLine is false, the footer stays clear and the queued-turn notice is suppressed, but pruning still works normally.
• When showStartupNotice is false, the passive pruner loaded — ... info notice is suppressed at session start.

v1 Limitations

• The context_tree_query tool is only active when the extension is loaded.
• The context_prune tool is only activated in agentic-auto mode.
• Summarizer latency is paid at the configured flush boundary (turn_end, message_end, context_checkpoint, /pruner now, or context_prune). More aggressive modes make that cost visible more often.
• Mid-turn pruning now supports completed subsets of a longer tool chain, but batching is still based on assistant-message groups rather than arbitrary semantic task labels.
• The /pruner tree browser shows pruned tool calls grouped under their summaries. Press Ctrl-O on a summary node to open the full pruned summary message in a bordered overlay. It still does not recover full original tool outputs inline (use context_tree_query for that).
• Summary grouping across multiple turns (e.g., "compress the last 5 summaries") is a follow-up item.

Maintainers

To review and merge upstream changes safely on a dedicated branch, run:

./scripts/sync-upstream-wizard.sh

The wizard fetches championswimmer/pi-context-prune, creates a sync/upstream-main-* branch, asks before merging, runs validation only with your confirmation, and can push/create a PR through gh.