pi-tuck

Compaction for Pi with no model call, and a history the agent can search by keyword, and on Apple silicon by meaning, all on your machine. In our 128k live run, a cut took a median 0.08 s (pi-tuck 1.0.1). It costs no tokens. The same session always gives the same summary. And nothing is lost: the agent can read back anything earlier, word for word, and on Apple silicon it can find it by meaning.

pi install npm:pi-tuck

pi-tuck against Pi's built-in compaction

Pi's built-in compaction asks a model to write a summary. pi-tuck builds the summary from the session itself. The time, context and task rows compare pi-tuck 1.0.1 with Pi 1.4.0's built-in compaction, where a local model writes the summary. Each measured result in this README names the pi-tuck build or version it was measured on; the release you install may be newer.

The time and context rows are from the 128k live run: 10 chains of 30 tasks, pi-tuck 1.0.1, medians over 46 pi-tuck cuts and 48 cuts by Pi's compaction. pi-tuck's cut time is an upper bound, read from the session file (benchmark/DEVIATIONS.md).

The retention row is a paired replay on mechanical probes only: files changed, test-runner summary, required tag, error class and the session's result-line rule. No probe judged how well an agent reasons. The claim line, as benchmark/RESULTS-MAIN.md words it: pi-tuck, answering with its recall tools, retains probed facts at a rate not more than 3 points below that of "Pi's compaction prompts on mac-studio/main:xhigh" (20 recordings) or of "Pi's compaction prompts on GPT-6.1 Terra" (19 of 20 recordings, s09-A missing), each answering from its summary alone, on these probes. This is not summary against summary: it compares pi-tuck's summary together with its recall tools against Pi's compaction without them, and the run has no pi-tuck arm without its recall tools. It says nothing about any other compactor, model or window.

In the replay behind pi-tuck's retention numbers, 52 of its 95 answer sets ended in an abort (0 of 95 for Pi's compaction prompts on the same local model), so those numbers cover cuts 1 to 4 only and no retention claim is made past cut 4; see benchmark/RESULTS-MAIN.md.

The 32k row is one model, one window and five chains. Observation masking, a third approach, resolved 117 of the same 150 tasks (pi-tuck +2.0 points, lower bound −2.7: no difference shown). In the 32k run only pi-tuck was held to the 16,384-token room: Pi's built-in compaction sent a request over it in 97 of 150 tasks. Every number comes from a file in this repository. Conditions, sample sizes and the full limits are in Benchmarks.

Ask your history

A summary cannot hold everything. So pi-tuck lets the agent look things up instead of guess. Say you fixed a login bug two hours ago, and the summary no longer holds the details. You ask:

"What was the reason we dropped the retry on the token refresh?"

The original message says "removing the backoff loop from the auth renewal". The words differ, so a keyword search finds nothing. The agent calls tuck_search, which also searches by meaning:

{ "query": "why did we drop the retry on token refresh" }

It gets back the original passage, with a pointer such as #212. It reads the whole message, exactly as it was written:

{ "index": 212 }

That is tuck_recall. The agent answers from the original text, not from a model's memory of it.

This example is an illustration. It is not a recorded session. The call shapes are real. Semantic search has no measured quality numbers yet. A test of it is pre-registered in benchmark/PREREG-RECALL.md; it has no result yet. The benchmarks below measure cut time, cost and answer retention after cuts, not how well tuck_search finds a passage. tuck_search exists only on macOS with Apple silicon. Elsewhere the agent has tuck_recall and its keyword search, and the example above would not find its match.

Install

pi install npm:pi-tuck

Then start a new Pi session. That is all. pi-tuck takes over Pi's compaction, both automatic and /compact. You change no setting.

• Needs Pi 1.2.0 or later (tested on 1.2.0 and 1.4.0), on macOS or Linux. On an older Pi, pi-tuck stays off, Pi's own compaction runs, and pi-tuck says so once. Full features on macOS with Apple silicon; elsewhere, compaction and word recall.
• On macOS with Apple silicon, the first session also downloads a local search model in the background (about 390 MB, once). Until it is ready, search works by keyword. See Semantic search.
• Prefer GitHub? pi install git:github.com/galyavin/pi-tuck@v1.0.10.
• To pin a version, add it: pi install npm:pi-tuck@1.0.10.

How it works

When context fills up, pi-tuck replaces the older turns with one summary. The summary keeps:

• The dialogue. What you asked and what the agent answered, in order, word for word. If it is too long for the room, it shrinks: the oldest assistant text is clipped first, then older prompts. Clipped text says how to read it in full.
• The files. Files the agent read, wrote or edited, newest first (up to 60), each with a pointer to read it back.
• The action trail. The agent's other tool calls, such as shell commands, newest first (up to 30). The output is left out. A pointer leads to it.
• Your first prompt, word for word (up to 12,000 characters).

Tool output stays in the session file, not in the summary. Each summary ends with a short instruction to the agent: re-read a file before you edit it, look things up, and treat anything not shown as unknown.

The summary is plain text, one line per record, such as #12 user @2026-10-30T14:00:00Z: text. The number is the index that tuck_recall takes. Summaries from earlier versions are still read.

The agent gets two tools to get anything back:

• tuck_recall finds earlier text by keyword. It reads any original message or tool result exactly, by the index shown in a summary. It can also write an original out to a file.
• tuck_search (macOS with Apple silicon only) finds earlier history related to a question, even when the wording differs. It combines keyword hits with local semantic search and returns the original text with pointers for tuck_recall.

On Apple silicon, both tools cost about 1,050 tokens of tool description per request. Elsewhere only tuck_recall is offered.

/compact works as usual, but pi-tuck does not take custom summary instructions. Pi's own setting "compaction": { "enabled": false } turns compaction off.

What is proven, and what is not

Measured so far. In the 128k live run (pi-tuck 1.0.1), a pi-tuck cut took a median 0.08 s, where Pi's compaction with a local model writing the summary took a median 235 s. A cut calls no model, so it costs no tokens. After a cut, both carried about the same context. On mechanical probes about facts cut away, pi-tuck with its recall tools was not more than 3 points below Pi's compaction prompts on mac-studio/main:xhigh, or below Pi's compaction prompts on GPT-6.1 Terra, each answering from its summary alone (claim line above; cuts 1 to 4 only). Over 300 tasks per arm, pi-tuck resolved 258 and Pi's compaction 251; the registered claim is only that pi-tuck lost no more than 0.4 points. In replay, answers held when pi-tuck cut at 60% of the room instead of 100%. At a 32k window, in one suite of 150 SWE-bench Verified tasks with one model, pi-tuck 1.0.5 resolved 120 tasks and Pi's built-in compaction 94 (+17.3 points, 95% lower bound +10.7). Observation masking resolved 117 (pi-tuck +2.0 points, lower bound −2.7: no difference shown). The limits of each result are in Benchmarks.

Not claimed. This README does not say that pi-tuck is as good as or better than any compactor, model, window or kind of task other than those named, and it never calls it "the best LLM compactor". The retention tests compare pi-tuck with its recall tools against Pi's compaction without them. The tests were pre-registered before any result came in. The plans, and every result, including those that did not favour pi-tuck, are in benchmark/.

The pre-registered plan has three parts:

1. A live run at a 128k window on a local model, with pi-tuck 1.0.1. Ten chains of 30 SWE-bench Verified tasks per arm, with Pi's built-in compaction in one arm and pi-tuck in the other. Official grading.
2. A paired replay with probe questions. The same cuts, made by pi-tuck and by Pi's summarize prompts, then questions about facts that fell before the cut. Pre-registered rule: the claim holds only if pi-tuck is no more than 3 points worse than the better compactor, with a one-sided 95% bound. The comparators are named in benchmark/PINS.md and above.
3. A 32k suite, where tasks are bigger than the room, so cuts must summarise work in progress. Its result is the 32k row above.

The plan: benchmark/PREREG.md, benchmark/PREREG-32K.md, benchmark/PREREG-TERRA.md, and every change after the freeze in benchmark/DEVIATIONS.md.

Known limits today:

• pi-tuck's summary does not explain why things happened. It holds the words that were said, not a model's reading of them. The agent must look things up. Each look-up costs a tool call.
• Small windows: an upstream that spends far more context on tool schemas than others (some OpenRouter upstreams do) can leave a small input room too little space. pi-tuck then refuses to cut rather than overflow. This made 1.0.1 refuse at a 32k window with Pi's default 16,384 reserve. Version 1.0.4 added a notice that says when the provider counts far above pi-tuck's estimate. In 1.0.4, after an oversize tool result or a "Nothing to compact" error from Pi, a session could keep refusing later prompts; 1.0.5 fixed both. Version 1.0.8 clips a fresh tool result that is too large for the room (its head and tail, with a tuck_recall locator to the rest) instead of refusing the run. If a small window still refuses, pin a different upstream, lower the reserve, or use a larger window; see Small windows.
• Small window, after a stopped run (1.0.8 and 1.0.9 only; fixed in 1.0.10): Pi's default compaction.keepRecentTokens (20,000) can exceed pi-tuck's input room, which is the window less the larger of compaction.reserveTokens and the model's maximum output (16,384 at a 32k window with the default reserve). After a run that is stopped (Esc, or a time limit) with the context above the room, every later prompt could be refused with "The request needs a cut and Pi has none to make (Nothing to compact (session too small))", and the session stayed stuck. In one check at these defaults (pi-tuck 1.0.8, 32k window, one session of 30 tasks, one local model), 28 tasks were lost, tasks 3 to 30 (benchmark/RESULTS-DEFAULTS.md). If you are on 1.0.8 or 1.0.9, upgrade to 1.0.10. In 1.0.10 the cut a prompt needs is pi-tuck's own, so keepRecentTokens no longer leaves it with nothing to cut. A rerun of the same check on the 1.0.10 candidate lost no task and got stuck nowhere, with the caveat that two of its tasks had a first request the local server never answered, which a literal no-token count would call lost (benchmark/RESULTS-DEFAULTS-1010.md). Windows of 128k and up were not affected at the defaults.
• Small window, a large prompt after a summary-only cut (1.0.10): when the previous summary alone fills the room and a large prompt arrives, the prompt is dropped, because a cut cannot yet shrink a summary on its own. 1.0.10 drops it with no message, where 1.0.9 refused it with one. The next prompt runs normally. Seen once, on a 25,160-character prompt at a 32k window (benchmark/RESULTS-CUTLATER.md, t03 task 24). A fix is planned for 1.0.11.
• Other extensions (1.0.10): a prompt's cut no longer goes through Pi's session_before_compact, so another extension's veto there no longer stops it. It never stopped a cut between tool rounds. Pi's own threshold and overflow cuts, and /compact, still ask other extensions as before.
• Semantic search (tuck_search) needs a Mac with Metal (Apple silicon). Elsewhere, compaction and tuck_recall's word search work, there is no tuck_search, and nothing is downloaded.

Benchmarks

Each row says what was measured, under which conditions, and where the data is.

Notes:

• The answer replays use one cheap hosted model (DeepSeek flash) as agent and judge. They compare pi-tuck with other pi-tuck settings and versions. They do not compare pi-tuck with an LLM compactor.
• The 128k rows measure pi-tuck 1.0.1. The newer versions are listed in the changelog. A cut still calls no model, but the 128k run has not been repeated on them.
• Everything here is from SWE-bench-style tasks on a small set of models. It may not carry to your work.

Settings

pi-tuck needs none. Two are optional. Put them in pi-tuck.json in Pi's agent folder (~/.pi/agent, or the folder PI_CODING_AGENT_DIR names):

{ "cutAt": 0.6, "semanticRecall": true }

• cutAt (above 0, at most 1; default 0.6) is the part of the input room a request may fill before pi-tuck cuts. At 0.6, each cut leaves room to grow. { "cutAt": 1 } cuts at the full room. A lower value cuts earlier and smaller. The whole room still decides which request pi-tuck refuses.
• semanticRecall (false turns semantic search off, on Apple silicon). Nothing is then downloaded, and tuck_search uses keywords only.

A bad file is reported at session start, and pi-tuck uses the defaults.

Small windows

pi-tuck keeps room for the model's reply: the larger of the model's output limit and Pi's compaction.reserveTokens (default 16,384). It refuses a request that cannot fit, rather than let the provider cut it off. It estimates the request's size, and it scales that estimate by the worst gap it has seen between its count and the provider's. In a small window, an upstream that spends far more context on tool schemas (some OpenRouter upstreams do) can leave too little room. pi-tuck then refuses to cut rather than overflow. Lower the reserve for that model in Pi's settings.json, or use a larger window. For example, to lower it to 8,192:

{ "compaction": { "modelOverrides": { "my-provider/my-model": { "reserveTokens": 8192 } } } }

The reserve is the larger of reserveTokens and the model's maxTokens, so lowering reserveTokens alone does nothing when models.json sets a larger maxTokens for that model. Lower both, and keep maxTokens above your largest expected reply.

If you reach the model through OpenRouter, you can also pin one upstream, so that every request is served by the same one. Put this in models.json in Pi's agent folder, and name the upstream you want:

{ "providers": { "openrouter": { "compat": { "openRouterRouting": { "only": ["your-upstream"], "allow_fallbacks": false } } } } }

Semantic search

Semantic search is on by default on macOS with Apple silicon. It runs entirely on your Mac: an embedding model, EmbeddingGemma-2, served by llama.cpp on your Mac's GPU (Metal). No text leaves your machine. Elsewhere (Linux, Intel Macs) there is no tuck_search, and nothing is downloaded; compaction and tuck_recall work. On a Mac where Metal fails to start, semantic recall goes off with the reason "Semantic recall needs Metal; word recall still works".

The first time a session starts, pi-tuck downloads what it needs, once, about 390 MB, in the background. Keyword search works meanwhile:

Every file is pinned to a SHA-256 in src/assets.ts and checked before use. Everything goes to ~/.pi/agent/pi-tuck; to remove it, delete that folder. Setup needs about 6.5 GB of free disk space while it runs, and uses 460 MB after. If it fails, for example offline, pi-tuck says so and tries again at the next session start. One-shot runs (pi -p) do not start the setup.

Indexing starts in the background the first time a session uses tuck_search, and catches up after each compaction. Semantic search is off when Pi runs in a folder that contains Pi's agent folder, such as your home folder, because its own files live there. Open Pi in a project folder.

Use from code

The package also exports its parts for hosts that embed Pi through its SDK:

import { installCompaction, createSemanticRecall, ensureSemanticRuntime } from "pi-tuck";

const config = JSON.parse(readFileSync(await ensureSemanticRuntime("/abs/path/outside/projects"), "utf8"));
const semanticRecall = createSemanticRecall(config);
const handle = await installCompaction(pi, { semanticRecall });
await handle.setSemanticEnabled(true, ctx); // e.g. in a session_start handler

installCompaction options (reply reserve, host-injected messages, recall policy, cutAt and others) are documented in src/compaction.ts. handle.close() stops the semantic worker; call it before disposing a session.

Development

npm install
npm run check   # types
npm test        # unit tests (Vitest) and the Python runtime tests
npm run build   # dist/ for the library export

Changes are listed in CHANGELOG.md. Releases are published from GitHub Actions by npm trusted publishing, with provenance.

Credits

pi-tuck grew out of pi-vcc by sting8k and includes code from pi-blackhole by k0valik and from Pi by Mario Zechner, all MIT; their notices are in LICENSE. The downloaded model is Apache-2.0, llama.cpp is MIT, and CPython is under the PSF license.

License

MIT