pi-protokoll

A Pi extension that makes very long sessions possible with local models: the model writes a short protocol at the end of every turn, and compaction builds a bounded, tiered summary from those protocols instead of re-reading and summarizing the whole history.

Deutsche Fassung: README.de.md

Why

Pi's built-in compaction sends the old history to the model and waits while it writes a summary. With a local model that is slow (minutes on a long session), and the summary is written after the fact from an overfull context.

Here the expensive part happens while working: each final answer ends with a protocol block, written while the model still has the full turn in view. At compaction time almost everything is ready; code assembles the summary and the model only updates a short overview.

Measured in one session with Qwen3.x on a local OpenAI-compatible server (2× RTX 3090, 262k context window):

• 22 compactions over 118 turns, each about 213,000 → 35,000 tokens in about 30 s
• resuming the session reads about 36,000–44,000 tokens
• every earlier turn stays reachable through tools

In the 12-day session shown in the chart above, also with a local Qwen model, a cut takes the context from about 213,000 tokens to a median of about 67,000 in the next request; about 38,000 of those are the summary, the rest is the system prompt and what stays word for word. Between cuts the server's prefix (KV) cache holds: in the median request 99.8 % of the prompt tokens came from it. The request right after a cut has only the system prompt cached and re-reads about 62,000 tokens once. What the extension saves is the re-summarization pass, not that re-read.

Practical example: 64kdemo, a procedural forest landscape in C and OpenGL, with screenshots, a camera flight and the story of how it came about. Its first ten sessions ran without the extension and kept ending in context poisoning; with the extension, from the tree system on, it stayed a single session, the one measured above.

How it works

A turn is one agent run: your message with all tool rounds up to the final answer. Messages you send while the model is working belong to the running turn.

1. System prompt. A fixed section asks the model to end every final answer with a <protocol> block: result, decision, files, open, rule, revoke, index, tags. The text never changes, so the server's prefix cache stays valid. It also states once what the model can and cannot know about the length of a session (see Known limits).

2. Answer done. The block is cut out of the answer and stored as a session entry, shown collapsed (▸ protocol turn 7: …). A block written early keeps the text after it in the answer; a tag quoted inside code or inside a sentence is left alone. The extension numbers the turns, not the model.

3. Block missing. The extension asks once more with the unchanged context plus a short instruction. The prompt starts like the one just run, so a prefix cache only reads the new tokens. Only the block is kept, not the exchange. If the model ignores the request, the block is written from the turn's transcript right away.

4. Turn interrupted (Esc, error, output limit). Before your next message goes out, the protocol of the interrupted turn is written: what was done, what is still open. Tool calls the model makes out of habit are answered with "tools unavailable" and not executed. If that fails, a second attempt works from the turn's transcript.

5. Compaction. Pi triggers it as usual. The summary has tiers, each with a hard limit:

   Tier	Content	Limit
   verbatim	the last turns as they are	keepRecentTurns, at most keepRecentPercent % of the window
   detail	the next older turns: your message verbatim + protocol; pinned turns in full; turns without protocol as a shortened transcript	detailMaxTokens
   status	everything older	statusMaxTokens

   The status consists of:

   • an index, one line per older turn: the index line the model wrote with that turn. Code assembles it again at every compaction from the stored protocols, so it cannot drift. When it no longer fits, the oldest turns are grouped into range lines built from their tags ([T1–T20] leaf palette (T14, T16); DEMO_LEAFPAL (T14); impostor atlas (T18–T19));
   • the active rules (rule minus revoke), open items and files touched, all taken from the protocols by code;
   • a state of work, the only part the model rewrites: it gets the previous state and the protocols of the turns now leaving the detail tier. Long backlogs are folded in chunks of foldChunkTokens.

   A review step lets the model clean up rules and open items. It can only choose entries by number (RULES: R1, R4), so it cannot reword or invent anything.

   If anything fails, or the result would not free enough space, the extension steps back and Pi compacts in its own way. The chat never blocks.

6. Lookup tools for the model:

   • protocol_read("12, 40-44") returns the full stored protocols plus your message,
   • protocol_search(query) finds turns by keywords (tags count double),
   • protocol_turn(n) returns the original transcript of one turn.

7. Switching inside a session. /protokoll off leaves the system prompt and the tool list byte-identical and inserts one message at the current position that withdraws the rules; the three protocol_* tools stay declared and answer pi-protokoll is disabled. That is what keeps the server's prefix cache. /protokoll on brings it back: the section is still in the prompt, so a short note applies it again. A session that ran without protokoll from the start never had the section in its prefix – enabling does not add it there either, it brings the rules once as a message, so the prefix stays untouched. Older turns have no protocols and are compacted as shortened transcripts; /protokoll repair writes theirs. The hard variant /protokoll off --strip really removes the section and unregisters the tools; the next request then reads the whole context again (once). Pi copies the system message of the branch onto the compaction entry, so the prompt (section included) outlives a compaction while a note can be cut away by it. At the first settle boundary after a compaction the extension retires notes that now contradict the state (context_edit with an empty replacement); if a needed withdrawal was cut away, the section leaves the prompt instead.

Progress is shown above the editor during compaction and backfills. Afterwards a widget shows the new context size; it stays until the next message, because Pi redraws the chat after compaction.

Every compaction additionally stores its own status entry (protokoll-compact): how it ended, how long it took in total, the time of every step and the notification lines of the pass. The duration is therefore readable long after the widget is gone – in the console and in anything that renders session entries, a web view for example. /protokoll export lists the time of every compaction.

While a pass runs, every step is also published on Pi's shared extension bus (channel pi-protokoll:progress, see ProgressEvent in src/core.ts), so another extension can show the running steps live. Each report carries the whole run: for every step its state, its own time, how many characters the model has sent and whether it is thinking or writing — the same status the console widget shows. Those lines are never stored in the session — one status entry per pass stays the record.

Install

pi install git:github.com/sebion7125/pi-protokoll

or from a local checkout:

pi install ./pi-protokoll

Restart Pi (or /reload) and run /protokoll status.

Configuration is optional. Copy protokoll.example.json to ~/.pi/agent/protokoll.json and adjust it; a project can override values in <project>/.pi/protokoll.json.

Upgrading from the manual install (~/.pi/agent/extensions/protokoll/): delete that folder, otherwise the extension loads twice. Your protokoll.json and all session data stay as they are.

Requirements and scope

• Pi ≥ 1.1. Used daily with 1.1.0; typechecked, unit-tested and run end to end (stub model) with 1.4.0.
• Built and tested with local models behind an OpenAI-compatible server (Qwen3.x). Thinking levels for the extension's own requests are written directly into the request (reasoning_effort, chat_template_kwargs.enable_thinking). This works for that kind of server only.
• Other providers and models are untested. The extension should work with any model that follows the block format, but cache behaviour and thinking control may differ.

Commands

The German command names of earlier versions (zeigen, anheften, lösen, an, aus, neu-laden, stand-neu, heilen) still work.

Configuration

The whole summary is at most about detailMaxTokens + statusMaxTokens + 400 tokens, about 38,000 with the defaults.

When to compact is still Pi's decision: context > contextWindow − compaction.reserveTokens (~/.pi/agent/settings.json). Make sure contextWindow in models.json matches your server.

Auxiliary model

auxiliaryModel applies to the extension's own requests only: follow-up, transcript backfill, folding the state of work, the review of rules and repair. The model of your chat stays exactly as it is.

{
  "auxiliaryModel": "openai/gpt-4.1-mini"
}

Pi's Cursor path has no session mapping for the kind of direct request the extension makes, so asking a Cursor model directly does not work. When your session model is a Cursor model (provider: "cursor" or api: "cursor-sdk") and no auxiliary model is configured, the extension sends no request at all: it says so once per session, treats the protocol as missing and leaves compaction and everything else working. Set auxiliaryModel to a model from another provider to get the follow-up, backfill and review back. A malformed or unknown reference is reported and the request is skipped – it never silently falls back to the session model. Note that a request which repeats the chat context has to fit the auxiliary model's context window.

Development

npm install
npm run check   # typecheck, lint, tests
npm run e2e     # real Pi CLI from node_modules + stub model: install, nine turns, compaction

The tests (tests/) drive the extension through a stand-in for Pi's extension API with a scripted model. They check the wiring and the pure logic, not Pi itself.

Known limits

• The model must write the block. Qwen3.x forgets it in roughly half of the long tool-heavy turns; the follow-up request fixes that at the cost of a few seconds.
• The model cannot see how full its context window is, yet occasionally invents a fill level (Context left: 12 %, Context exhausted) and advises a fresh chat. Nothing in Pi, in this extension or in any installed package prints such lines – the numbers are made up. The system prompt section states the facts instead: without a fill level given in the current turn the model says nothing about the window, and it suggests a fresh session only on a concrete verifiable reason. This softens it; measured effect: none yet. The text is fixed from here on so the prefix cache stays valid.
• Whether a model follows a withdrawal note at the end of a long transcript is model-dependent. If it keeps writing protocol blocks after /protokoll off, use /protokoll off --strip.
• The context rule assumes pi-protokoll does the compaction. With other compaction extensions it is untested. A fill level another extension hands over in the current turn is deliberately not denied – such a number is information the model may use.
• A soft /protokoll off keeps the three protocol tools declared, because the tool list is part of the prefix. A model can still call them and gets pi-protokoll is disabled back.
• /compact <instructions> is not yet passed on to the state of work.
• The state of work is the one part the model rewrites, so it can drift; the index, the lists and the stored protocols cannot. /protokoll rebuild rebuilds it from scratch.

License

MIT