🗜️ pi-codex-compact — Use Codex Remote Compaction in Pi

Use Responses compaction in Pi through Remote Compaction V2, unary responses/compact, or opt-in streaming context_management. The extension stores an opaque server-generated checkpoint and replays it in later compatible requests instead of generating a local plaintext summary. Pi still decides when compaction runs and keeps its normal /compact, threshold, overflow, and session-publication behavior.

✨ Features

• Supports openai-codex-responses, openai-responses, and azure-openai-responses, plus explicitly profiled custom APIs.
• Uses Remote V2, same-origin unary responses/compact, or experimental Context Management through the active provider and credentials.
• Handles manual, threshold, and overflow compaction through Pi's existing lifecycle.
• Validates and persists one bounded opaque checkpoint that survives compatible reloads, resumes, and forks.
• Replays the latest checkpoint while preserving newer conversation and extension context.
• Supports repeated and cross-protocol compaction by carrying the previous checkpoint forward.
• Falls back to Pi's native plaintext compaction on non-cancellation failures.
• Provides /codex-compact for effective-route status, settings, and manual compaction.

Context Management is experimental and requires explicit opt-in. It makes a normal inference request and consumes the active route's quota or API usage; support depends on the backend, model, and account. Default routing is unchanged.

📦 Install

Install persistently from npm:

pi install npm:@narumitw/pi-codex-compact

Try the published package without installing:

pi -e npm:@narumitw/pi-codex-compact

Try a local checkout from the repository root:

npm --workspace @narumitw/pi-codex-compact run build
pi -e ./packages/pi-codex-compact

The package declares dist/index.ts, so build a local checkout before loading its package directory. Loading the package enables automatic Responses compaction routing with the documented defaults. Do not load a global npm installation and the local workspace at the same time.

Pi extensions run with your user permissions. Review third-party extension source before installing it.

🚀 Quick start

1. Sign in through Pi's built-in OpenAI, OpenAI Codex, or Azure OpenAI provider, or configure a compatible custom provider.
2. Select a model using openai-codex-responses, openai-responses, or azure-openai-responses.
3. Work normally; Pi's automatic compaction and built-in /compact continue to operate.

Run /codex-compact when you want to inspect the effective route or choose Compact now. After successful remote compaction, compatible requests replay the opaque checkpoint automatically. Signing in does not guarantee compaction permission; see ChatGPT OAuth rejection if the backend refuses the operation.

With the default auto protocol, Codex Responses uses Remote V2 while OpenAI and Azure OpenAI Responses use unary responses/compact. When the active model uses another API, compaction remains entirely Pi-native. To opt in a compatible custom API, add it to apiProfiles as described below.

💬 Commands

Run /codex-compact to inspect the effective compaction path, change settings, or request manual compaction in TUI mode. It accepts no arguments. RPC reports the manual settings path without compacting; print and JSON modes reject the command. Closing the menu with Escape or Ctrl+C does not compact the session.

Manual compaction uses Responses Remote V2, Responses Compact API, opt-in Responses Context Management, or Pi native, as described in Settings. Pi's built-in /compact remains available and follows the same extension hook.

⚙️ Settings

The extension has one optional, global-only JSON settings file:

<getAgentDir()>/pi-codex-compact.json

The normal path is ~/.pi/agent/pi-codex-compact.json. There is no environment-variable or project-level override.

{
  "enabled": true,
  "protocol": "auto",
  "apiProfiles": {
    "custom-responses": "codex-responses-v1"
  },
  "requestTimeoutMs": 300000,
  "maxRetries": 2,
  "replacementTokenBudget": 64000,
  "notifyOnFallback": true
}

Missing fields use defaults. Settings reload on every session_start, including /reload, resume, and fork. Menu writes apply immediately, preserve unknown JSON fields, serialize within the current Pi process, and use a final conflict check plus same-directory atomic rename. On Unix, temporary files use mode 0600.

Malformed, invalid, oversized, or symlinked settings files remain unchanged. Defaults stay active, and the menu remains read-only until the file is fixed and Pi is reloaded. Separate Pi processes do not share a mutation lock; a detected concurrent edit is rejected so the user can reopen Settings and retry.

Relationship to Codex configuration

This extension does not read ~/.codex/config.toml.

✅ Requirements and compatibility

• Pi APIs compatible with the package's declared peer dependencies.
• One of openai-codex-responses, openai-responses, or azure-openai-responses on the active model, or an explicit apiProfiles entry for a custom Codex-compatible API.
• A provider whose HTTP Responses adapter honors Pi's public onPayload and injected fetch options; Context Management also requires ordered onProviderStreamEvent observations.
• A backend supporting the selected protocol and opaque compaction replay.
• Working credentials and any required compaction entitlement.

The installed built-in adapters are covered by deterministic transport tests. A custom provider or proxy is eligible when its model explicitly uses one of these APIs, but its backend still owns compatible routing, authentication, request transforms, and opaque replay. For a custom API label, eligibility is opt-in: apiProfiles must map that exact non-built-in label to "codex-responses-v1". The default is empty, and an unconfigured custom API remains Pi-native. codex-responses-v1 uses Remote V2 in auto mode. With responses-compact explicitly selected, it retains the Codex Compact request fields, including tools and reasoning. The provider must honor Pi's public transport: "sse", onPayload, and injected fetch options, and its backend must support the selected compaction protocol and opaque item replay. The profile declares compatibility; it does not probe the backend. The extension does not send a separate capability probe or automatically retry a failed billable request through the other protocol. A failed remote attempt falls back to Pi native, but an ordinary request cannot transparently recover after an incompatible provider has already received an existing opaque checkpoint. Provider provenance is stored for diagnosis but is not a replay gate. Switching providers can replay a checkpoint only when the API label, compatibility profile, and exact model ID still match. Custom API checkpoints also require a current matching apiProfiles entry; removing or changing it leaves Pi's visible fallback marker plus retained recent messages in context.

Experimental Context Management

For a backend supporting server-side compaction, select Context Management (experimental) in /codex-compact → Settings → Protocol, or save this global setting and reload Pi:

{
  "protocol": "context-management"
}

This route was verified with Pi's official OpenAI → Sign in with ChatGPT, gpt-5.5, and streaming api.openai.com/v1/responses: it produced a checkpoint and recovered an assistant-only synthetic fact when replaying without the original plaintext. This is not a guarantee for other accounts, models, Azure, Codex, or proxies. Built-in adapter tests verify request/event plumbing, not hosted entitlement.

Pi still decides when to compact. Only the extension-owned compaction request adds context_management: [{"type":"compaction","compact_threshold":1024}], forces streaming with store: false, requests reasoning.encrypted_content while preserving other include fields and the provider's thinking effort, and appends a deterministic maintenance-only user message. Original instructions, history, and tool schemas remain provider-owned; tool_choice: "none" prevents new tool work. The request uses no compaction_trigger, server-side response ID, alternate credentials, or endpoint rewrite.

Successful completed-item events are authoritative: a stream can emit several checkpoints while omitting them from terminal response output. The extension persists the latest checkpoint and its exact completed assistant/reasoning suffix only after successful completion (response.completed or Codex's response.done). Post-checkpoint reasoning must have non-empty encrypted content for stateless replay; terminal output can supply missing encryption before publication, otherwise compaction falls back to Pi native. It never executes generated tools or silently truncates that suffix. Missing checkpoints, malformed/unsafe output, exceeded bounds, permission errors, and unsupported capabilities fall back to native compaction; very short history may not cross the 1,024-token server threshold. The normal inference can consume quota even if no checkpoint is returned, and native fallback can make an additional summarization request. Safe generated output is retained as provider context, not displayed as new assistant work.

Ordinary requests do not enable server compaction or add maintenance instructions. To stop new Context Management attempts, select auto or another protocol while leaving the extension enabled for compatible replay. Older package versions do not understand this protocol's checkpoints; retain a version supporting Context Management for sessions containing them.

ChatGPT OAuth rejection

Pi's OpenAI → Sign in with ChatGPT uses official OAuth with openai-responses at api.openai.com. This is distinct from the legacy OpenAI Codex provider using openai-codex-responses at chatgpt.com/backend-api/codex; permission on one route does not establish permission on the other. API-key, Azure, and custom-provider routes retain their existing behavior and backend-specific requirements.

OpenAI documents streaming Responses inference for ChatGPT plan usage. General API documentation for compaction controls does not establish OAuth authorization for unary responses/compact or Remote V2's final compaction_trigger. The backend can reject a compaction operation with:

code=hardened_oauth_rule_missing
type=rejected_by_hardened_oauth_boundary
message=This ChatPass credential is not authorized for the requested operation.

The extension reports this operation-specific rejection and returns control to Pi-native compaction without publishing a new checkpoint or switching credentials, providers, or protocols. This does not prove the account is broken or ordinary chat is unavailable, and upgrading Pi alone is not a guaranteed fix. See OpenAI's errors and recovery for route and grant checks; re-login is not a guaranteed remedy for an endpoint permission rejection.

For sessions without an existing opaque checkpoint, turn Remote compaction off in /codex-compact → Settings, or set "enabled": false in the documented global settings file and reload Pi, to use native compaction directly. Disabling remote compaction also disables opaque checkpoint replay. If a session already contains a checkpoint, leave the extension enabled to preserve compatible replay and allow native fallback on rejected attempts; disabling it exposes only the fallback marker and retained recent messages. The selected route shown in the menu is not an entitlement check, and forcing Remote V2 is not a verified workaround for this rejection.

🔄 How it works

1. Pi prepares compaction and selects the recent message suffix it will retain.
2. If an earlier compatible checkpoint is present, the extension identifies its boundary from the summary persisted on the active CompactionEntry and validates the retained suffix fingerprints.
3. Remote V2 projects that checkpoint into a normal Responses SSE request and appends exactly one final compaction_trigger.
4. Unary compact captures the provider-built payload and authentication, rewrites only the same-origin /responses path to /responses/compact, and requests JSON without making a normal inference call.
5. Context Management makes a streaming inference request, selecting the latest completed checkpoint and all safe completed output after it in event order.
6. Remote V2 and unary compact require one bounded non-empty opaque compaction item; unary output may precede it only with user-role retained messages.
7. It stores bounded replacement history with Pi suffix fingerprints, the real API label, and the resolved compatibility profile in versioned CompactionEntry.details. Context Management stores checkpoint-first history with an exact output suffix; the other protocols keep bounded user history with the checkpoint last.
8. On later compatible requests, it replaces an exactly validated marker with the persisted replacement history immediately before provider dispatch.

The persisted entry remains the summary identity source, so replay does not depend on the wording generated by the currently installed extension version. If persisted summary identity, fingerprints, model identity, payload shape, or marker count do not match exactly, the extension leaves Pi's visible fallback context unchanged instead of guessing. Older v1 and v2 checkpoints are normalized to the current checkpoint format when read; their built-in API label determines the profile. They still replay only when the current route authorizes that API/profile/model combination.

🔒 Security and privacy

Remote compaction sends the active conversation context and system prompt to the configured Responses backend. Remote V2 and Context Management also send active tool schemas; Codex-profile unary requests retain provider-supported tools. The Pi session stores the producing provider ID, encrypted compaction item, and bounded recent user-role Responses items, or Context Management's completed post-checkpoint reasoning/assistant output. It does not store credentials, authorization headers, or request headers in checkpoint details.

An individually oversized media item is dropped rather than making the session entry unbounded. The oldest fitting text item may be partially truncated to preserve newer context. Context Management does not retain the original plaintext alongside its checkpoint and fails rather than truncating oversized post-checkpoint output. These hard byte ceilings are intentionally not configurable.

🚧 Limitations

• Hosted compaction contracts and OAuth capability restrictions can change independently of Pi or this package; keep backups of important sessions.
• Full older history depends on this extension, the checkpoint API label and compatibility profile, the exact checkpoint model ID, and a provider route whose backend accepts the opaque item. Removing the extension exposes only the portability fallback marker and Pi-retained recent messages.
• The package does not reproduce Codex core's context-window UUID/number lineage, previous-model compatibility fallback, exact pre-turn ordering, or exact mid-turn model-session ownership.
• Pi's public getAllTools() metadata does not expose constrainedSampling; Remote V2 preserves active tool order, names, descriptions, and parameter schemas but cannot reproduce that optional field.
• Remote failure falls back to Pi's plaintext summary, so a session can contain both remote opaque and native compaction entries over time.
• Settings concurrency is coordinated only within one Pi process; separate processes rely on the final conflict check.

📊 Benchmark

The repository includes a seeded benchmark that compares uncompressed full context, Pi-native plaintext compaction, and this extension's Remote V2 path. It keeps history length nearly fixed while varying information density across five state categories and ten history epochs. Benchmark v3 uses repeated artifacts, isolated evaluator probes, seed-level paired statistics, one Pi SDK estimator for dry and live fixtures, and committed protocol manifests for confirmatory candidates. It never treats nominal Pi 20K and Codex 20K settings as equal information capacity or automatically claims that protocol-conformant evidence was genuinely held out.

Preview its exploratory diagnostic without making a provider request:

npm run benchmark:codex-compact

A live run requires --live, review of the request and cost exposure, OpenAI Codex OAuth, and Remote V2 entitlement. The repository preserves the explicitly labeled v2 matched-tail diagnostic and the v3 calibration evidence, while seeds 301–304 remain consumed and unavailable for future confirmatory protocols. See the benchmark guide for manifests, repetitions, commands, privacy, cost semantics, and interpretation limits.

🧪 Development

From the repository root:

npm --workspace @narumitw/pi-codex-compact run check
npm test
npm run package:pack -- codex-compact

See the Codex compaction mechanism notes for the underlying Codex mechanism research and the extension boundary.

🗂️ Package layout

packages/pi-codex-compact/
├── src/                               # Authoritative implementation and helpers
│   ├── index.ts                       # Thin Pi entrypoint
│   └── codex-compact.ts               # Compaction routing, replay, and fallback
├── dist/                              # Generated Jiti runtime
├── scripts/build-runtime.mjs          # Runtime builder
├── benchmark/                         # Repository-only benchmark and methodology
└── test/                              # Behavior and lifecycle coverage

The generated runtime is built from src/index.ts and does not import back into src.

🔎 Keywords

Pi extension, Pi coding agent, OpenAI Codex, ChatGPT OAuth, Azure OpenAI, custom provider, proxy, Remote Compaction V2, Responses Compact API, context management, opaque checkpoint, Responses API, context compaction.

📄 License

MIT