@brglng/pi-auto-review

This package is a fork of @erichll/pi-auto-review

The differences from the upstream:

• Retry count is removed.
• timeoutMs no longer has an upper bound.

Original README from @erichll/pi-auto-review

A fail-closed, model-backed boundary reviewer for the Pi coding agent.

The extension participates in @gotgenes/pi-permission-system as the pi-auto-review authorizer and exposes a process-local broker for OS sandbox adapters. It is an authorizer inside the permission system, so installing that dependency is a hard prerequisite (see Install and enable). The npm package and reviewer model have separate names:

• package: @brglng/pi-auto-review
• authorizer: pi-auto-review
• default reviewer model: codex-auto-review

Contents

• Install and enable
• Security model
• Configuration
  • Deadlines and retries
• Operator feedback and exact retry
• Boundary broker API
• Reviewer context and token budgets
• Sandbox integration
• Trust boundary
• Permission policy audit
• Telemetry
• Real-model smoke test

Install and enable

Prerequisite: pi-auto-review is an authorizer inside @gotgenes/pi-permission-system. Pi does not auto-install peer packages, so install the permission system separately (once per machine) before this extension. Version 0.10.0 requires permission-system 27.x:

Node.js 22.13.0 or newer is required. Permission auditing uses Node's built-in node:sqlite; it does not require a SQLite CLI, system SQLite library, or npm SQLite package.

pi install npm:@gotgenes/pi-permission-system
pi install npm:@brglng/pi-auto-review

Install the package outside the agent-writable workspace (see Trust boundary):

Add it to the permission-system authorizer chain:

{
  "authorizerChain": ["pi-auto-review"]
}

The configured reviewer provider and model must also be registered in Pi.

Security model

Requests pass through these controls in order:

1. deterministic hard denies;
2. bounded model review;
3. the local permission terminal when required; and
4. an exact, expiring, one-use grant for an external sandbox adapter.

The model cannot override hard denies or grant authority directly. External adapters must consume the exact grant before retrying an operation. Changing the command, path, resolved path, destination, cwd, agent, or tool input invalidates that grant.

Permission-system downgrades authorizer allows on path and external_directory to defer. In an interactive TUI, autoConfirmBoundedAllows can bind the exact model allow to the immediately following recognized permission dialog. The bridge is request-ID-bound, expires after ten seconds, and is consumed once. Mode, component, request, or event-order mismatches leave the original human dialog in place.

Automatic review stops for the current turn after three consecutive denials or ten denials in the last fifty reviews. An explicit denial tells the agent that automatic policy denied the request (not a human click), not to rephrase or circumvent the same action, and points to /auto-review-approve for an exact non-critical reviewer retry or /auto-review-break-glass for a critical model denial. Local deterministic hard denies never offer an override command.

Deterministic hard denies cover recursive forced wipes of /, ~, and $HOME. A named path under /home/... is reviewed by the model as high-risk, not treated as a home-directory wipe.

Configuration

Configuration is resolved in this order:

1. package defaults in src/config.json;
2. optional trusted user overlay at ~/.pi/agent/extensions/pi-auto-review/config.json; and
3. optional project tighten-only settings at .pi/pi-auto-review.json.

Use the user-global file for normal customization. It may set any legal key:

{
  "model": "provider/reviewer-model",
  "autoConfirmBoundedAllows": ["external_directory", "path"]
}

For a complete @gotgenes/pi-permission-system config that wires pi-auto-review into the authorizer chain — a copyable baseline covering read/write/edit, a read-only bash allowlist, an MCP discovery policy, and a path deny block for secret and credential files — see examples/pi-permission-system.config.example.json.

Package defaults:

{
  "model": "codex-auto-review",
  "reasoning": "low",
  "timeoutMs": 90000,
  "maxTokens": 256,
  "retries": 2,
  "maxUserTranscriptTokens": 1200,
  "maxToolTranscriptTokens": 1200,
  "maxRelevantResultTokens": 800,
  "maxReviewerInputTokens": 8192,
  "breakGlassEnabled": true,
  "failureMode": "deny",
  "grantTtlMs": 60000,
  "autoConfirmBoundedAllows": ["external_directory", "path"],
  "policyAudit": {
    "enabled": true,
    "retentionDays": 180
  }
}

failureMode: "deny" is the default. "defer" falls through to the human terminal. Set autoConfirmBoundedAllows to [] to keep every bounded allow manual.

Project configuration may only lower timeouts, token/evidence limits, retries, and grant TTL, set failureMode to "deny", set breakGlassEnabled to false, or remove auto-confirmed surfaces. It cannot re-enable break glass, select a model, raise a trusted limit, or weaken fail-closed behavior. Invalid configuration disables the reviewer for that session.

The trusted user config may set policyAudit.enabled and a retention of 1–3,650 days. Project config may only set enabled: false or shorten the inherited retention; it cannot re-enable globally disabled collection or extend retention.

Deadlines and retries

timeoutMs is one deadline shared by model resolution, authentication, model attempts, and retry delays. Each attempt receives only the remaining time. Provider-internal retries are disabled. A review may make up to retries + 1 actual model calls.

Valid decisions, output-length stops, timeouts, aborts, authentication/model/ request errors, and unknown failures do not retry. Empty, non-JSON, or schema-invalid output and recognized connection, temporary 5xx, or 429 failures may retry once when the retry budget and deadline allow it. A Retry-After above five seconds or beyond the remaining deadline fails closed. Format retries preserve the canonical request and selected evidence and append only a fixed, budget-checked schema correction.

Operator feedback and exact retry

Interactive sessions show the current permission check in a single widget above the editor. Each check first shows its surface, compact target, and the dynamically configured reviewer model, then replaces that content in place with the outcome, target and rationale, model, token usage, duration, and any extra call count. A new check replaces the previous result; the latest result remains visible until then and is cleared when the session changes or shuts down. Concurrent older checks cannot overwrite the most recently started one.

Every request still has its own model call, verdict, grant, local confirmation, and audit record. No new review-result transcript entries are written. Existing pi-auto-review entries in historical sessions remain renderable. Non-TUI modes retain best-effort notifications, and a failed TUI widget update falls back to the same notification path. UI delivery never changes the authorization result.

In an interactive TUI, /auto-review-approve lists up to ten recent non-critical model denials from the current session. Selecting one asks the agent to retry exactly that request. The old /approve command is not registered. The host-generated override:

• binds the complete request hash;
• expires after 60 seconds and is consumed once;
• remains separate from untrusted user/tool evidence;
• still goes through deterministic hard denies and model review; and
• cannot be reissued for the same request semantics in that session.

It is authorization evidence, not a direct allow.

/auto-review-break-glass is a separate, high-friction path for model denials whose risk level is critical. It lists only critical model denials from the same session made in the last five minutes. After showing the rationale, surface, cwd, command or target summary, and request-hash fingerprint, it requires an explicit confirmation and a random BREAK-GLASS <CODE> phrase within 60 seconds. Successful confirmation creates a 60-second, one-use authorization bound to the complete request hash, session, scope, and original request ID. The exact retry reruns local hard-deny checks, then allows directly without calling the reviewer and, for sandbox adapters, still issues the normal one-shot grant. Break glass does not reset circuit-breaker history and can be disabled in trusted or project configuration with breakGlassEnabled: false.

Boundary broker API

The extension publishes a process-local service at:

Symbol.for("pi-auto-review:boundary-approval-broker")

Adapters should use the exported helper:

import {
  getBoundaryBroker,
  type BoundaryRequest,
} from "@brglng/pi-auto-review";

const request: BoundaryRequest = {
  id: "sandbox-runtime-query-id",
  source: "sandbox-runtime",
  surface: "network",
  operation: "connect",
  cwd: "/workspace/project",
  command: "npm install",
  destination: "registry.npmjs.org:443",
};

const broker = getBoundaryBroker();
const decision = await broker?.review(request, {
  sessionId: "pi-session-id",
  scopeKey: "pi-session-id:turn-id",
  issueGrant: true,
});

// A break-glass allow includes structured provenance:
// decision.authorization = {
//   kind: "break-glass",
//   originalRequestId: "...",
//   confirmedAt: 0,
// };

if (
  decision?.kind === "allow" &&
  decision.grant &&
  broker?.consumeGrant(request, "pi-session-id", decision.grant.token)
) {
  // Retry this exact operation once inside the OS sandbox.
}

Grants expire after grantTtlMs and cannot be reused.

Reviewer context and token budgets

The reviewer receives one compact canonical request plus bounded, explicitly untrusted evidence. Selection is deterministic rather than semantic:

• the latest raw user message is the authorization anchor;
• older user messages require an exact request/tool/requester association or an exact trusted retry;
• tool calls require an exact tool-call ID, exact structured request fields, a surface profile, or a security-combination classification; and
• selected results stay paired with their producer and are limited to exact results, deletion prechecks, Git push context, matching branch protection, and Sandbox Runtime process evidence.

Unrelated reads, directory listings, builds, tests, assistant prose, and old task history are excluded. Compaction and branch summaries are labeled as non-authorization context and are never injected as user intent. The host does not rewrite a model allow from regex matches on user text, vague continuations, or a computed authorization ceiling; those judgments stay with the reviewer model. Hard denies still terminate before the model.

The current operation appears once as stable, key-sorted JSON. Duplicate exact tool-call arguments collapse to an ID/name/reason linkage shell, while fields not represented by the request remain available as evidence. This compact reviewer representation does not affect request hashes, grants, overrides, or audit evidence.

maxReviewerInputTokens covers the fixed policy, canonical request, override, evidence, omissions, JSON framing, and a 64-token provider-framing reserve. Its legal range is 2,048–32,768. Because no matching tokenizer is bundled, the conservative:utf8 estimator counts every UTF-8 byte as one token.

When over budget, the host removes secondary reasons, older structured tool matches, then optional producer/result units, re-estimating after each step. It never silently removes the canonical request, exact override, latest user evidence, security-combination evidence, exact tool linkage, or a required surface profile. If mandatory evidence does not fit, review fails closed before calling the model. More than four security-combination candidates also fails closed locally.

Sandbox integration

This package exposes the broker contract but does not intercept OS sandbox events itself. Adapters translate a concrete boundary into a BoundaryRequest and consume the exact grant before allowing it.

Sandbox adapters such as a pi-sandbox adapter built on Anthropic Sandbox Runtime use the broker this way: filesystem policy is static and fail-closed; unmatched public network destinations use the broker for one connection. Each Bash command or built-in subagent session owns an independent Sandbox Runtime broker process. Adapter implementations should use the package's ./sandbox export and must not create broad permanent rules in .pi/sandbox.json.

Trust boundary

Production copies must live outside the agent-writable workspace. Pi installs user npm and Git packages under ~/.pi/agent/npm/ and ~/.pi/agent/git/. Workspace-loaded copies are rejected unless local development explicitly opts in:

PI_AUTO_REVIEW_ALLOW_UNTRUSTED_DEV=1 pi --approve

Writes to the installed reviewer package, its user-global configuration, project and global security configuration, and the global audit directory are deterministically denied.

Permission policy audit

The extension observes terminal permissions:decision broadcasts and stores only daily, redacted aggregates. Collection is enabled by default and retains 180 days. It starts when this version first initializes successfully; no permission-system JSONL or RTK history is read or imported.

Before storage, Bash values become observed command templates such as git status --short, cat *, or npm test * --token *. URLs, paths, assignments, quoted values, and option values are replaced with *; safe bare command words remain in plaintext so the report can emit a matching rule. Paths used for path-surface statistics become only workspace, temp, home, external, sensitive, or unknown. Request IDs, project locations, and matched-rule patterns are HMACed. Syntactically valid custom tool names are also retained in plaintext; invalid names stay anonymous as <custom-tool>. The database never stores raw commands, raw paths, URLs, credentials, or non-Bash tool arguments and inputs. Because safe bare Bash words are retained, command templates can contain non-secret filenames or project-specific labels; inspect suggestions before copying them.

Data lives at ~/.pi/agent/extensions/pi-auto-review/policy-audit.sqlite, beside an owner-only HMAC key. The directory is mode 0700; the key, database, WAL, and SHM are mode 0600. Initialization, lock, write, or corruption failures disable auditing and warn once without changing any permission result. A corrupt database is not deleted or rebuilt automatically. Disable new collection with:

{
  "policyAudit": { "enabled": false }
}

Run /auto-review-policy-audit for a durable TUI report that is not sent to the LLM. Options are --days 1..retention, --top 1..50, --min-count >=1, and --scope current|all; defaults are 30, 20, 5, and current.

In addition to the redacted statistics, the report proposes copyable permission-system fragments. --scope current targets .pi/extensions/pi-permission-system/config.json; --scope all targets ~/.pi/agent/extensions/pi-permission-system/config.json. The extension never reads, edits, or rewrites either file. Merge suggested Bash patterns into the existing permission.bash map and place the narrow allow entries after broader matching ask entries, because permission-system uses last-match-wins.

Suggestions are discovered entirely from observed audit data; there is no built-in tool, executable, Git action, package-manager action, or read-only command catalog. Every valid observed permission surface and reliably templated Bash command can qualify, including previously unknown tools and write operations. Environment prefixes, compound syntax, pipelines, redirection, path executables, and unreliable parsing block that Bash template. Forwarded requests are statistics-only because the requester's cwd is not available. A candidate needs at least --min-count successful ask-path approvals and no denial, gate error, unavailable confirmation, blocked structural variant, or forwarded-only evidence in the selected window. Existing policy/infrastructure allows do not count as evidence of user friction. Repeated approval is evidence of user preference, not independent proof that a capability is safe.

Schema v2 migrates v1 aggregates transactionally. Old totals remain visible, collecting_since is preserved, and recommendations_since records when safe recommendation evidence began; pre-migration data cannot produce a suggestion.

The report is an extension-owned custom entry. It is deliberately not exposed as an Agent tool or packaged skill, so neither a tool schema nor skill metadata is added to the model context. Permission changes remain a separate, explicit user operation.

PI_AUTO_REVIEW_AUDIT_FILE remains a test/release observation sink. It is not an input to this audit and supplies no RTK token or parsing metrics.

Telemetry

Every actual model call emits an internal review_attempt; each approval emits one review_complete. Events contain stable status/error classes, timings, usage counters, evidence metadata, and prompt-part counts. They do not contain prompt or response text, provider errors, credentials, headers, or URL query values. Usage is marked unknown_provenance when pi-ai cannot distinguish provider counters from initialized values, and unavailable when absent.

Real-model smoke test

For a controlled real-model smoke test, load only the provider, reviewer, sandbox, and audit listener:

PI_AUTO_REVIEW_ALLOW_UNTRUSTED_DEV=1 \
PI_AUTO_REVIEW_SMOKE_AUDIT_PATH=/tmp/pi-auto-review-smoke-audit.jsonl \
PI_AUTO_REVIEW_BASELINE_ID=reviewer-check \
PI_AUTO_REVIEW_BASELINE_CACHE_STATE=cold \
PI_AUTO_REVIEW_BASELINE_RUN_ORDER=1 \
PI_AUTO_REVIEW_BASELINE_SAMPLE_SET=reviewer-v1 \
PI_AUTO_REVIEW_SMOKE_TRIGGER=baseline-v1 \
pi --no-extensions --no-skills --no-prompt-templates --no-context-files \
  --no-builtin-tools --no-session --print \
  --extension /trusted/path/to/provider/extensions/index.ts \
  --extension ./packages/pi-auto-review/src/index.ts \
  --extension ./packages/pi-sandbox/src/index.ts \
  --extension ./scripts/real-model-smoke-audit.ts \
  --model provider/reviewer-model \
  "Run the configured synthetic reviewer baseline sample, then reply done."

The listener submits filesystem-write, network, delete, Git-push, and forwarded subagent boundaries. No represented operation is executed; the main-agent request is aborted after the samples finish. --no-builtin-tools ensures Bash comes from pi-sandbox rather than Pi's built-in implementation.