pi-jev-router

简体中文

pi-jev-router is a pi extension. It registers one virtual model, jev/auto, and picks a physical model for every request pi sends to it.

The point is to plan on a strong model and let a cheaper model carry out the plan. The classifier only rates text. It reads no model names and no prices.

flowchart TB
  R["agent-loop request"] --> Q{"request kind"}
  Q -- "first request of a stage" --> C{"classify: probability reaches threshold?"}
  C -- "yes" --> CX["models.complex"]
  C -- "no" --> ST["models.standard"]
  Q -- "after an eligible successful write, when handoff is allowed" --> IM["models.implementation"]
  D["direct request (compaction, title)"] --> DR["models.direct → models.implementation → models.standard"]

Three strategies decide how often the router re-reads the prompt.

Installation

pi install npm:@caph42/pi-jev-router

Then select Auto (Jev) in /model, or start pi with pi --model jev/auto.

Update with pi update --extensions. Installing the package does not touch an existing jev-router.json.

Requirements

• pi: it must provide virtual models, persistent branch ids and buildSessionProjection(). The suite passes against pi 1.4.0; the command is under Development.
• Node 20 or newer, as declared in engines. The tests for this release ran on Node 24.15.0.
• Models: the defaults are the openai-codex models gpt-6.1-sol (complex work, high thinking level recommended), gpt-6.1-terra (everything else), and gpt-6-luna (implementation, max thinking level recommended). Each configured model should appear in pi --list-models under provider; the fields and how to change them are under route fields and thinking levels.
• Classifier (optional): a model of pi's classifier type, configured inside pi. The default is typesafe/jev-latest; adapters, credentials, and how to turn classification off are under classifier endpoints.

Strategies

The router plans on either models.complex or models.standard. Before classifying, it reuses an available previous planning model. The three diagrams below show the main paths under the default config; setting models.implementation to null or false turns handoff off.

adaptive (default)

flowchart TB
  B["new user message / compaction boundary"] --> C{"classifier available?"}
  C -- "no" --> K["keep the current model"]
  C -- "yes" --> P["complex probability p"]
  P --> S{"current slot"}
  S -- "not complex and p reaches threshold" --> UP["upgrade to models.complex"]
  S -- "complex and p at most downgradeThreshold after minDwellTurns" --> DN["downgrade to models.standard"]
  S -- "standard and p below threshold" --> W["a successful write in handoffTools"]
  W --> G{"every guarded condition holds and implementation is on?"}
  G -- "yes" --> I["hand off to models.implementation"]
  G -- "no" --> H["stay on models.standard"]
  S -- "implementation and p below threshold" --> IM2["stay on models.implementation"]

fixed

flowchart TB
  A["first user message"] --> C["classify once, or reuse an available previous planning model"]
  C --> P["plan on models.complex or models.standard"]
  P --> W{"first successful edit / write of the turn?"}
  W -- "yes" --> I["hand off to models.implementation for the rest of the branch"]
  W -- "no" --> H["keep the current model"]
  N["later user messages"] --> H

record

flowchart TB
  A["first user message"] --> P["same route as fixed"]
  P --> W{"first successful edit / write of the turn?"}
  W -- "yes" --> I["hand off to models.implementation"]
  N["later user messages"] --> S{"slot is models.complex?"}
  S -- "no" --> L["classify and log proposedModel with applied false"]
  S -- "yes" --> K["keep the current model"]

When the first classification is unavailable, the router selects models.standard. When a later classification is unavailable, the router keeps the current model.

A missing initial model, a missing models.direct target, or a missing models.standard model fails the request with a configuration error. A missing handoff target or upgrade target keeps the current model. A model saved in state that is no longer in the catalog is replaced with models.standard, and that recovered state locks handoff.

A classifier call sends the projected text with maxRetries set to 0, and it stops at the local deadline. An eligible retry stays on the failed model; Retries lists the conditions. Cancellation aborts routing and returns no candidate state. A direct request skips classification and saved state, and cancellation stops it the same way.

Configuration

The default adaptive strategy needs no config file when an OpenAI Codex login can see the default models. This file turns on diagnostics:

{
  "diagnostics": { "record": true }
}

A Codex login covers the default models, and the default classifier needs a TypeSafe credential. With no usable classifier, adaptive has no probability for the current input. It reuses an available planning model or starts on models.standard, guarded handoff stays off, and upgrades stay off. To reach models.implementation without a classifier, set "adaptive": { "handoffPolicy": "first-write" } or switch to "strategy": "fixed".

pi reads the configuration at startup, so restart pi after an edit. jev-router.example.json lists every built-in default.

Configuration files

pi reads the files in this table.

Nested objects merge. An array in a later file replaces that array. A missing file is valid, a malformed file fails startup, and an unknown key prints a warning and is ignored.

Route fields

classifier.temperature is read by adapters that rank label probabilities; other adapters ignore it. Omitting classifier.temperature or classifier.timeoutMs keeps the inherited value.

The deadline field keeps the name adaptive.decisionTimeoutMs so older files still parse. 3000 milliseconds is the compatibility default.

Classifier endpoints

classifier.provider and classifier.id name a model whose pi type is classifier. The router does not parse endpoint URLs. The catalog entry selects the adapter.

llama-cpp-classify renders the classifier state twice. Set classifier.timeoutMs long enough for tokenization, templating, and the probability read. Raw label probabilities from llama.cpp are often overconfident; a classifier.temperature greater than 1 divides the logits before normalization, which pulls a probability near 0 or 1 back toward the middle.

Models on openrouter, vercel-ai-gateway, and opencode proxy the System One format and use that provider's credentials.

A classifier missing from the catalog yields missing-classifier, and an adapter failure yields error. With no previous planning model to reuse, initialization selects models.standard; a later failure keeps the current model. Neither result allows guarded handoff.

Set threshold again after you change the classifier, the model, or classifier.temperature. Each of those changes the probability scale.

Adaptive fields

Handoff fields apply to adaptive. fixed and record treat a successful edit or write as a write, including a successful nested call.

An empty adaptive.handoffTools array turns adaptive handoff off. bash and codemode are rejected. A successful wrapper tool is not a write. A custom name is valid when success from that tool means the work moved to implementation.

Virtual model and diagnostics

After the first route, the virtual model reports the routed model's context window and max output.

strategy, adaptive, and diagnostics reject null and other invalid values. classifier.temperature and classifier.timeoutMs reject null as well. models.implementation, models.direct, and classifier accept null or false.

Routing decisions

The first request

On initialization, the router reuses an available previous planning model. With no planning model available, it classifies the current native user entry once.

The native user entry is the last projection source whose original role is user. The classifier reads that entry's projected visible text. A converted custom message, a summary, or bash output does not replace that source. The entry can come from the SDK or another extension, so a user role does not mean a person typed the text. A transform that exists for one request does not replace the stored projection.

A probability at or above threshold selects models.complex. A lower probability selects models.standard. When classification is unavailable, initialization selects models.standard, and a later failure keeps the current model.

Handoff to the implementation model

Guarded adaptive handoff requires all of the following:

• The resulting slot is models.standard. The complex probability was valid, below threshold, and at most adaptive.handoffThreshold. A reused planning model has no stored probability, and guarded handoff needs one.
• The current native projected text is non-empty, within promptLimit, and has no image. The classifier input still matches the input used for the decision, which the state tracks as a SHA-256 digest.
• A successful call from adaptive.handoffTools appears after both the current native user entry and stageStartEntryId.
• The context-token estimate is finite, zero or greater, and at most adaptive.maxHandoffContextTokens.
• models.implementation is still in the catalog.

A complex selection locks handoff. Disabled classification, a missing classifier, an invalid answer, an error, or a timeout does not allow guarded handoff. A new user boundary replaces the previous handoff evidence.

A nested call with status ok counts even when the outer tool later fails or its record is incomplete. The call shows that a write finished, not that the whole task finished. A failed or unfinished nested call does not count. Tool arguments do not count, prose that says the file was written does not count, and a successful wrapper tool does not count.

The check uses entry ids on the current branch. A write before stageStartEntryId, or before the current native user entry, does not start this handoff. Without nested tool records, pi recognizes a successful outer write and does not recognize a nested write.

Upgrades and downgrades

Each new native user entry is assessed once, including on complex. A probability at or above threshold upgrades a non-complex slot. Complex downgrades to standard when the probability is at or below adaptive.downgradeThreshold, the current input is complete, and adaptive.minDwellTurns evaluated user turns have elapsed in the slot. Incomplete input prevents downgrade, and unavailable classification keeps the model. A slot change resets the dwell count, and a branch may upgrade and hand off more than once.

Each new compaction entry also triggers one assessment, on the next user or continuation request. Compaction does not increase the dwell count. A new user entry and a compaction together cause one assessment, and a retry does not consume a pending compaction. Re-enabling handoff sets a new write boundary, so earlier writes cannot trigger it.

To keep the older first-write rule, set the policy in config:

{ "strategy": "adaptive", "adaptive": { "handoffPolicy": "first-write" } }

first-write allows handoff from standard even when classification was unavailable. It skips the text checks and the context-token limit. Complex remains locked.

Retries

A retry neither classifies nor hands off. It keeps the failed model when that model is still available on the configured provider, and the failed id matches a configured model id or the id already stored in state. It also keeps the failed model's thinking level. When those checks fail, the retry uses valid state, an available previous planning model, or models.standard.

A retry whose selected model matches the valid state model keeps the handoff state, including when it falls back to that model. A retry onto a different model locks handoff. Repeating a retry does not append a second copy of the same state. pi stores the returned state before it sends the main request, and a later model failure does not undo that stored state.

Deadlines and cancellation

adaptive.decisionTimeoutMs is the local deadline, and it applies to every strategy. When classifier.timeoutMs is set, the classifier call uses that value instead. If the deadline is reached first, a later success or failure is ignored.

Caller cancellation or a service abort stops routing and returns no candidate state. A direct request skips classification and saved state, and cancellation stops that request too.

Saved state and migration

Adaptive state uses schemaVersion 1 and policyVersion "adaptive-v3". Older phase and model state, adaptive-v1, and adaptive-v2 migrate with handoff locked, and existing compactions are marked handled during migration.

A configuration change, a provider change, or a lost stageStartEntryId ends the previous handoff. Invalid adaptive state fails with invalid adaptive state; start a new session or switch strategy to fixed.

fixed reads adaptive phase and model state. Setting strategy to fixed restores fixed handoff. Fixed state records the provider, the handoff lock, and stageStartEntryId. State follows resume, compaction, and branch ancestry, and a new session starts with no saved routing state.

Record mode

record uses the fixed route and classifies the initial prompt. On a later native user entry, while the slot is not models.complex, record reads the same input adaptive would use, then keeps the current model or names an upgrade.

stderr carries a line with proposedModel, available, and applied set to false. The line is printed when debug is off. When debug is on, the debug log carries the same decision. record keeps the fixed model, and the logged model is the upgrade it would have selected. The fields include no quality score and no cost delta.

The last assessed entry id is stored, so reopening a session does not classify that entry again. A retry and a continuation do not run record classification.

The /jev-router command

Every subcommand is reachable through one namespaced command:

/jev-usage is gone; use /jev-router usage with the same --session and --json flags. pi treats an unknown slash command as an ordinary message, so the old spelling is sent to the model instead of failing. Update scripts that still use it.

status and usage call no model and change no state. status reports the configuration loaded at startup, not live routing and not whether a model or classifier is available. It also names the config files pi actually read, so a missing or unexpected file is visible. Restart pi after editing a config file. Both subcommands work on a host without a UI by writing to stdout.

A usage query reports the same summary as summarizeClassifierUsage(manager, virtual, scope). virtual defaults to provider jev and id auto, and scope defaults to branch. The apis field lists the classifier adapters recorded on those attempts.

Each classification attempt has a random call id, a pending snapshot, and a terminal snapshot. A snapshot is counted once, and a real retry after a cancellation is a separate attempt. The summary counts attempts recorded by this version.

Reported tokens are not a bill. Missing usage, pending calls, and unknown or zero catalog prices stay visible. When any cost is unknown, knownCost is the subtotal of the known costs. A timeout or a cancelled call can still create service usage that the summary does not show.

Reopening a session and compacting keep the usage entries. A branch query follows ancestry, a session query includes abandoned branches, and a fork includes ancestor usage. Adding a parent total to a fork total counts the shared usage twice.

On pi 1.2.0, /session counted only host usage, so direct classifier usage sat outside it. After a pi upgrade, compare host accounting with /jev-router usage again before combining totals. Main assistant outcome records are already in pi's session accounting.

Thinking levels

The routed model is the catalog model that runs the request. Dispatch reads settings.modelThinkingLevels for the virtual model, then for the routed model, then uses the request thinking level. A retry that keeps the failed model uses failed.thinkingLevel before those keys. pi clamps the level to the routed model.

Levels are configured per physical model; jev-router has no per-slot thinking field. To give the defaults high and max, set them in pi's settings.json:

{
  "modelThinkingLevels": {
    "openai-codex/gpt-6.1-sol": "high",
    "openai-codex/gpt-6-luna": "max"
  }
}

A key is provider/model, for example openai-codex/gpt-6.1-terra or the virtual model jev/auto.

Diagnostics

Set diagnostics.record to true to store entries.

A pi-jev-router.decision entry includes decisionReason, the source model, the target model, and the requested thinking level. It also includes classifierStatus, classifierApi, the probability, the duration, safe usage, and the context estimate.

A random decisionId links that entry to the pi-jev-router.outcome entry for the final main assistant response. The outcome includes the actual thinking level, the stop reason, the failure category, the reported usage, and the elapsed time after routing.

A missing outcome stays missing. The outcome is not an HTTP log and does not record time to the first token. A direct request writes neither entry.

The router checks session and branch ancestry before it writes an outcome, and it clears pending links at session and agent boundaries. A classifier result that arrives after the deadline is ignored. A result is ignored when the session, the branch, or the input is no longer the one that started the call.

Decision entries and classification records omit prompts, code, tool arguments, response bodies, credentials, and raw errors. A failed write prints diagnostics-unavailable and leaves the route unchanged, which can leave usage incomplete or pending.

Diagnostics stay out of the model context. They are not a second billable usage source, because main assistant outcome usage is already in pi's session totals.

Privacy

By default, classifier input is the projected visible text of the last native user entry. A converted custom message, a summary, or bash output does not replace that source. Images, thinking, and tool data are excluded.

fixed and the initial record classification send a state.prompt payload, and adaptive sends state.current. With no source entry, or with empty projected text, the router does not call the classifier.

Adaptive classification and later record checks always include state.recentTurns and state.historyPartial alongside the current user text. The removed adaptive.contextMode is ignored with a warning. Fixed and initial record classification remain unchanged.

A previous turn holds native user text and successful assistant text, in time order. Assistant text counts when stopReason is stop or toolUse. The history omits tools, thinking, images, system messages, custom messages, summaries, and failed or aborted responses.

The newest complete turns have to fit both adaptive.historyMaxTurns and adaptive.historyCharLimit. The router drops whole oldest turns until the history fits. If the newest turn alone exceeds a budget, the history is empty, and the router does not replace that turn with an older one.

Current text uses promptLimit, separate from the history budgets. Those budgets leave out the classifier instructions and the JSON overhead. The router does not call a summary model to shorten history.

Classification reads the text that compaction and context edits left in the projection. The router sets historyPartial when that history is incomplete, and the flag does not report the classifier probability.

User text and assistant text can contain a pasted secret or file contents, and the filter does not remove those secrets. Set classifier to null or false to stop every classifier request.

Troubleshooting

• jev/auto is missing from /model when the package is not loaded, pi has not been restarted, or a checkout is loaded beside the npm package. Install one copy and restart pi.
• A configured model is missing from pi --list-models when provider or models does not match the catalog. Set those fields to ids from that command.
• Classification fails or times out. Initialization then uses models.standard when no previous planning model can be reused, and a later failure keeps the current model. Set classifier to null to turn classification off.
• missing-classifier means the catalog has no classifier model for that provider and id. error means the adapter request failed, from a missing credential, an unreachable endpoint, or a server error. For llama.cpp, raise classifier.timeoutMs before treating the router as failed.
• A config edit applies on the next pi start, because pi reads the sources in configuration files. An unknown key prints a warning.
• A write is ignored when pi has no nested tool records and the successful write is not the outer tool result.
• adaptive never upgrades and never performs guarded handoff when classification is unavailable. Without a reusable planning model it starts on models.standard. Configure a classifier, set "handoffPolicy": "first-write", or switch to fixed.

Development

Run the routing and lifecycle tests:

npm test

Include the public SessionManager cases:

PI_ROUTER_HOST_MODULE=/absolute/path/to/pi-coding-agent/dist/index.js npm test

Without PI_ROUTER_HOST_MODULE, the host cases skip and say so. Against pi 1.4.0 on Node 24.15.0 the suite reports 313 tests in 32 suites. Without the host module, 305 pass and 8 skip; with it, all 313 pass.

The tests use fixed classifier and catalog fixtures. They check routing and session lifecycle. They do not measure classifier accuracy, production latency, task quality, or cost. Adapter tests supply normalized results through registry mocks, and live sign-in and the wire protocol are outside those tests.

scripts/live-agent-smoke.ts bills the configured accounts. It runs separately from npm test, uses the configured codex-sub2api sol and luna models and typesafe/jev-latest, and runs in a temporary directory. The core scenario counts as validated only when the live classifier meets the guarded handoff threshold; a run that never hands off leaves that scenario unvalidated. This release has not rerun the live scenarios.

Review scripts/live-agent-smoke.ts, then run it:

PI_ROUTER_HOST_MODULE=/absolute/path/to/pi-coding-agent/dist/index.js \
  JEV_ROUTER_LIVE=1 node scripts/live-agent-smoke.ts fixed

The other scenario names are core, cancel, and retry. Release steps for maintainers are in RELEASING.md.

License

MIT. The terms are in LICENSE.