proper-llm-router

A Pi extension that chooses a model for the first task in a session, switches before generation starts, and leaves later turns on that model. Each pi-subagents child is a separate session and gets its own route.

The router uses one measured judge decision per task, Pi's authenticated model registry, optional CPA account quota data, and fixed cross-provider swaps. It has no project-local service. CPA/CLIProxyAPI is optional.

Routing behavior

Fresh and new sessions move to the placeholder llm-router/auto. Resumed sessions keep their current model. Routing runs only while the selected provider is llm-router; /llm-router or manually selecting llm-router/auto re-arms the next prompt.

The first eligible input follows this order:

1. A configured slash-command pin switches directly and skips the judge.
2. An unpinned bare slash command uses fallbackModel because it has no task text to judge.
3. [[llm-router: <model>]] forces one arm and removes the marker.
4. Every other prompt goes to the judge.

A command pin takes precedence over a sentinel. Slash commands with arguments are normal task text unless pinned.

Judged routes

• The judge receives the first 4000 characters of text, not attached image contents.
• It selects one of seven stable capability slots through strict JSON schema.
• Up to three related measured tasks from exemplars.jsonl are added as evidence when TF-IDF similarity is useful.
• Judge model overrides can replace the model occupying a slot without changing that slot's calibrated use cases.
• A provider-qualified judge model uses Pi's configured provider runtime. When CPA is absent, an unqualified model also uses Pi when it resolves uniquely. The registry path forces one strict route_model tool call.
• Otherwise the router uses <judge.baseUrl>/chat/completions with strict JSON Schema, preserving existing compatible endpoint configurations.
• The judge makes at most two 60-second attempts. Pressing Esc cancels judging, discards the prompt, and leaves routing armed.
• Later turns make no judge call.

Model slots

Slot	Intended use
claude-fable-5	Ambiguous architecture, protocol work, concurrency, migrations, and unclear scope.
claude-opus-5	Cross-component diagnosis, authentication, data-loss risk, and high-impact changes.
claude-sonnet-5	Routine multi-file repository work and test suites.
claude-haiku-4-5	Localized repository fixes and mechanical edits.
gpt-5-6-sol	Subtle standalone correctness, algorithms, and performance work.
gpt-5-6-terra	Fully specified standalone functions, endpoints, or classes.
gpt-5-6-luna	Trivial or mechanical standalone edits.

Repository inspection and agentic tool use belong to the Claude lane. Self-contained work whose code and specification are already in the prompt can use the GPT lane. When two adjacent tiers fit, the judge chooses the stronger one.

Availability, quota, and fallback

The router resolves each arm against Pi's authenticated models. Unqualified model IDs prefer cliproxyapi for backward compatibility, then the direct provider for that model family. Use provider/model-id in overrides or fallbackModel when the same ID exists under several providers.

CPA-backed targets also check CPA's /v1/models catalog while the judge runs. When quotaMaxPct and a CPA management key are configured, the router averages Claude or Codex account usage and treats CPA-backed slots at or above the threshold as down. Usage is cached for 60 seconds. Without an available cliproxyapi model, no CPA availability or quota request runs.

A down slot swaps once to a fixed partner:

• Fable and Sol swap with each other.
• Opus and Terra swap with each other.
• Sonnet swaps to Luna.
• Haiku swaps to Luna; Luna swaps to Haiku.

If both judged choices are down, or judging fails, the router uses fallbackModel without quota-checking that fallback. Quota data failures skip only the percentage gate and produce a visible warning. Direct pins and sentinels fail open: if availability cannot produce a usable swap, they keep the requested arm rather than block the prompt. If a direct target is missing from Pi's registry, routing falls through to the remaining precedence rules.

Notices show judging state, selected model, latency, rationale, command pins, forced routes, swaps, overrides, skipped quota checks, and fallback errors.

Direct model overrides

Use a sentinel in a typed prompt or subagent task:

[[llm-router: claude-opus-5]] Fix the race in the session cache

Names can be an arm key, its default model ID, or a unique fragment such as opus or sol. Unknown names are removed and sent to the judge with a warning.

pi-subagents spawn-time model options are overwritten when the child starts on llm-router/auto. The sentinel is the supported per-child override:

runs.run("retry", {
  agent: "worker",
  task: "[[llm-router: claude-fable-5]] Diagnose the failed migration",
})

Default command pins

Command	Model	Thinking effort
/file	claude-fable-5	xhigh
/triage	claude-fable-5	xhigh
/spec	claude-fable-5	xhigh
/backlog	claude-fable-5	xhigh
/implement-ready	gpt-5-6-sol	xhigh

Pins still use the quota swap. Their effort is applied after the final model switch and is clamped to that model's supported levels. A null effort leaves the current session effort unchanged.

Commands

Command	Behavior
/llm-router	Select llm-router/auto so the next prompt routes again.
/llm-router-config	Open the interactive configuration menu.

The configuration menu can:

• Choose judge model, reasoning effort, and priority service tier.
• Replace judged model slots with any authenticated Pi model.
• Add, repoint, remove, and set effort for command pins.
• Edit the complete JSON config.
• Run a live end-to-end route test.

When cliproxyapi is available, the menu also shows the CPA quota threshold and management key. Without CPA, those actions and CPA-only JSON fields are hidden.

Ultra thinking support

Pi 0.84.2 stops its native thinking controls at max. When the running Pi host exposes the expected compatibility points, this extension adds ultra to Shift+Tab and Pi's thinking selector only for models whose thinkingLevelMap.ultra contains a value. Switching to an unsupported model clamps effort to its highest available level. If the host layout differs, model routing still works without the extra native control. No installed Pi files are modified.

Install

Use Node 22.19 or newer. The extension and ultra compatibility layer are tested against Pi 0.84.2.

Install the published package:

pi install npm:proper-llm-router

For extension development, install a local checkout instead:

pi install /path/to/proper-pi-extensions/proper-llm-router

The package has no runtime dependencies and no build step, so a local install needs no npm install; that command only prepares the development checks below.

The extension registers the llm-router/auto placeholder provider itself at load, so no ~/.pi/agent/models.json edit is needed on install or update. An existing manual entry keeps working — Pi composes it with the registration — and remains the fallback for hosts without pi.registerProvider():

{
  "providers": {
    "llm-router": {
      "baseUrl": "http://127.0.0.1:1/v1",
      "api": "openai-completions",
      "apiKey": "unused",
      "models": [{ "id": "auto" }]
    }
  }
}

The placeholder appears in /model, but a healthy route switches away before any request reaches its dead port-1 URL. Missing execution or fallback registry models can leave the placeholder selected; fix the model registry before retrying.

The seven execution model IDs, fallbackModel, and active override targets must resolve to authenticated models in Pi's registry. They may come from built-in providers, models.json, or extension-registered providers. Exact provider/model-id values remove ambiguity.

CPA remains backward compatible. When authenticated cliproxyapi models are available, unqualified IDs prefer them and CPA availability and quota checks apply. Provide its key through the variable configured by cpaKeyEnv, ANTHROPIC_AUTH_TOKEN by default.

A direct-provider setup can keep the defaults when the same model IDs are available, or qualify ambiguous choices:

{
  "judge": {
    "model": "openai-codex/gpt-5.6-terra"
  },
  "fallbackModel": "anthropic/claude-opus-5",
  "judgeModelOverrides": {
    "claude-fable-5": "anthropic/claude-fable-5"
  }
}

Remove any older direct llm-router.ts extension registration so the package loads once.

Configuration file

Settings live at ~/.pi/agent/llm-router.json. The file is read before every routed prompt. Most edits need no restart; exemplar path changes need a restart after the corpus has loaded, and quota data may remain cached for 60 seconds.

Field	Default	Behavior
judge.baseUrl	http://127.0.0.1:8317/v1	OpenAI-compatible judge API base.
judge.apiKeyEnv	ANTHROPIC_AUTH_TOKEN	Environment variable containing the judge key.
judge.model	gpt-5.6-terra	Judge model ID or provider/model-id. A registry-resolved model uses Pi's provider runtime; otherwise the configured HTTP endpoint is used.
judge.effort	medium	Judge reasoning_effort; null omits it.
judge.fast	false	Sends service_tier: "priority" when enabled.
fallbackModel	gpt-5.6-terra	Model ID or provider/model-id used after judged failure and for bare commands.
cpaBase	http://127.0.0.1:8317	CPA base for model and quota requests.
cpaKeyEnv	ANTHROPIC_AUTH_TOKEN	Environment variable containing the CPA API key.
exemplarsPath	package exemplars.jsonl	Optional measured-outcome corpus.
quotaMaxPct	null	Average lane usage threshold; null disables it.
cpaManagementKey	empty	Plaintext management key, preferred over the environment.
cpaManagementKeyEnv	CPA_MANAGEMENT_KEY	Management-key environment fallback.
judgeModelOverrides	{}	Stable slot to authenticated model ID or provider/model-id for judged routes.
commandPins	five defaults above	Slash command to model and effort mapping.

commandPins and judgeModelOverrides replace their whole default maps when present. Invalid or unreadable JSON falls back to defaults. The loader does not validate field types or URL shapes, so use /llm-router-config when possible. The full JSON editor writes a complete merged config.

Environment controls

Variable	Effect
LLM_ROUTER_OFF=1	Stops automatic startup activation and sentinel help. A session already on llm-router/auto can still route. A pinned workflow command in an interactive session opens a confirm dialog: continue unrouted, or stop the run before the agent starts.
JUDGE_EXEMPLARS=0	Skips measured exemplar retrieval.
CPA_SIMULATE_UNAVAILABLE="arm1,arm2"	Treats exact arm keys as down for swap testing.
CPA_MANAGEMENT_KEY	Default management-key fallback.

Development

npm install
npm run typecheck
npm run test:unit
npm run test:coverage
npm run test:smoke -- ["task text"]

Type checks and unit tests are offline. They include a no-CPA route through Pi provider models. The smoke command then runs one legacy live judge, CPA availability, quota, exemplar, and swap route because the standalone harness has no Pi model registry.