pi-tier-router

Pay for difficulty, not for every message. Routine work runs on the cheap model; only genuinely hard issues escalate to the strong one.

Project page: rioliu.github.io/pi-tier-router

Route a Pi coding session between a cheap flash model and a strong pro model, based on how hard the issue actually is.

This is the same tiering idea as Claude Code's haiku/sonnet/opus model levels — most work belongs on the fast, inexpensive model, and a minority of hard issues deserves the strong one. Unlike a fixed tier list, the two roles are configuration: point it at any model family (MiMo, DeepSeek, an OpenAI-compatible endpoint, or a mix of providers).

What it does

Pi exposes the router as a virtual model, router/auto. While it is selected, every prompt is decided on its own the moment it arrives, and that decision serves the whole agent run (tool loop included):

prompt arrives (before_agent_start)
  0. local router reads the prompt: 0 ms, no tokens, deterministic
       clear signals  -> decided right away (flash or pro)
       mixed/missing  -> abstain
  1. on abstain: the decision store is consulted first
       a similar past verdict is reused (signal-profile overlap or trigram
       match, k-NN vote) -> no model call at all
  2. otherwise: flash model rates the issue itself (rating + confidence)
       confidence >= 0.8  -> decision is final, nothing else is consulted
       and the verdict is stored for future prompts
  3. unsure  -> ask Jev, but only if a credentialed classifier model is available
                (TYPESAFE_API_KEY, or any other Jev-capable provider in Pi's registry)
  4. no Jev  -> keep the flash model's lean
  5. neither -> keep the current model (flash for a new session); routing stops

• One prompt, one decision — the next prompt re-decides, so a hard task can escalate without locking the whole session to pro.
• Local first: routine and obviously hard prompts never reach a model. /auto-router status reports how often the local layer decided on its own. Signals cover English and Chinese (stack traces are language-neutral), and the scan looks at the first 32 KB only — a 1 MB paste still decides in ~0.3 ms, and past the window the layer abstains rather than guess.
• It remembers: chain verdicts from the abstain zone are stored next to the extension — a JSONL journal on disk and a fixed typed-array ring in memory (2 000 entries, ~285 KB, never grows with age). Similar prompts reuse them instead of paying for a rating call; neighbours that disagree keep abstaining. The file is removed together with the extension, and deleting it by hand is always safe — the store rebuilds itself.
• Jev is optional, never a dependency — consulted only when the local layer abstains and the flash model is unsure.
• Compaction summaries always run on the flash model.
• The model-based decision is bounded at 10s and uses minimal reasoning, so it never holds up a turn.

Install

pi install npm:pi-tier-router

Or straight from GitHub while the npm package is pending:

pi install git:github.com/rioliu/pi-tier-router

Install one or the other. Loading both silently breaks the wizard: /auto-router stops dispatching and the literal text is sent to the model instead.

Then select it: /model → router/auto, and press Ctrl+S to make it the default for new sessions. Or set it directly in settings.json:

{ "defaultProvider": "router", "defaultModel": "auto" }

Routing only runs while router/auto is selected. With any other model — including a physical pro model as your default — the extension does nothing.

Configure

The extension owns its config, next to the extension file — so uninstalling the extension takes its config with it:

<install-dir>/extensions/auto-router.json

pi update preserves the file — verified for both npm and git sources. If an older copy is sitting in ~/.pi/agent/extensions/, it is read and moved next to the extension the first time it loads, so uninstalling leaves nothing behind.

{
  "flash-model": "deepseek/deepseek-v4-flash",
  "pro-model":   "deepseek/deepseek-v4-pro"
}

Values are provider/model-id (split on the first slash, so ids containing slashes work, e.g. openrouter/typesafe/jev-1.13). The two roles may live on different providers. Omitted or malformed keys fall back to a MiMo pair.

Create or change it with the built-in wizard:

The wizard writes only its own file. models.json is never touched — availability is read from Pi's live model registry — and the only value read from settings.json is your default model id, used to warn you when it changes.

Behaviour when configuration changes

The extension re-checks at session start and before every prompt, so it follows your model settings instead of going silently dead:

Issues are announced when they appear, not on every prompt.

Watching a decision

AUTO_ROUTER_DEBUG=1 pi
[auto-router] roles flash=... pro=...
[auto-router] local pro conf=0.93 via concurrency

Local decisions print a single line and cost nothing. Only an abstain reaches the store or the model chain:

[auto-router] local abstain; flash rating chain decides
[auto-router] self rated=pro conf=0.92
[auto-router] decided self/pro in 1432ms
[auto-router] stored pro by self (4 entries)

[auto-router] local abstain; flash rating chain decides
[auto-router] memory flash (3 neighbours, 100% agree)      <- no rating call at all

The interactive footer shows the routed model (auto • medium → deepseek-v4-pro) and /session lists cost per physical model.

Disable it

Pick any physical model in /model, or set your default to the pro model — routing then never engages. Removing the config file deactivates the router entirely.

Requirements

Both role models must be registered in models.json with working credentials.

The Jev step is optional: it needs a credentialed classifier model in Pi's registry (TYPESAFE_API_KEY for typesafe/jev-latest, or another Jev-capable provider). Without one, the chain simply keeps the flash model's lean — routing never depends on Jev.

Development

npm test        # node >= 23 (native TypeScript stripping)

The suite covers the local router's signal table, the decision chain, the hook → route() wiring, the decision store (round-trip through the journal, torn lines, ring and journal caps, the agreement gate), config persistence and refresh, change warnings, and the provider/model-id parser. It runs against a temporary copy of the extension, so it never writes into your real Pi directory.

References

The routing logic in this extension is built on the papers below, selected from the Awesome-Routing-LLMs survey — each one directly shapes a layer of the local -> flash rating -> Jev chain. The full analysis is in docs/local-router-research.md.

Local feature routing (layer 0)

1. I. Ong, A. Almahairi, V. Wu, et al., "RouteLLM: Learning to Route LLMs with Preference Data," ICLR 2024. arXiv:2406.18665
2. C. Tran, S. Paracha, A. Hafeez, S. Chen, "Arch-Router: Aligning LLM Routing with Human Preferences," ArXiv 2025. arXiv:2506.16655
3. S. N. Hari, M. Thomson, "Tryage: Real-time, Intelligent Routing of User Prompts to Large Language Models," ArXiv 2023. arXiv:2308.11601

Confidence-gated two-tier escalation (flash self-rating, abstain band)

4. G. Ramírez, A. Birch, I. Titov, "Optimising Calls to Large Language Models with Uncertainty-Based Two-Tier Selection," COLM 2024. arXiv:2405.02134
5. Y.-N. Chuang et al., "Confident or Seek Stronger: Exploring Uncertainty-Based On-Device LLM Routing from Benchmarking to Generalization," NeurIPS 2025 Workshop. arXiv:2502.04428
6. P. Aggarwal et al., "AutoMix: Automatically Mixing Language Models," NeurIPS 2024. arXiv:2310.12963

Memory-based routing (decision store, kNN reuse)

7. Y. Li, "Rethinking Predictive Modeling for LLM Routing: When Simple kNN Beats Complex Learned Routers," ArXiv 2025. arXiv:2505.12601
8. Z. Zhao, S. Jin, Z. M. Mao, "Eagle: Efficient Training-Free Router for Multi-LLM Inference," NeurIPS 2024 Workshop. arXiv:2409.15518

Benchmarks (coverage/risk measurement)

9. Q. J. Hu et al., "RouterBench: A Benchmark for Multi-LLM Routing System," ICML 2024 Workshop. arXiv:2403.12031
10. Z. Huang et al., "RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMs," EMNLP 2025. arXiv:2503.10657

License

MIT