pi-nimble-router

A local-first model router for Pi. Register one virtual model, nimble/auto, and Pi routes each turn between a fast local model, an optional remote model, and an optional stronger remote model. The routing decision is made by a local model through Ollama. There is no third-party routing gateway. When a turn is routed remotely, Pi sends it to the provider you configured and authenticated.

The idea mirrors the jev-router pattern (a virtual model that picks a physical model per request), but replaces the paid Vercel/TypeSafe Jev call with a local Ollama call. That keeps the routing decision local and avoids adding another service to the request path.

Where Nimble fits

Nimble is for a specific trade-off: keep the routing decision on the machine running Pi, then use a remote model only when trusted configuration and provider authentication make that route available. The task still goes to the selected provider when a remote tier is chosen. Local routing therefore reduces one decision-layer hop, but it does not make a remote generation private.

Choose a remote decision layer when its classification features, adaptive policies, or hosted telemetry are more valuable to your workflow than keeping that decision local. Choose Nimble when the local boundary is the requirement. The two approaches solve different operational problems.

Install

# install the pinned public GitHub release
pi install git:github.com/mdabydeen/pi-nimble-router@v0.2.2

# or install the published npm package
npm install pi-nimble-router

# or, from a clone
pi install ./

# or load for one invocation while developing
pi --extension ./index.ts --model nimble/auto

Pi's package manager supports pinned Git refs, so the GitHub command keeps the install on the reviewed v0.2.2 release. The npm package is published as pi-nimble-router; use the Git ref when you need the exact reviewed Pi release and npm when you prefer registry installation.

Use

pi --model nimble/auto

Start and type /model nimble/auto. The footer shows which physical model each response used, e.g. nimble/auto • low → ollama/qwen3.8:27b-mlx or ... → openai/gpt-5.

Configure

Merge nimbleRouter into ~/.pi/agent/settings.json (or <cwd>/.pi/settings.json, which overrides non-exfil fields), then /reload.

{
    "nimbleRouter": {
        "router": "ollama/llama3.2:3b",
        "local": "ollama/qwen3.8:27b-mlx",
        "cloud": "anthropic/claude-sonnet-4-6",
        "heavy": "anthropic/claude-opus-5-5",
        "default": "local",
        "timeoutMs": 15000,
        "cloudWhen": "architecture, hard debugging, cross-cutting design, research"
    }
}

router/local/cloud/heavy are provider/id. The router and local targets just need their provider configured (ollama always is). The remote targets are only used when that provider is logged in; otherwise every turn stays local. A heavy verdict downgrades to cloud, then local, if that model isn't logged in. default must be a real model.

How it decides

• New user turn (reason === "user"): the router model is asked, in one short local call, to answer local, cloud, or heavy. The last mention wins (a thinking model narrates first, its verdict lands last); an unparseable or empty reply falls through to default.
• Within a turn (continuation, retry): the model from the last successful/failed response is kept, so prompt caches and thinking signatures stay valid.
• Off the agent loop (direct, e.g. compaction summaries): goes straight to the local target; the router is not called.
• Failures are soft: router unavailable, timed out, or unparseable → default. A remote target that isn't configured or logged in (or a heavy requested but not available) downgrades to cloud, then local.
• Cancellation is soft: if a request is already cancelled, the router call is skipped and the configured default is returned. If cancellation arrives while the router is running, the call is aborted and follows the same fallback path. With NIMBLE_DEBUG=1, the cancel record contains only the router reference, cancellation marker, and fallback target; it does not echo task text or the router response.

Thinking routers

If your router model reasons (a "thinking" model), it can spend its whole output on a preamble and leave the verdict out of the response's content. The defaults handle this two ways: reasoning_effort: "none" asks ollama to skip the preamble (the verdict then arrives in content in ~0.1s), and temperature: 0 makes the one-word choice deterministic. Non-thinking models ignore the suppression key. To run a router the opposite way, set "routerOptions": {}. Debug any decision with NIMBLE_DEBUG=1 pi ... (it logs the router ref, response length, and parsed choice to stderr; it intentionally does not echo the raw response).

Security

• Prompt injection of the router. Routing is decided by a local model, whose prompt contains your (possibly pasted, untrusted) text. The router input is therefore wrapped in explicit UNTRUSTED DATA markers and the parser only reads the router's own final line, so a cloud/local token embedded in pasted content can no longer force a route. This bounds but cannot fully defeat injection; the verdict still must come from outside the delimited region.
• Exfil targets come from trusted config only. cloud/heavy/cloudWhen may be set in the global/user settings, never in a project-local <cwd>/.pi/settings.json, so a cloned repo cannot quietly point routing at a cloud provider you have logged into. A project file may still tune the non-exfil fields.
• Fail-closed on uncertainty. A timed-out, unavailable, or unparseable router call routes to your default (local by default); a cloud that is unset or not logged in is ignored, so an unexpected turn never surprises its way to the cloud.

Files

Development

# routing logic self-check (Node strips TS types; no deps)
npm test
# or, directly
node selfcheck.ts

Every push and pull request runs the same self-check, a redacted-debug scan, a package dry-run, and a whitespace check in GitHub Actions. These checks verify the repository contract; they are not a benchmark or a production-readiness claim.

If the documented install or fallback behaviour does not match a test environment, use the installation feedback template with a sanitized, reproducible example. The evaluation checklist organises a local-only baseline and fallback checks before you add an authenticated remote tier. Do not include credentials, private prompts, or private file paths.

MIT.