pi-llama-chooser

Pi extension: manage remote llama.cpp servers and use their models via /model.

Installation

Install from Pi:

pi install npm:pi-llama-chooser

Or directly with npm:

npm install pi-llama-chooser

Requires @earendil-works/pi-coding-agent and Node.js ≥ 22.19.0.

Usage

Commands

Interactive Menu

Run /llama-chooser config to open a menu:

1. Add a server — step-by-step wizard for name, host, port, protocol, API key
2. Model rows in /model — choose one row per server, or one merged row per identical model
3. Select a server — see status (🟢 online / 🔴 offline / ⚪ disabled, ✓ enabled / ✗ disabled)
4. Server actions — edit values, enable/disable, refresh models, remove

Models

Enabled servers' models appear under the Llama Chooser provider in /model. Each entry is named <server> / <model basename> (<quant>), e.g. myServer / Qwen2.5 (Q6_K), so when the same model is loaded on two machines you get two rows and pick the one you want to answer.

If you would rather have a single row per model with automatic failover, run:

/llama-chooser config dedup on

Identical models are then merged into one entry (<model>@<server1>|<server2>) that routes to the first online server. config dedup status shows the current mode, config dedup off restores one row per server. The choice is persisted and applies immediately.

Config changes apply immediately — no /reload or restart needed. Disabling or removing a server drops its models from /model at once, and enabling or adding one fetches and publishes its models right away. If the model you are currently using disappears, you get a notification to pick another one with /model.

Run /llama-chooser list to see full details per model:

• Quantization level (e.g. Q6_K)
• Context window (e.g. 116K ctx)
• File size (e.g. 27.9 GB)
• Owner (e.g. [llamacpp])

Models with the same basename across servers are automatically deduplicated into a single entry that routes to any online server. See config dedup to turn this off and keep one selectable row per server.

Speed Tracker

The footer shows prefill and generation speed while streaming:

⚡450t/s 🔥85.3t/s

Rates are color-coded (red → orange → yellow → lime → cyan → violet) and use a moving window for generation speed.

Configuration

Servers are stored in ~/.pi/agent/llama-chooser-servers.json. Preferences are stored next to it in ~/.pi/agent/llama-chooser-servers-settings.json. Set the LLAMA_CHOOSER_STORAGE environment variable to use a different path.

[
  {
    "name": "my-server",
    "host": "192.168.1.50",
    "port": 8080,
    "protocol": "http",
    "enabled": true,
    "apiKey": "optional-api-key"
  }
]

• name — unique identifier (no spaces, no __)
• host — IP address or hostname
• port — 1–65535
• protocol — http or https
• apiKey — optional Bearer token for authentication

Settings file:

{
  "dedupModels": false
}

• dedupModels — false (default) lists one /model row per server; true merges identical models into one row that routes to the first online server

Development

npm run typecheck   # TypeScript check
npm run dev         # Run with local extension