pi-local

A Pi extension for configuring and switching between multiple local LLM inference engine connections.

Features

• Multiple connections — configure several local inference servers (LM Studio, oMLX, llama.cpp, llama-swap, etc.) and switch between them
• Auto-detection — queries each endpoint for its models and shows whatever that server advertises: display name, size, context window, model type, and which models are loaded right now
• Model loading — load/unload models on servers that support it (oMLX, LM Studio, llama-swap)
• Persistence — your default provider and model are restored automatically on Pi restart
• macOS keychain — offers to store API keys in the macOS keychain via security commands
• Reasoning support — forwards pi's thinking levels as reasoning_effort, using each server's advertised vocabulary where available

Supported backends

Backend	Detection	Load/Unload	Metadata read
oMLX	/v1/models/status + /api/status	Yes	alias, size, context window, max tokens, model type, reasoning vocabulary, pinned/favorite
LM Studio	/api/v1/models	Yes	display name, size, context window, quantization, publisher, format, architecture, reasoning
llama-swap	/v1/models, recognised by owned_by	Yes	display name, context window, load state, vision
OpenAI-compatible	/v1/models	No	context window and size (llama.cpp), context window inherited by LoRA adapters (vLLM)

The extension tries oMLX first, then LM Studio, then falls back to the generic OpenAI listing. llama-swap answers that generic listing, so it is identified from the response itself — every card it serves carries owned_by: "llama-swap" — and that is what earns it load/unload, which the generic tier has nothing to call.

Servers disagree about where the same quantity lives, so each is read where it is actually advertised: llama.cpp nests meta.n_ctx and meta.size, vLLM puts max_model_len flat on the card (LoRA adapters inherit their base model's through parent), and llama-swap mirrors the operator's capabilities.context into meta.n_ctx alongside name, status.value and capabilities.

Load and unload

Each server is asked its own way:

Backend	Load	Unload
oMLX	POST /admin/api/models/{id}/load	admin session + POST .../unload
LM Studio	POST /api/v1/models/load	POST /api/v1/models/unload
llama-swap	GET /props?model={id}	POST /api/models/unload/{id}

llama-swap has no load endpoint: dispatching a request at a model is what swaps its server in. /props?model= is the cheapest route that dispatches — a GET for properties, so no tokens are generated, and the process is up and health-checked when it returns. Two quirks of doing it that way, both confirmed against a live server:

• /props is a llama.cpp route, so an upstream that is not llama.cpp answers 404. The model is loaded by then — the load is a side effect of dispatching, not of the upstream answering — so that 404 is not reported as a failure. llama-swap's own rejections are, and they are recognisable by their {"src":"llama-swap",...} envelope.
• Unload answers OK as plain text, not JSON, so nothing there parses a body.

llama-swap may swap out one model to load another; the picker does not model that, it re-queries after every action and shows what is actually running.

Reasoning levels

Thinking levels (/thinking) are forwarded as the OpenAI-style reasoning_effort field on /v1/chat/completions. What each backend advertises decides which levels you get:

Backend	Levels	Wire format
oMLX	whatever /v1/models/status advertises (reasoning_effort_options)	reasoning_effort, enable_thinking fallback
llama-swap	off, minimal, low, medium, high, xhigh	reasoning_effort forwarded untouched
OpenAI-compatible (llama.cpp, vLLM, ...)	off, minimal, low, medium, high, xhigh	reasoning_effort passed through
LM Studio	pi default (off … high)	reasoning_effort

llama-swap is treated like llama.cpp because it behaves like llama.cpp: it forwards the request body untouched — reasoning_effort and chat_template_kwargs included, confirmed by reading the forwarded request back out of its own capture endpoint — and resolves the model name internally without changing what the upstream sees. (It rewrites model only when the operator sets useModelName or a filter.) So thinking levels do exactly what the engine behind it does, and everything below about llama.cpp applies to llama-swap-fronting-llama.cpp. One llama-swap-specific thing to be aware of: sendLoadingState: true (default false) makes it inject its own loading progress into the reasoning field of a stream, so a thinking block can carry text that did not come from the model.

For servers we cannot identify (llama.cpp, including llama-server), the level is sent verbatim and off is sent as reasoning_effort: "none" — llama.cpp treats "none" as "don't think" (enable_thinking = false) rather than forwarding it to the chat template. Levels only take effect if the model's chat template reads reasoning_effort; llama.cpp ignores it otherwise. The server must not be started with --reasoning off (default auto is fine), and --reasoning-effort on the command line only sets the default that a request can override.

Commands

/local-endpoints

Add or remove connections. Each connection is identified by its base URL.

Manage Connections
> Remove: http://127.0.0.1:1234
  Add new connection
  Done

On macOS, if you enter a direct API key, you have the option to store it in the keychain. The key is then referenced via a !security command.

/local-model

Select a connection and model. Shows server stats where the server reports them (oMLX: version, loaded/loading counts, memory headroom) and, per model, its name, size, context window, model type and whether it is loaded.

Installation

You can either install this directly with the pi command via npm:

pi install npm:@monroewilliams/pi-local

or check out this repository and add the extension to your Pi configuration in ~/.pi/agent/settings.json (useful if you want to modify it to better suit your purposes):

{
  "extensions": [
    "/path/to/pi-local"
  ]
}

Then reload Pi (/reload) and run /local-login to add your first connection.

API key formats

The API key field accepts all Pi auth key formats:

Format	Example	Description
Direct key	sk-1234567890abcdef	Stored as-is
Environment variable	$MY_API_KEY or ${MY_API_KEY}	Resolved via resolveConfigValue
Shell command	!security find-generic-password -s 'pi-local' -a 'http://...' -w	Shell execution, stdout used
Empty	(leave blank)	No authentication

On macOS, direct keys are optionally stored on the keychain.

Storage

• Connections — stored in ~/.pi/agent/auth.json keyed by base URL
• Default provider/model — stored in ~/.pi/agent/settings.json (defaultProvider / defaultModel)
• Model metadata — cached alongside the connection in auth.json (display name, contextWindow, maxTokens, reasoning, reasoningEffortOptions, modelType, pinned, favorite) so the picker has something to show before the first live query

Development

npm install
npm run typecheck    # TypeScript check
npm run check        # Biome + TypeScript + tests
npm run format       # Auto-format

License

MIT