pi-gateway

A Pi Coding Agent extension that bridges your AI workflows to llama.cpp, LiteLLM, and Headroom — live model discovery, loading, and switching, all from within Pi.

---

⚡ Quick Start

pi install npm:@faks/pi-gateway

Or directly from GitHub:

pi install https://github.com/faks/pi-gateway

# Point it at your server
export LLAMA_SERVER_URL="http://127.0.0.1:8080"

# Open Pi → run /models → pick a model → go

That's it.

---

🧠 What It Does

Capability	Details
Auto-discovery	Scans your server and surfaces every available model — no manual config
Live status	🟢 Loaded · 🟡 Loading · 🔴 Failed · 🔵 Sleeping · ⚪ Unloaded · ⛔ Unauthorized
Load / Unload / Switch	Manage models from Pi's command palette — no CLI needed
Multi-model router	Full support for llama.cpp's multi-model presets
Proxy support	Connect to LiteLLM or Headroom proxies
Thinking budgets	Configurable token budgets mapped to Pi's thinking levels (off → max)
Real-time progress	SSE streaming for load progress — falls back to polling automatically
Multiple servers	Semicolon-separated URLs to connect to several servers at once
API auth	Built-in /login flow or manual auth.json configuration

---

🏗 Architecture

┌─────────────────────────────────────────────────┐
│                   Pi Coding Agent                │
│                                                  │
│  /models ──► CommandManager ──► ServerManager    │
│  model_select ──► EventManager ──► Server        │
│                                                  │
│  Server (llama.cpp / LiteLLM / Headroom)        │
│  ├── ApiClient ──► /v1/models, /health, /props   │
│  ├── SSEManager ──► Real-time load progress      │
│  └── Model (Router / Single / Legacy)            │
└─────────────────────────────────────────────────┘

---

📦 Installation

pi install https://github.com/faks/pi-gateway

---

⚙️ Configuration

URL resolution follows a simple priority chain:

Priority	Source	Example
1️⃣	Project config (.pi/settings.json)	"llamaServerUrl": "http://127.0.0.1:8080"
2️⃣	Environment variable	LLAMA_SERVER_URL="http://127.0.0.1:8080"
3️⃣	Global settings (~/.pi/agent/settings.json)	"llamaServerUrl": "http://127.0.0.1:8080"
4️⃣	Default	http://127.0.0.1:8080

Multiple Servers

Separate URLs with ; or ,:

LLAMA_SERVER_URL="http://127.0.0.1:8080;http://10.0.0.5:8081"

Each server gets its own provider and model list.

LiteLLM Proxy

{ "litellmUrl": "http://localhost:4000" }

export LITELLM_URL="http://localhost:4000"

Note: LiteLLM proxies use polling (no SSE support).

Custom Server Names

Give your servers human-readable names instead of URLs:

{
  "llamaServerUrl": "http://127.0.0.1:8080",
  "llamaServerName": "Local Dev"
}

export LLAMA_SERVER_NAME="Local Dev"

For LiteLLM, use litellmName instead.

Authentication

Run /login in Pi and select API key, or configure manually in ~/.pi/agent/auth.json:

{
  "llama-server=http://127.0.0.1:8080": {
    "type": "api_key",
    "key": "your-key"
  },
  "litellm=http://localhost:4000": {
    "type": "api_key",
    "key": "your-litellm-key"
  }
}

---

🖥 Usage

Prerequisites

Start your llama.cpp server:

# Multi-model router mode
llama-server --models-preset path/to/presets.ini --port 8080

# Single model mode
llama-server --model path/to/model.gguf --port 8080

# With API auth
llama-server --model path/to/model.gguf --port 8080 --api-key your-secret

# With auto-sleep
llama-server --model path/to/model.gguf --port 8080 --sleep-idle-seconds 300

Commands

Command	Description
/models	Browse models with live status — load, switch, or unload
/models info	Show detailed info for all models
/models unload	Unload all loaded models (router mode only)

Model Actions

From the /models browser:

• Load & switch — Load an unloaded model and switch to it
• Switch — Swap to an already-loaded model
• Unload — Free memory (router mode)
• Retry — Retry a failed load
• Info — View model details (ID, capabilities, context size)
• Cancel — Abort the current operation

Thinking Budgets

Control how many tokens the model uses for reasoning:

Level	Tokens	Description
off	0	Thinking disabled
minimal	1,024	Short reasoning steps
low	2,048	Light reasoning
medium	8,192	Balanced (default)
high	16,384	Extended reasoning
xhigh	-1	Unlimited (extended)
max	-1	Unlimited (maximum depth)

Customize in settings:

{
  "thinkingBudgets": {
    "minimal": 256,
    "low": 1024,
    "medium": 2048,
    "high": 4096
  }
}

Model Selection Event

Switching models via Pi's model picker automatically loads the requested model on the server — no manual /models needed. The extension listens for model_select and keeps the server in sync.

---

🔍 Model Modes

The extension auto-detects your server's mode:

Mode	How It Works
Router	Reads meta.n_ctx from /v1/models when loaded, or --ctx-size/--fit-ctx from server args
Single	Reads meta.n_ctx from /v1/models
Legacy	Reads max_model_len from /v1/models, falls back to n_ctx from /props

Falls back to 128000 context if nothing is available.

---

📊 Status Indicators

Icon	Status	Meaning
🟢	Loaded	Model is active and ready
🟡	Loading	Model is currently loading
🔴	Failed	Load failed — retry available
🔵	Sleeping	Model is idle (requires --sleep-idle-seconds)
⚪	Unloaded	Model not loaded on server
⛔	Unauthorized	API key required

---

🤝 Dependencies

Peer dependency	Purpose
@earendil-works/pi-coding-agent	Pi Coding Agent SDK
@earendil-works/pi-tui	Pi TUI SDK

---

📚 Docs

• Getting Started
• Configuration
• Authentication
• Usage
• Troubleshooting

---

🐛 Getting Help

1. Check GitHub Issues
2. Run /models to retry with full timeout
3. Check Pi logs: cat ~/.pi/agent/logs/*.log | grep -i "llama"

---

Built by Oskars Germovs — AGPL v3 + BSL 1.1