pi-vision-watcher

Give text-only pi models vision — images are described by a vision model you pick, then handed off to text-only coding models without interrupting your workflow.

pi package · npm · Issues

What it does

Paste an image, attach a file, or have the agent read one — pi-vision-watcher describes it in the background with your chosen vision model and feeds the description to whatever text-only model you're using (DeepSeek, local models, etc.).

• Connected-only picker: /vision-watcher shows only vision-capable models from providers where you actually have credentials.
• Batching & cache: multiple images across parallel tool calls are batched into one describer request; cached images (SHA-256) are never re-described.
• False-vision healing: aggregator providers sometimes flag text-only models as multimodal, causing HTTP 400s. The extension proactively forces handoff for them and auto-heals models.json in-process.
• Thinking controls: adjust reasoning effort (off – max) for reasoning-capable vision models.
• Fallback chains: automatically falls back to backup vision models when the primary is rate-limited or down.

Install

pi install npm:@bismawy/pi-vision-watcher

Then run /vision-watcher to pick your vision model (or set it directly: /vision-watcher model openai/gpt-4o). Handoff is on by default — just switch to any text-only model and work as usual.

Commands

Command	Action
/vision-watcher	Interactive picker for connected vision models
/vision-watcher model <provider/id>	Set primary vision describer directly
/vision-watcher status	View current configuration
/vision-watcher auto <on|off>	Toggle automatic handoff (default: on)
/vision-watcher add <provider/id>	Force handoff on a specific model
/vision-watcher remove <provider/id>	Remove model from forced handoff list
/vision-watcher thinking <level>	Configure reasoning effort
/vision-watcher timeout <ms>	Set the base per-image description timeout (default 45000)
/vision-watcher prewarm <on|off>	Describe pasted images at paste-time (opt-in)
/vision-watcher async <on|off>	Inject pasted-image descriptions asynchronously when no matching read wins (alias: fallback)
/vision-watcher clear	Clear the configured vision model
/vision-watcher enable / disable	Toggle extension active state
/vision-watcher help	List all subcommands

async is the async clipboard fallback and has nothing to do with the fallbackModels failover chain.

In the picker:

Key	Action
space	Select the highlighted model as the primary describer (press again to clear)
ctrl+q	Toggle the highlighted model in/out of the failover chain (marked 🔁, max 3). Not a mnemonic, and that's on purpose: ctrl+f is pi's find-text (bound since pi 0.85), alt+f is editor word-right, and ctrl+alt+f never survives Windows conhost/Windows Terminal. pi 0.85 also binds ctrl+q to app.message.followUp, which is inert while a picker is open — if it ever double-fires, f2 is the free fallback (unbound in 0.84 and 0.85)
ctrl+t	Walk the thinking ladder: off → minimal → low → medium → high → xhigh → max → off → …
ctrl+a	Toggle async paste handoff
enter / ctrl+s	Save (primary and chain together)
esc	Cancel

The detail pane always shows the current configuration — primary, chain, thinking, async handoff — so every keypress shows exactly what will be saved. space is left to the search box while a filter query is present, so multi-word searches like gemini 3.8 stay typeable.

How it works

{
  "enabled": true,
  "visionModel": "openai/gpt-4o",
  "fallbackModels": [],
  "autoHandoff": true,
  "handoffModels": [],
  "thinking": false,
  "thinkingLevel": "medium",
  "describeTimeoutMs": 45000,
  "prewarmPastedImages": false,
  "asyncClipboardHandoff": false,
  "maxTokens": null,
  "cacheMax": 50,
  "maxDescriptionLines": 0
}

Most fields have sane defaults — visionModel is the only one you normally set.

fallbackModels is the failover chain: when the primary describer fails (timeout, rate limit, auth), each entry is tried in order — fallback 1 fails, fallback 2 runs, and so on until one returns a description. The picker caps the chain at 3 entries (ctrl+q). Set it from the picker instead of hand-editing the file.

• Failed vision calls log with stack traces to ~/.pi/agent/logs/pi-vision-watcher/errors.log and degrade gracefully to [Image: description unavailable].
• When a model falsely advertises image capability and 400s, the error is captured on message_end, modelOverrides.<model>.input = ["text"] is written to models.json, and the registry refreshes in-process.

bun install
bun run test          # Vitest suite (240+ unit tests)
bun run typecheck
bun run lint:dead

License

Distributed under the MIT license.

Developer

Developed and maintained by Bisma.