pi-voice-input

Voice-to-prompt extension for the pi coding agent. Speak into your microphone; a fully local pipeline denoises the audio, detects speech boundaries, transcribes it with Whisper, and drops the text straight into pi's prompt editor — or auto-sends it. No cloud, no API keys.

flowchart LR
    Mic[🎙️ Mic] --> Denoise[RNNoise denoise]
    Denoise --> VAD[Multi-cue VAD]
    VAD --> Whisper[whisper-rs]
    Whisper --> Pi[pi prompt editor]

Features

• Single utterance — /voice: listen, transcribe, insert into the editor
• Auto-send — /voice send: transcribe and submit as a prompt
• Continuous dictation — /voice start / stop
• Keyboard toggle — Alt+V
• Agent tool — voice_input: lets the LLM ask you to speak
• Barge-in — speaking during TTS playback silences the audio so the mic doesn't capture it
• Live feedback — real-time status widgets (listening / speaking / transcribing)

Requirements

• Linux with a working microphone (PulseAudio / PipeWire / ALSA, via cpal)
• pi coding agent
• Rust toolchain (the first build compiles whisper.cpp — allow a few minutes)
• ~140 MB disk for the default Whisper model

Install

1. Download a Whisper model

mkdir -p ~/.cache/whisper
curl -L -o ~/.cache/whisper/ggml-base.en.bin \
  https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-base.en.bin

Any model path works via WHISPER_MODEL=/path/to/model.bin.

2. Build the native transcriber

The native/ directory vendors both Rust crates — no external path dependencies:

cargo install --path native/live_whisper_transcription

The binary lands in ~/.cargo/bin/live_whisper_transcription. For GPU acceleration, add --features cuda (also available: vulkan, metal, openmp).

3. Install the pi extension

pi install git:github.com/benjaminjamesxyz/pi-voice-input

Or symlink the extension file manually:

ln -s "$PWD/extensions/voice-input.ts" ~/.pi/agent/extensions/pi-voice-input.ts

If the binary isn't on PATH, point the extension at it with LIVE_WHISPER_BIN=/path/to/binary.

Usage

How it works

extensions/voice-input.ts           pi extension (TypeScript, loaded via jiti)
native/live_whisper_transcription   CLI transcriber: capture → DSP → VAD → Whisper
native/clean_voice_dsp              DSP library: RNNoise denoise, resampling, VAD

The extension spawns the binary with --ipc --no-save and consumes JSON events (ready, speech_start, segment, done, …) from stdout — a strictly typed bridge, compiled clean under tsc --strict.

Troubleshooting

• "No Whisper GGML model file found" — no model in ~/.cache/whisper or ~/.local/share/whisper; set WHISPER_MODEL or download one (step 1).
• "Binary: ❌ Not found" from /voice status — build step 2, or set LIVE_WHISPER_BIN.
• Silence / no speech detected — check mic permissions and input device with arecord -l.

Development

npm install
npm run typecheck   # strict tsc --noEmit against pi's published types

License

To be added — native/clean_voice_dsp is dual-licensed MIT / Apache-2.0 (see LICENSE-MIT, LICENSE-APACHE).