pi-web-voice

A PI WEB plugin for dictation: a Dictate button in the composer turns what you say into prompt text.

• Live dictation streams to Azure Speech, and words appear in the draft while you speak.
• Without streaming, the clip is recorded and sent to a speech-to-text endpoint when you stop, such as Azure Speech or a Whisper-compatible service.
• The Azure Speech key stays on the machine; the page only gets a token that expires.

Requirements

• A PI WEB release later than 2.202610.12. Earlier releases bundle their own copy of this plugin, which wins over an installed one (Settings → PI WEB plugins reports the duplicate id).
• PI WEB opened over HTTPS (or on localhost): browsers allow the microphone only on a secure page. See remote access.
• A speech-to-text service you configure, such as an Azure Speech resource.

Install

pi install npm:@gang-of-beads/pi-web-voice

The installer's Gang of Beads preset installs it too. To follow main instead of releases, install git:github.com/Gang-of-Beads/pi-web-voice.

The plugin has a server part, so restart the PI WEB session daemon, then reload the PI WEB tab. To remove it, run pi remove npm:@gang-of-beads/pi-web-voice and restart the daemon again.

Configure

The composer shows Dictate only once plugins.voice.settings names a service with an endpoint. For Azure Speech, name the service in settings and put the key in serverSettings, which no browser receives (private plugin settings):

{
  "plugins": {
    "voice": {
      "settings": {
        "endpoint": "https://westeurope.stt.speech.microsoft.com/speech/recognition/conversation/cognitiveservices/v1",
        "language": "en-US",
        "streaming": {
          "protocol": "azure-speech",
          "url": "wss://westeurope.stt.speech.microsoft.com/speech/recognition/conversation/cognitiveservices/v1"
        },
        "azureSpeech": { "region": "westeurope", "resource": "example-speech" }
      },
      "serverSettings": {
        "azureSpeech": { "key": "replace-with-your-key" }
      }
    }
  }
}

Then restart the session daemon and reload the tab; a changed settings or serverSettings block applies at the daemon's next start, and the plugin's card shows Restart required until then.

• Live protocols. Only azure-speech streams in this version. A streaming block that names browser, openai-realtime or deepgram, or has no url, shows the reason in the composer instead of recording a clip. streaming.tokenEndpoint is ignored.
• Other services. An endpoint with no azureSpeech block, such as a Whisper-compatible service on the same host, receives the clip as a form upload (file, plus model and language when set), without a credential.
• Sessions. A dictation and its status line belong to the session you started it in. Switching to another session or machine stops the dictation and releases the microphone, and text that arrives afterwards goes into the draft of the session it was started in. A new session finishing its start while you dictate is not a switch: the dictation goes on in it.

Move a key out of public settings

Earlier PI WEB releases read the key from settings.azureSpeech.key. A key there still works, but every browser that opens PI WEB can read it. To move it:

1. Add "serverSettings": { "azureSpeech": { "key": "…" } } to the voice entry, by editing the config file or with a serverSettingsUpdates write.
2. Remove key from settings.azureSpeech.
3. Restart the session daemon, then reload the tab.

The old key has been readable by browsers; rotate it in Azure if that matters for your deployment.

Health

The plugin's card in Settings → PI WEB Plugins says where the key comes from:

When no token can be had, the composer says why and sends nothing.

Privacy

Voice sends your audio only to the services you configure, and only while you dictate.

• Microphone. Pressing Dictate opens the microphone, after the browser's permission prompt; it is released when the dictation stops.
• Live dictation connects from the page straight to settings.streaming.url.
• Recorded clips are uploaded from the page to settings.endpoint, with Authorization: Bearer <token> when an azureSpeech block is set.
• Tokens. The server part runs in the PI WEB session daemon, keeps the key, and exchanges it for a ten-minute token at https://<resource>.cognitiveservices.azure.com/sts/v1.0/issueToken, or https://<region>.api.cognitive.microsoft.com/sts/v1.0/issueToken without a resource. The page asks for a token through the plugin's speech.token operation.
• On disk. The key sits in PI WEB's config file in plaintext; serverSettings keeps it from browsers, not from the machine.

The plugin is machine-specific: with several machines, each machine's own copy uses that machine's settings and key.

Development

npm install
npm run check

npm run check type-checks, runs the tests, and builds two bundles with esbuild: dist/browser/pi-web-plugin.js for the page and dist/server-plugin.js for the daemon. dist/ is committed so that pi install git: works without a build step, and the check fails when it is out of date.

To try a change in a running PI WEB, link the working copy as a local plugin, restart the session daemon and reload the tab:

mkdir -p ~/.pi-web/plugins
ln -s "$PWD" ~/.pi-web/plugins/voice

See PI WEB's plugin documentation for the plugin contract.

Contributing

Issues and pull requests are welcome. Please keep changes small and focused, run npm run check before opening a pull request, and describe what you changed and how you checked it in a running PI WEB.

Releasing

Update version in package.json and package-lock.json, give CHANGELOG.md a section for it, commit, and push a matching v<version> tag. The release workflow checks the package, publishes it to npm, and creates the GitHub Release from that CHANGELOG section.

License

MIT