pi-onnx

Runs Hugging Face onnx-community models locally inside the pi coding agent using @huggingface/transformers.

Implements a chat provider and several tool calls:

• onnx_embed({ texts: string[] }): array of vectors (and dimensionality).
• onnx_classify({ text, labels? }): top-K labels with scores; when labels is provided, runs zero-shot classification.
• onnx_transcribe({ path, language?, task? }): transcript text and segments.

Install

pi install npm:pi-onnx

Configure

Copy example-config.json from this package as a starting point:

cp example-config.json ~/.pi/agent/pi-onnx.json

Top-level

Field	Type	Default	Notes
cacheDir	string | null	null (HF default)	Forwarded to env.cacheDir.
device	"cpu" | "webgpu" | "wasm" | "gpu"	"cpu"	onnxruntime execution provider hint.
defaultDtype	Dtype	"q4"	Per-model dtype overrides this.
preloadDefaultModel	boolean	false	Preload the first configured model on session start.
models	ModelEntry[]	[Qwen2.5-Coder-0.5B-Instruct]	Each entry becomes a onnx-community/<id> chat model.
discovery	object	enabled, limit 50	Append compatible onnx-community/* models from the HF Hub.
tools	object	embed only	Toggles for onnx_embed / _classify / _transcribe.

Dtype is one of "fp32", "fp16", "q8", "int8", "uint8", "q4", "bnb4", "q4f16".

models[]

Field	Type	Default	Notes
id	string	—	Hugging Face repo path (onnx-community/ prefixed).
name	string	id	Display name shown in the model picker.
contextWindow	number	—	Context window size in tokens.
maxTokens	number	1024	Default max_new_tokens for completions.
dtype	Dtype	defaultDtype	Quantization for this model only.

Only id is required; the onnx-community/ prefix is added if missing. Pinned models are checked against the Hugging Face Hub when possible. Repositories that are not compatible with @huggingface/transformers text generation, such as onnxruntime-genai image-text-to-text exports, are skipped instead of being offered as broken chat models.

Example:

{
  "id": "onnx-community/Qwen3-0.6B-ONNX",
  "name": "Qwen3-0.6B (ONNX, q4)",
  "contextWindow": 32768,
  "maxTokens": 2048,
  "dtype": "q4"
}

discovery

Field	Type	Default	Notes
enabled	boolean	true	Append discovered models to models[].
limit	number	50	Per pipeline tag.
pipelineTags	PipelineTag[]	["text-generation"]	Hugging Face pipeline tags to scan.

Discovery only registers transformers.js text-generation repositories that expose a supported onnx/model*.onnx file. It also records the matching dtype, so models such as gpt-oss-20b-ONNX use q4f16 instead of the global default q4. If a discovered model is also pinned in models without an explicit dtype, discovery fills in that dtype automatically.

tools.embed

Field	Type	Default	Notes
enabled	boolean	true	Toggles onnx_embed.
model	string	onnx-community/all-MiniLM-L6-v2	Any feature-extraction model.
pooling	"mean" | "cls"	"mean"	Pooling strategy.
normalize	boolean	true	L2-normalize output vectors.

tools.classify

Field	Type	Default	Notes
enabled	boolean	false	Toggles onnx_classify.
model	string	onnx-community/distilbert-base-uncased-finetuned-sst-2-english	Classifier or NLI model (zero-shot).
topK	number	5	Maximum labels returned.

tools.transcribe

Field	Type	Default	Notes
enabled	boolean	false	Toggles onnx_transcribe.
model	string	onnx-community/whisper-tiny	Any ASR model.
language	string | null	null	Default language hint (e.g. "en").
task	"transcribe" | "translate"	"transcribe"	Default ASR task.
maxDecodedBytes	number	268435456	Maximum decoded f32 audio bytes to buffer in RAM.

Limitations

• No tool calling support for ONNX chat models.
• Tokens are approximated from the tokenizer.
• First call to a model blocks while weights download.
• onnx_transcribe shells out to ffmpeg (must be on PATH) to decode the input audio file to a Float32Array before inference, capped by maxDecodedBytes.

License

pi-onnx is licensed under MIT. See LICENSE for more information.