pi-inferx-provider

A pi coding-agent extension that registers InferX (https://model.inferx.net) as a model provider.

InferX is a serverless GPU inference platform exposing an OpenAI-compatible, vLLM-backed endpoint (/endpoints/v1). This extension discovers its models and applies the small set of compatibility shims its vLLM serving stack needs, so InferX models "just work" in pi.

Features

• Provider registration — registers the inferx provider with https://model.inferx.net/endpoints/v1 as base URL.
• Live model discovery — when INFERX_API_KEY is set, models are fetched from /v1/models (with a 5-minute on-disk cache keyed by base URL + key hash, so the round-trip is skipped on subsequent launches).
• Self-refreshing public catalog — the endpoint catalog page (https://model.inferx.net/catalog/endpoints) is publicly readable, so the provider scrapes it at startup (24-hour on-disk cache) to pick up new endpoints as InferX publishes them — even before the API key is set or when /v1/models is unreachable.
• Embedded catalog floor — the current catalog is also embedded in the package, so there is always a model list even if both live sources fail.
• Model metadata — context windows from the catalog (DeepSeek V4 Flash: 1M, GLM 5.2 / Qwen3.6 / Agents-A1 / Ornith: ~260K, …), reasoning enabled where the model supports thinking, off for the -no-thinking variant.
• vLLM thinking control — DeepSeek and GLM models use chat_template_kwargs.thinking.enabled (chat-template), verified against the live endpoint: both families separate reasoning (streamed as delta.reasoning / reasoning, which pi reads into the thinking block) with clean content. Qwen endpoints are registered non-reasoning — live testing showed they render their chain of thought into plain content with an empty reasoning field, so pi does not wait on thinking blocks that never arrive.
• system role instead of developer — vLLM chat templates generally reject unknown roles; pi's developer system prompt is downgraded via compat.supportsDeveloperRole: false.
• max_tokens field — vLLM uses max_tokens, not max_completion_tokens (compat.maxTokensField).
• Transient-error retry — wraps the provider stream with a 1 s / 3 s / 8 s backoff ladder for connect-time blips (cold-start 5xx, gateway resets), and requests provider-level retries so Retry-After on 429 is honoured via pi-ai. Bad-key errors (401/403) are never retried.

All behaviour is scoped to the inferx provider; other providers are unaffected.

Requirements

• pi (the @earendil-works/pi-coding-agent CLI), v0.80 or newer.
• An InferX API key in the INFERX_API_KEY environment variable (required to make requests; models are listed from the public catalog even without it).
• Node 18+ (for global fetch).

Install

pi install npm:pi-inferx-provider

Then set your key and select a model:

export INFERX_API_KEY="ix-..."        # PowerShell: $env:INFERX_API_KEY = "ix-..."
pi
# /model -> pick an inferx-* model

Add the export line to your shell config (~/.zshrc, ~/.bashrc) to keep the key across sessions.

Other useful commands:

pi list                              # show installed packages
pi update npm:pi-inferx-provider     # update to the latest release
pi remove npm:pi-inferx-provider     # uninstall
pi install -l npm:pi-inferx-provider # install into this project only (.pi/npm/)

Alternatively, list the package in the packages array of your pi settings.json (~/.pi/agent/settings.json) and pi will install it on next start:

{
  "packages": [
    "npm:pi-inferx-provider"
  ]
}

If you previously had a local inferx.ts in ~/.pi/agent/extensions/, delete it after installing this package to avoid registering the provider twice.

Configuration

Variable	Required	Description
INFERX_API_KEY	yes*	Your InferX API key. *Required to make requests; without it the models are still listed from the embedded catalog but every request fails with an auth error.
INFERX_DEBUG	no	Set to true for per-request debug logging and a metrics summary on exit.
INFERX_PROVIDER_RETRIES	no	Provider-level retries for InferX traffic (default 2). These are what honour a Retry-After header on 429. Set to 0 to disable.

The base URL is https://model.inferx.net/endpoints/v1.

Models

As of the embedded catalog, the provider registers (thinking flags reflect live verification against the endpoint):

Model ID	Context	Thinking
deepseek-v4-flash	1,000,000	yes*
deepseek-v4-flash-0731	1,048,576	yes*
glm-52	260,000	yes
Agents-A1	262,000	no
Devstral-2-123B-Instruct-2512-int4-AutoRound	128,000	no
Ornith-1.0-35B-FP8	262,000	no
Qwen3-Coder-Next-FP8	256,144	no
Qwen3.6-35B-A3B-FP8	262,000	no
Qwen3.6-35B-A3B-fp8-no-thinking	262,000	no

* DeepSeek separated reasoning correctly in non-streaming tests; its streaming key could not be re-confirmed (the endpoint was at capacity), but pi reads both reasoning_content and reasoning stream keys, so either shape works.

Notes from live testing:

• glm-52 and deepseek-v4-flash emit thinking as a separate reasoning field with chat_template_kwargs.thinking (verified). GLM also thinks by default even without kwargs; thinking: false disables the reasoning field but the model then writes its analysis into content — leave thinking on.
• Qwen3.6-35B-A3B-FP8 (the "thinking" variant) always writes its chain of thought into content — with and without enable_thinking kwargs in any shape — so it is registered non-reasoning. For clean short answers prefer Qwen3.6-35B-A3B-fp8-no-thinking.
• Qwen3-Coder-Next-FP8 ignores thinking kwargs entirely (clean direct answers, empty reasoning field).
• The endpoint accepts the developer role; the provider still downgrades to system for safety across vLLM templates.

When INFERX_API_KEY is set, live /v1/models discovery replaces this table with the server's authoritative list; ids not in the catalog keep their discovered metadata and get conservative defaults (text input, no reasoning). New endpoints published on the catalog page are picked up automatically at startup (see Model resolution); to force a re-scrape sooner, delete ~/.cache/pi-inferx-provider/catalog.json.

Notes & limitations

• Thinking controls are chosen to match vLLM serving conventions for each model family. If a model's thinking toggle differs on the InferX side, override the model in ~/.pi/agent/models.json (e.g. thinkingFormat, chatTemplateKwargs).
• The transient retry handles brief gateway blips (seconds), not sustained outages. For a sustained outage, wait or switch models.

Development

pnpm install
pnpm run typecheck

To try a local change without installing the package, load the source directly — -ne disables other extensions so nothing else interferes:

pi -ne -e ./src/index.ts --provider inferx --model deepseek-v4-flash

Note that an installed copy of this package registers the same provider name as a local one. Remove the installed copy (pi remove npm:pi-inferx-provider) before testing locally.

Publishing

Releases follow Semantic Versioning. Before publishing a new version:

1. Bump version in package.json and add an entry to CHANGELOG.md under the new version heading.

2. Run the checks:

   pnpm run typecheck
   pnpm pack --dry-run   # confirm the tarball contains only src/, README, CHANGELOG, LICENSE
   

3. Publish (the tarball is what pi installs; the pi.extensions field points pi at ./src/index.ts):

   npm publish            # or: pnpm run publish
   

4. Tag the release in git:

   git tag v0.1.0 && git push origin main --tags
   

First-time publishing requires npm authentication:

npm login

Users then install with pi install npm:pi-inferx-provider (see Install).

Internals

The package exports one default function — pi's extension entry point. Everything below lives inside it; nothing else is importable.

Model resolution

Three tiers, each falling through to the next on failure (registration never hard-fails):

1. Live API — if INFERX_API_KEY is set, try the on-disk cache ($XDG_CACHE_HOME/pi-inferx-provider/models.json, mode 600, 5-minute TTL, invalidated by base URL + key hash), then GET /v1/models with the key. Success is authoritative and replaces the list below.
2. Public catalog scrape — fetch the unauthored endpoint catalog page (https://model.inferx.net/catalog/endpoints) and parse the model table (id, context length, upstream provider). Cached for 24 hours in $XDG_CACHE_HOME/pi-inferx-provider/catalog.json. This is what keeps newly published endpoints visible without waiting for a package update.
3. Embedded catalog — the table in this source file, so there is always a floor even fully offline.

Models found by tiers 1–2 that are not in the embedded catalog get conservative metadata: context from the live source, maxTokens 16384, and reasoning enabled only when the family is known to think (qwen/deepseek/glm) and the id does not say no-thinking.

Retry layers

Two independent layers covering different failures:

Layer	Handles	Timing
pi-ai retryProviderRequest	408, 409, 429, 5xx	Retry-After when sent, else exponential backoff with jitter, capped by retry.provider.maxRetryDelayMs (60 s)
This extension's stream wrapper	Connect-time transient errors	Fixed ladder: 1 s / 3 s / 8 s

pi ships retry.provider.maxRetries: 0, which leaves the first layer inert, so the extension requests retries for InferX traffic (INFERX_PROVIDER_RETRIES) to get Retry-After honoured. The stream wrapper only retries when the first event is a transient error (safe: a connect-time failure streams nothing); 401/403 auth errors are never retried.

Thinking control

Verified against the live endpoint:

• deepseek / glm models → thinkingFormat: "chat-template" with chatTemplateKwargs: { thinking: { $var: "thinking.enabled" } }, sending chat_template_kwargs: { thinking: true|false }. Both families stream reasoning under delta.reasoning (pi reads reasoning_content / reasoning / reasoning_text), so thinking lands in pi's thinking block.
• Qwen endpoints render thinking into plain content, so their presets are reasoning: false (no kwargs sent). The qwen-chat-template format (chat_template_kwargs.enable_thinking) is kept in compatFor for future Qwen deployments that separate reasoning, but no registered model uses it today.
• Unknown ids discovered later default to non-reasoning, except new deepseek/glm ids (families verified to separate reasoning), unless the id says no-thinking.

License

MIT