pi-extension-nvidia-nim

NVIDIA NIM exposes a lot of reasoning models through an OpenAI-compatible API, but their thinking controls are not actually compatible with each other. Pi's standard --thinking option may get ignored, or reasoning might need model-family-specific request fields.

pi-extension-nvidia-nim adds a model-aware nvidia-nim provider for Pi. It maps Pi thinking levels to the request format each NVIDIA NIM family expects, while keeping Pi's built-in openai-completions streaming path.

Features

• ~83 curated models for chat, reasoning, code, and vision
• 123 scraped entries, filtered, deduplicated, and family-mapped
• 8 handler-based thinking formats: DeepSeek V4, DeepSeek NIM, thinking-budget, Nemotron system modes (3 variants), MiniMax inline, Qwen chat-template, plus native pi handling for reasoning-effort
• Per-model chat_template_kwargs injection (thinking effort, budgets, system-message toggles) and request content-array normalization for older models
• No custom streaming. Uses pi's built-in openai-completions.

Which NVIDIA provider?

Provider	Use it when
Built-in nvidia	You need basic NVIDIA model access with minimal configuration
nvidia-nim	You need model-family-aware reasoning controls and NIM-specific compatibility

Install the npm package pi-extension-nvidia-nim. It registers a separate Pi provider named nvidia-nim; it does not replace Pi's built-in nvidia provider. Both can be installed and used side by side.

Install

pi install npm:pi-extension-nvidia-nim

Configure

1. Get an API key

Sign up at build.nvidia.com (free tier, 40 requests per minute, 1,000 inference credits on signup, no credit card required).

2. Set the credential (pick one)

Option A: Environment variable

export NVIDIA_NIM_API_KEY="nvapi-..."

PowerShell:

$env:NVIDIA_NIM_API_KEY = "nvapi-..."

NVIDIA_API_KEY is accepted as a fallback for backward compatibility with pi's built-in nvidia provider.

Option B: Auth file (~/.pi/agent/auth.json)

Add an entry so pi resolves the key automatically for all NVIDIA providers:

{
  "nvidia-nim": { "type": "api_key", "key": "nvapi-..." }
}

Option C: Interactive login

Run /login nvidia-nim in pi's interactive mode and select the API-key login. The key is stored under the nvidia-nim provider in auth.json and managed automatically. Selecting built-in nvidia authenticates a different provider.

3. Select a model and test reasoning

pi --provider nvidia-nim \
  --model deepseek-ai/deepseek-v4-flash \
  --thinking high \
  -p "Give me a short solution to this coding problem: reverse a linked list."

This smoke test should show Pi's structured reasoning indicator and a separate final answer. Do not copy private reasoning content into issue reports. You can also select models interactively with /model or Ctrl+P. Look for the nvidia-nim/ prefix in the model picker.

Design

• Uses pi's built-in openai-completions streaming. No custom streamSimple.
• Model-specific quirks (thinking formats, extra body kwargs, compat flags) are handled via before_provider_request and pi's compat system.
• Family-based config in config/model-families.ts (46 families, first-match-wins) drives thinking format routing and model metadata.
• All cost fields are $0 because NVIDIA NIM is free tier.
• Works alongside pi's built-in nvidia provider. Use nvidia-nim/... for the full feature set, nvidia/... as a basic fallback.

Comparison with pi's built-in nvidia provider

Pi ships a built-in nvidia provider with about 20 models. This extension (nvidia-nim) fills in the gaps with more models and thinking support:

Aspect	Built-in nvidia	This extension nvidia-nim
Models	~20 curated	~83 curated (full NIM catalog)
Thinking formats	None	8 handler-based formats + reasoning-effort
Request normalization	No	Yes
Rate-limit warnings	No	Yes (429 handler)
API key	NVIDIA_API_KEY env	NVIDIA_NIM_API_KEY + NVIDIA_API_KEY fallback

Use nvidia-nim/... for the full feature set, nvidia/... as a lightweight fallback.

Models with thinking support

DeepSeek V4, Kimi K2.6, Qwen3, GLM-5.2, MiniMax M3, Seed OSS, Nemotron (Ultra, Super, 3-Super), GPT-OSS, StepFun, Inkling, and Laguna XS 2.1.

• GLM-5.2 exposes three Pi thinking choices: off, high, and max. It uses boolean NIM thinking control via enable_thinking and clear_thinking, plus top-level reasoning_effort (high or max). Nested effort inside chat_template_kwargs is ignored by hosted NIM.
• StepFun: live NIM probing confirmed reasoning_effort requests return separate reasoning_content. Step-3.7 Flash stays always-on on the hosted endpoint even when enable_thinking: false is sent.
• MiniMax M3 has a three-mode thinking toggle (disabled, adaptive, enabled) mapped from pi's thinking levels.
• Nemotron uses system-message-driven thinking modes (detailed think, /think, and reasoning budget variants).
• DeepSeek V4 Flash and Pro: live NIM requests confirmed content-only non-think and separate reasoning_content for high and max via chat_template_kwargs. Pi exposes only off, high, and max for these models.
• DeepSeek V4 puts reasoning_effort inside chat_template_kwargs, with off mapped to none and max mapped to max.

Verified compatibility matrix

A probe-passed transport result means the request shape produced the expected response. It does not guarantee every tool or prompt combination works.

Model	Reasoning control	Request	Response	Streaming	Tools
DeepSeek V4 Flash	off / high / max	chat_template_kwargs (probe-passed)	reasoning_content (probe-passed)	probe-passed	documented
GLM-5.2	boolean toggle + high/max effort (probe-passed)	chat_template_kwargs + top-level reasoning_effort (probe-passed)	reasoning_content (probe-passed)	probe-passed	claimed
MiniMax M3	disabled / adaptive / enabled	thinking_mode (probe-passed)	reasoning_content (probe-passed)	probe-passed	documented
Step-3.7 Flash	low / medium / high; always-on hosted	reasoning_effort (probe-passed)	reasoning_content (probe-passed)	probe-passed	claimed
Inkling	always-on; no toggle	no control exposed	reasoning_content (probe-passed)	probe-passed	unknown
Laguna XS 2.1	on / off toggle	enable_thinking (probe-passed)	reasoning_content (probe-passed)	probe-passed	unknown

The remaining models work through their family rules, but don't call them live-verified unless they appear in this matrix or have a matching compatibility report.

Additional capabilities

• Rate-limit warnings: shows HTTP 429 responses with retry-after info.
• Request content normalization: converts [{type:"text"}] to plain strings for older models that reject structured content arrays.
• 46-family regex routing: assigns thinking formats and compat settings across all ~83 models.
• Per-model reasoning effort mapping: non-standard values like off or minimal are mapped automatically to what the model expects.
• No custom streamSimple: uses before_provider_request event hook, avoiding provider conflicts.

Troubleshooting

Handling transient NIM 429 errors

This extension relies on Pi's built-in retry handling. For occasional NVIDIA NIM rate-limit responses, I currently use the following global setting in ~/.pi/agent/settings.json as a practical starting point:

{
  "retry": {
    "enabled": true,
    "maxRetries": 4,
    "baseDelayMs": 2000,
    "provider": {
      "maxRetries": 1,
      "maxRetryDelayMs": 60000
    }
  }
}

Pi retries the failed turn after approximately 2, 4, 8, and 16 seconds. The single provider retry can help with an immediately transient 429, while keeping the retry count limited. This configuration applies globally to Pi and is not required by the extension. Persistent 429 responses usually indicate throttling or exhausted quota; wait or select another NIM model instead of continually increasing retries.

• Confirm the selected model starts with nvidia-nim/. Pi's built-in nvidia/ provider uses a different catalog and compatibility path.
• If --thinking appears ignored, run npm run probe -- --model=... from the extension checkout and check the selected model's family and verification status.
• If a model is missing, refresh the catalog and confirm the exact NIM model ID still exists on its NVIDIA model page.
• If authentication fails, check NVIDIA_NIM_API_KEY first, then the NVIDIA_API_KEY fallback, and verify the variable is visible to the Pi process.
• Tool calling and reasoning are tracked separately. A reasoning-capable model is not automatically tool-call verified.
• Enable NIM_DEBUG=1 only when needed. Avoid sharing payload logs without removing prompts and other sensitive data.

Verification

The probe never runs on startup and does not write credentials, prompts, or full responses. Run it when you have an NVIDIA credential:

npm run probe -- --model=z-ai/glm-5.2 --output=glm-5.2-probe.json

Use --cases and --timeout-ms to skip models that are slow to respond.