pi-nvidia-nim-diegovisk

NVIDIA NIM provider for pi with the full chat model catalog, per-family thinking/request transforms, and aggressive rate-limit retry for uninterrupted free-tier development.

Forked from pi-extension-nvidia-nim (MIT, Tibbee) with merged pi-nvidia-rate-limit-retry (MIT, Diegovisk).

Why this fork

Pi's built-in nvidia provider ships ~20 models with no per-model thinking transforms. The upstream pi-extension-nvidia-nim package registers nvidia-nim with ~82 chat models and 8+ thinking handlers, but does not retry 429s aggressively. This fork combines both.

Recommended free-dev stack

Role	Model
Auto failover (sticky router)	nvidia-nim/nvidia-router
Primary (tools + vision)	nvidia-nim/minimaxai/minimax-m3
Reasoning fallback	nvidia-nim/nvidia/nemotron-3-super-120b-a12b
Fast fallback	nvidia-nim/meta/llama-3.3-70b-instruct

Set defaultModel to nvidia-router to let the extension pick from the chain above and only switch on rate-limit exhaustion or context overflow.

Install

npm (recommended)

pi install npm:@diegovisk/pi-nvidia-nim@1.1.0

Or add to ~/.pi/agent/settings.json:

{
  "packages": ["npm:@diegovisk/pi-nvidia-nim"]
}

Note: Published as @diegovisk/pi-nvidia-nim because unscoped pi-nvidia-nim is taken on npm.

git

pi install git:github.com/Diegovisk/pi-nvidia-nim@main

Local path (development)

pi install file:C:/Users/diego/projects/pi-nvidia-nim

Configure API key

Set one of:

export NVIDIA_NIM_API_KEY="nvapi-..."
# or (compatible with pi /login and auth.json "nvidia" entry)
export NVIDIA_API_KEY="nvapi-..."

Switch default provider

In ~/.pi/agent/settings.json:

{
  "defaultProvider": "nvidia-nim",
  "defaultModel": "nvidia-router",
  "packages": ["file:C:/Users/diego/projects/pi-nvidia-nim"]
}

Or pin a concrete model:

{
  "defaultProvider": "nvidia-nim",
  "defaultModel": "minimaxai/minimax-m3"
}

Remove npm:pi-extension-nvidia-nim and the standalone ~/.pi/agent/extensions/nvidia-rate-limit-retry.ts to avoid double-wrapping.

Then /reload in pi.

Features

• ~82 curated chat models from scraped NIM metadata + 44 regex families
• Per-model request transforms via before_provider_request (DeepSeek V4, GLM, MiniMax M3, Nemotron, Kimi K2.6, Seed OSS, etc.)
• Content normalization for legacy models that reject structured content arrays
• Rate-limit retry — up to 20 attempts with exponential backoff (env-tunable); covers nvidia, nvidia-nim, and OpenRouter nvidia/*
• Compaction-safe summarization — auto-compact and /compact calls force thinking off so MiniMax/NIM does not 400 during context overflow recovery
• NVIDIA Router — virtual nvidia-router model with sticky per-session routing across the free-dev fallback chain; re-selects only on 429 exhaustion or context overflow
• Catalog refresh — /nim-refresh caches live API model list; npm run refresh-metadata merges new IDs into metadata

Commands

Command	Description
/nim-models	List registered models by family + recommended stack
/nim-refresh	Refresh ~/.pi/nvidia-nim-cache.json from NIM API
/nim-router	Show sticky routed model, fallback chain, and eligibility
/nim-router reset	Clear sticky router state (re-pick from chain head)
/nim-retry-status	Show rate-limit retry stats for this session + audit log path
/nim-retry-status reset	Clear retry session counters

Retry events are appended to ~/.pi/nvidia-nim-retry.log (JSONL). Disable with NVIDIA_RETRY_AUDIT=0.

Router env vars

Variable	Default	Meaning
NVIDIA_ROUTER_CHAIN	minimaxai/minimax-m3,nvidia/nemotron-3-super-120b-a12b,meta/llama-3.3-70b-instruct	Comma-separated fallback order
NVIDIA_ROUTER_MARGIN	1.15	Context headroom multiplier for eligibility filter

Router behavior:

• Picks one concrete model per session (sticky) from the chain.
• Only advances on 429 retry budget exhausted or context overflow.
• Filters candidates by estimated context size and vision requirements.
• Advertises 1M context on the virtual model so pi does not pre-reject long sessions.

Verify after /reload:

[nvidia-router] loaded — sticky failover router for nvidia-router.

Rate-limit env vars

Variable	Default	Meaning
NVIDIA_RETRY_MAX	20	Max attempts per request
NVIDIA_RETRY_BASE_MS	2000	Base backoff
NVIDIA_RETRY_CAP_MS	60000	Max single delay
NVIDIA_RETRY_JITTER	0.2	Jitter fraction
NVIDIA_RETRY_AUDIT	1	Set 0 to disable JSONL audit file
NVIDIA_RETRY_AUDIT_LOG	~/.pi/nvidia-nim-retry.log	Audit log path

Verify after /reload:

[nvidia-rate-limit-retry] loaded — NVIDIA NIM retries: max 20, base 2000ms, cap 60000ms, jitter 20%.

Debug

NIM_DEBUG=1 pi

Inspect ~/.pi/nim-debug.log for final request payloads per model.

Validation matrix

Smoke-test these families at thinking levels off, medium, high:

Family	Example model	Pass criteria
minimax-inline	minimaxai/minimax-m3	Tool calls; thinking toggles
nemotron-3-super-effort	nvidia/nemotron-3-super-120b-a12b	Reasoning streams; no 400
deepseek-v4	deepseek-ai/deepseek-v4-flash	chat_template_kwargs in debug log
qwen-chat-template	qwen/qwen3.5-122b-a10b	Native pi compat sufficient
glm	z-ai/glm-5.2	enable_thinking / clear_thinking correct
Plain	meta/llama-3.3-70b-instruct	No structured-array 400
Rate limit	any free model	[nvidia-rate-limit-retry] logs; succeeds by attempt 2–3

Run unit tests:

npm test

Upstream

• Tibbee/pi-nvidia-nim-provider — base provider + handlers
• Diegovisk/pi-nvidia-rate-limit-retry — deprecated; retry is bundled in this repo

License

MIT — see LICENSE.