pi-model-fallback

Pi extension that switches to a fallback model after provider failures such as 429 rate limits.

What this does

pi-model-fallback watches provider failures and automatically moves Pi to a safer fallback model when a matching rule fires.

Current default:

• source provider: zai
• matching statuses: 429, 500, 502, 503, 504
• fallback model: deepseek/deepseek-v4-flash

When a failure matches, the extension also stores persistent fallback state so future sessions can preselect the fallback model until the cooldown expires.

Install

Requires Pi 1.1 or later for the continuation boundary API.

Install from npm:

pi install npm:pi-model-fallback

Install into the current project only:

pi install npm:pi-model-fallback -l

Or install from GitHub:

pi install git:github.com/eiei114/pi-model-fallback

Try it without permanently installing:

pi -e npm:pi-model-fallback

Commands

/model-fallback:status
/model-fallback:clear [provider/model]
/model-fallback:reset

• status: shows whether fallback is enabled, active persistent entries, and current paths
• clear: drops one persistent entry (or all of them when the argument is omitted) without changing the current model
• reset: clears persistent fallback state and switches back to the remembered original model when possible

Configuration

The extension exposes the model_fallback_config tool for reading, validating, and saving config JSON.

Default config shape:

{
  "version": 1,
  "enabled": true,
  "autoRetry": true,
  "rules": [
    {
      "name": "zai-to-deepseek-flash",
      "matchProviders": ["zai"],
      "statuses": [429, 500, 502, 503, 504],
      "fallback": {
        "provider": "deepseek",
        "model": "deepseek-v4-flash"
      }
    }
  ]
}

Rule fields:

• matchProviders: match all models from a provider
• matchModels: match specific provider + model pairs
• statuses: optional; defaults to 429, 500, 502, 503, 504
• reasons: optional list of failure reason codes (for example context_length_exceeded); when set, the rule only fires if the parsed failure reason is listed. Reasons come from quoted machine codes in the error message (for example "provider_error_code":"context_length_exceeded") or prose such as exceeds this model's context length. Useful for status codes like 400 that should only trigger a fallback for specific causes, such as context-length overflow.
• cooldownMs: optional persistent fallback window
• fallback: target model Pi should switch to

Top-level fields:

• enabled: toggle the extension without deleting rules
• autoRetry: after Pi exhausts its own retries, continue the existing conversation on the fallback model without re-queueing the user prompt (defaults to true; set to false to only switch models). Completed tool results and image inputs remain in context; only the failed assistant entry is excluded.
• transientCooldownMs: persistent cooldown for a 429/529 that carries no quota wording, such as 该模型当前访问量过大 or temporarily overloaded (defaults to 10 minutes). See Cooldowns.
• recoveryProbeMs: how long a transient entry may preselect the fallback before the source model is retried instead (defaults to 5 minutes; 0 disables retrying). A successful turn from the probed model clears the entry.

Rules use first-match order: the first rule whose provider/model and status match wins. Put specific matchModels rules before broad matchProviders rules when they should take priority. model_fallback_config validate, save, read, and status report warning-only diagnostics when a later rule or model entry is completely shadowed by an earlier rule; /model-fallback:status also includes a concise warning summary for the current config. Warning-bearing config remains valid and can still be saved.

Cooldowns

When a rule does not set cooldownMs, the extension uses these defaults:

• 429/529 reporting an exhausted quota (usage limit, insufficient_quota, out of budget, 额度, 余额) → 72 hours
• 429/529 reporting overload or throttling (no quota wording, which is also the default for unrecognized text) → transientCooldownMs, 10 minutes
• 5xx → 10 minutes

A transient entry is not allowed to hide a healthy model for its whole window: once it is older than recoveryProbeMs (5 minutes by default, and deliberately shorter than the 10-minute transientCooldownMs so the retry can actually happen before the entry expires), the next session retries the source model instead of preselecting the fallback, and a successful turn from that model clears the entry. An interrupted turn (Esc) does not count as success, because it proves nothing about the provider. A repeated failure simply persists a fresh entry. Quota entries keep their full cooldown and are never probed, and an explicit rule cooldownMs or a Retry-After/x-ratelimit-reset* header disables early retrying too: those state how long the source must be avoided. Entries written by earlier versions have no transient flag and keep their original behavior; /model-fallback:clear drops them without changing the current model.

When the provider response includes Retry-After or x-ratelimit-reset* headers, those values override cooldownMs, transientCooldownMs, and the defaults for the persisted fallback window.

State and paths

The extension stores:

• config: model-fallback/config.json
• state: model-fallback/state.json

If the package is installed project-locally and the current project references it from .pi/settings.json, those files live under the project .pi/ directory. Otherwise they live under the user agent directory.

Cooldown updates re-read and merge the current state under a shared proper-lockfile lock, including across Pi processes. Reset uses the same lock. State is published by atomic replacement, so readers never see a partially written document; expired entries are ignored on read and removed during the next mutation. Lock acquisition is bounded (about two seconds of retries); a lock or write error is reported rather than overwriting state without the lock. A reset clears entries committed before it; a new failure committed afterward can create a new cooldown.

Behavior notes

• Successful responses do nothing.
• Failed HTTP responses are recorded by after_provider_response; model switching waits until Pi's retry/recovery paths finish at agent_before_settle.
• Assistant error messages parsed at turn_end identify SDK/provider failures that do not emit the normal response hook. When friendly error text omits a status, the extension uses the observed HTTP status and headers from the same model request. Status extraction looks for HTTP-style tokens (for example status 429, HTTP 503, or rate limit) rather than any bare 3-digit number in the message. A status in parentheses is only recognized when an error wrapper keyword precedes it, as in omni API error (503): ...; a parenthesized number with no such wrapper, such as (503) items processed, stays unmatched.
• Anthropic SDK errors with a bare status followed by JSON (for example 429 {"type":"error",...}) are recognized. Status-free Anthropic streaming errors map rate_limit_error to 429, overloaded_error to 529, and api_error to 500; an interrupted stream maps to 502. Add 529 to a rule's statuses if it should cover Anthropic overloads. Supported connection failures without an HTTP response map to 503, including a bare terminated stream error and Request timed out. with an SDK diagnostic suffix. ChatGPT subscription quota errors without a response status map to 429: the Subscription Sharing usage limit message, its raw subscription_sharing_..._usage_limit_exceeded code, and the SDK's ChatGPT usage-limit and workspace-limit wording. Cancellation, refusals, and ordinary numeric prose stay unmatched, and explicit HTTP statuses take precedence.
• Router-relayed stream death maps to 502 in both wordings: the provider-level stream ended without a terminal event and the router-level stream closed before a terminal event (for example CLIProxyAPI's upstream stream closed before a terminal event (last event: response.output_item.done)). Strict raw-text detection needs the terminal-event object, so an ordinary stream closed before headers stays unmatched.
• With the default autoRetry: true, the existing run continues on the fallback model without a second user request. Set autoRetry: false to switch models without continuing.
• Interrupting a pending fallback prevents continuation. An in-flight model change is not atomic with cancellation: the selected model may change, but the extension will not request another inference after the interrupt.
• Switching providers sends the existing model context to the target provider. Preserving completed tool results prevents extension-level replay, not a model independently choosing to repeat an action.

Rules never fall back to the failing model itself; a rule whose fallback target equals the failing model is skipped so the next matching rule (or no rule) applies.

Fallback chains cascade: when the active fallback model itself fails, another matching rule may move the session further (for example free model → kimi-k2.6 → glm-5.3-flash). A cascade never revisits a model already used in the current run (the original or an earlier fallback), so circular rule configurations cannot ping-pong; when no unvisited target remains, the failure surfaces normally.

Development

npm install
npm run ci

Run locally in Pi:

pi -e .

Links

• npm: https://www.npmjs.com/package/pi-model-fallback
• GitHub: https://github.com/eiei114/pi-model-fallback
• Issues: https://github.com/eiei114/pi-model-fallback/issues
• Usage notes: docs/usage.md

License

MIT