English | 简体中文

pi-provider-gzip

A pi extension that compresses model request bodies to cut time-to-first-token (TTFT) on LLM gateways that ingest large request bodies slowly. Gzip is the default; Brotli (br) and Zstandard (zstd) are opt-in per provider.

before:  2.7 MB request body  ──►  gateway chews on it  ──►  ~55–80 s TTFT
after:   0.75 MB gzip body    ──►  gateway chews on it  ──►  ~12–22 s TTFT

• One switch turns the whole feature on or off (PI_GZIP=0).
• Works for every provider by default — no allowlists, no per-provider setup.
• Per-provider opt-out lives in the provider's own settings: "compat": { "gzip": false }.
• Per-provider encoding selects the algorithm: "compat": { "encoding": "br" }. Missing or unrecognized values fall back to gzip.

Why does this help? Many model relays (new-api, one-api, LiteLLM proxies, corporate gateways …) do work proportional to the received byte count before dispatching a request — token counting, quota pre-checks, body logging, WAF scanning. That cost is independent of the model and can dominate TTFT once your context reaches hundreds of KB. Compressing the JSON body shrinks those bytes. See docs/benchmarks.md for the measurements.

Install

# From git
pi install git:github.com/mscststs/pi-provider-gzip

# From npm (once published)
pi install npm:@mscststs/pi-provider-gzip

# From a local checkout
pi install /absolute/path/to/pi-provider-gzip

Try it without installing:

pi -e /absolute/path/to/pi-provider-gzip

After installing, start a new session (or run /reload in an existing one).

Controls

The master switch

PI_GZIP=0 pi        # temporarily disable everything

Advanced tuning (rarely needed):

PI_GZIP_DEBUG=1 pi
# [pi-provider-gzip] relay.example gzip 2708795 -> 753098 bytes (3.6x)
# [pi-provider-gzip] relay.example br   2708795 -> 502144 bytes (5.4x)

The level is shared across codecs and mapped onto each codec's native scale (0–9 → gzip level, Brotli quality 0–11, Zstandard level 1–19).

Per-provider opt-out

Providers that reject gzip request bodies can opt out in models.json:

{
  "providers": {
    "my-relay": {
      "baseUrl": "https://relay.example/v1",
      "api": "openai-completions",
      "apiKey": "...",
      "models": [{ "id": "my-model", "name": "my-model", "contextWindow": 128000 }],
      "compat": { "gzip": false }
    }
  }
}

This also works for built-in providers — no baseUrl required, since compat alone is a valid provider entry:

{
  "providers": {
    "openai": { "compat": { "gzip": false } }
  }
}

compat is the only provider-level field pi exposes to extensions, so it is where the switches live. compat: { "gzip": false } is an opt-out; everything else is enabled.

Per-provider encoding

Some relays (new-api, one-api, …) also decode request bodies compressed with Brotli or Zstandard. Those algorithms usually shrink JSON transcripts 15–35% more than gzip, so they are worth opting into when the gateway supports them:

{
  "providers": {
    "my-relay": {
      "baseUrl": "https://relay.example/v1",
      "api": "openai-completions",
      "apiKey": "...",
      "models": [{ "id": "my-model", "name": "my-model", "contextWindow": 128000 }],
      "compat": { "encoding": "br" }
    }
  }
}

Anything else — an unknown string, an empty value, a non-string, or a codec the running Node.js does not provide — falls back to gzip. compat.gzip: false still wins over compat.encoding and disables compression for that provider.

Not all relays accept Brotli or Zstandard, and an unsupported encoding is rejected with 400 Bad Request (unlike an unknown value, which the relay ignores). scripts/probe-encodings.mjs checks what a given provider accepts before you switch.

How it works

At session_start the extension reads pi's live model registry, builds a host → encoding map from every configured provider, and installs a single process-wide fetch interceptor. The interceptor compresses a request only when all of these hold:

1. the method is POST,
2. the destination host is in the allowlist (and not opted out),
3. the body is a string at or above PI_GZIP_MIN_BYTES.

It then compresses the body with the host's encoding (gzip unless compat.encoding says otherwise), sets Content-Encoding, and drops the stale Content-Length so the HTTP stack recomputes it. Everything else is forwarded untouched. The gateway decompresses transparently; the model sees the exact same JSON.

Because the hook is at the transport layer, it covers all fetch-based APIs pi supports, whether the provider is built-in or user-defined:

Bedrock uses the AWS SDK with a node:http transport and SigV4 payload signing; WebSocket transports never go through fetch. Both are intentionally out of scope.

Compatibility

• Server must accept the Content-Encoding we send on request bodies (gzip by default, or the provider's compat.encoding). Most gateways accept gzip; fewer accept Brotli or Zstandard. If a provider rejects the chosen encoding with 400, either set a different compat.encoding or disable compression with "compat": { "gzip": false }.
• Node: requires Node 22.6+ (pi bundles a compatible runtime). Zstandard needs Node 22.15+; on older runtimes a zstd request falls back to gzip automatically.

Benchmarks

Measured against an OpenAI-compatible relay serving a large reasoning model, using real pi sessions. Full methodology and raw numbers live in docs/benchmarks.md.

The relay itself returned a constant ~0.54 s for any body size once it was gzipped, versus 48.6 s for a plain 2 MB body.

Troubleshooting

Development

npm install        # dev deps: pi-coding-agent, @types/node, typescript
npm run check      # typecheck + unit tests
npm test           # node --test only

Project layout

extensions/
  index.ts               # pi extension entry point (wiring only)
lib/
  config.ts              # master switch + tuning (pure)
  encodings.ts           # gzip/br/zstd codecs + fallback (pure)
  hosts.ts               # model registry -> host/encoding map (pure)
  compress-fetch.ts      # fetch wrapper (pure)
  interceptor.ts         # idempotent global fetch install/uninstall
test/                    # node:test suites
docs/benchmarks.md       # measurement notes
scripts/probe-encodings.mjs  # probe which encodings a provider accepts

License

MIT