📊 pi-tps

Tokens-per-second tracker for pi

Generation speed, TTFT, stall detection, and cost — after every agent turn.

---

Originally from badlogic/pi-mono. Packaged as an installable pi extension.

---

Quick start

pi install https://github.com/monotykamary/pi-tps

What's included

Extension	Tracks TPS, TTFT, stall time, token usage, and cost after each turn
Export	/tps-export command — dump telemetry as JSONL with session structure

Features

• Accurate TPS: Uses performance.now() sub-millisecond timing; excludes TTFT, tool-execution gaps, and network latency from generation speed
• Stall detection: Detects inference pauses (GPU queuing, request queuing) and subtracts them from generation TPS — no inflated rates
• Burst discrimination: Distinguishes genuine streaming from buffer-flush dispatch; shows — when the rate is structurally unidentifiable
• Multi-message turns: Aggregates tokens and timing across tool-call chains within one turn
• Notification banner: Shows a transient popup with TPS, TTFT, total time, tokens, and stalls
• Persisted notifications: Restored on session resume and /tree navigation (structured + legacy backward compatible)
• Export command: Dump telemetry as JSONL with automatic tree re-chaining for web inspectors
• Extensible: Emits tps:telemetry events so other extensions can react to telemetry

Install

pi install https://github.com/monotykamary/pi-tps

cp -r extensions/pi-tps ~/.pi/agent/extensions/

Then /reload in pi.

---

Output format

TPS 42.5 tok/s · TTFT 1.2s · 29.7s · in 567 · out 1.2K · stall 4.3s×1

Field	Description
TPS	Tokens per second (generation speed, excludes TTFT & stalls)
TTFT	Time to first token (seconds, 1 decimal)
s	Total wall-clock time from request to completion
in	Input tokens (human-readable: K/M/B)
out	Output tokens (human-readable: K/M/B)
stall	Accumulated stall time × stall count (shown only when stalls exist)

When TPS can't be determined (burst delivery, too few chunks), the field shows —:

TPS — · TTFT 0.8s · 1.3s · in 291 · out 46

Human-readable scaling (for token counts):

• < 1K: raw integer (567)
• ≥ 1K: one decimal, drops .0 (1.2K, 2K, 15.3K)
• ≥ 1M: same pattern (1.5M)
• ≥ 1B: same pattern (1.2B)

Duration formatting:

• < 60s: one decimal (2.3s, 45.0s)
• ≥ 60s: up to two units with no decimals (1m 30s, 2h 15m, 3d 12h, 1w 3d, 1mo 0d, 1y 0d)

---

How it works

The extension hooks into pi's lifecycle events. The critical detail: message_start fires at stream creation (before any tokens), so TTFT is measured at the first message_update, which carries the first real token content.

Event sequence

turn_start         →  request sent to LLM, timer starts
message_start      →  stream created, stall-tracking reset for this message
message_update (1) →  first token arrives → TTFT captured
message_update (N) →  streaming tokens arrive → inter-update span & stall detection
message_end        →  message complete, generation time accumulated
turn_end           →  telemetry computed and displayed

Timing breakdown

Phase	Measured by
TTFT	turn_start → first message_update
Generation	per-message wall clock (message_start → message_end), summed across messages in the turn
Stream span	first message_update (post-TTFT) → last message_update — the pure streaming window
Total	turn_start → last message_end in the turn

This approach excludes:

• Network latency (included in TTFT)
• Tool-execution gaps between messages (stall clock resets on each message_start)
• Server queue time (included in TTFT)

Stall detection

Every message_update (after TTFT) measures the gap since the last update. Gaps ≥ 500ms are classified as inference stalls:

• The full gap is accumulated as stallMs
• Consecutive stalled updates count as one stall event
• Stalls are subtracted from the streaming window when computing generation TPS
• The stall clock resets at each message_start, so tool-execution gaps between messages are never counted as stalls

When a stall occurs before the first stream update (common in request-queuing scenarios), the TPS algorithm detects the artifact and falls back to a conservative estimate rather than producing an inflated rate.

TPS algorithm (three-branch gate)

The extension uses a defense-in-depth strategy to produce reliable TPS:

1. Primary — Requires ≥5 streaming updates with ≥1ms average inter-chunk gap and stall time < active generation time. Subtracts stalls from the streaming window for pure generation speed.

2. Fallback — When primary conditions fail but ≥2 updates exist and total generation time ≥50ms. Uses the full generation window (includes TTFT, so it underestimates — safe by design). Applies partial stall reduction when stalls dominate.

3. Null — Returns null (displayed as —) when the timing is structurally unidentifiable: burst delivery (all tokens arrive in the same tick), too few chunks, or generation time too short for a reliable rate.

---

Rehydration

When you resume a session (or navigate branches with /tree), pi-tps restores the most recent TPS notification — so you can see your last turn's stats after a reload.

Supports both the current structured TurnTelemetry format and legacy { message, timestamp } entries for backward compatibility with session files created by earlier versions.

---

Export command

Dump telemetry as JSONL for inspection or analysis:

/tps-export             # current branch, all custom entries
/tps-export --full      # all branches in the session
/tps-export tps         # current branch, filter by customType "tps"
/tps-export tps --full  # all branches, filter by customType "tps"

Each exported file is written to ~/.cache/pi-telemetry/pi-telemetry-{scope}-{sessionId}-{timestamp}.jsonl.

The exporter includes structural entries (model_change, branch_summary) alongside telemetry entries so the exported tree is fully resolvable — the web inspector can show model switches and branch points. Parent IDs are automatically re-chained to point to the nearest ancestor that's included in the export, producing a self-contained tree.

---

Telemetry event

After each turn, pi-tps emits a tps:telemetry event on pi's shared event bus. Other extensions can listen to build custom widgets, dashboards, or cost trackers.

pi.events.on('tps:telemetry', (data) => {
  // data matches the TurnTelemetry structure below
  console.log(data.tps, data.tokens, data.timing);
});

The event payload:

Field	Type	Description
tps	number | null	Tokens per second, or null when unidentifiable
model.provider	string	Provider name (e.g. openai)
model.modelId	string	Model identifier (e.g. gpt-4)
tokens.input	number	Input tokens (summed across all assistant messages)
tokens.output	number	Output tokens generated by the LLM
tokens.cacheRead	number	Cache-read tokens (provider-dependent)
tokens.cacheWrite	number	Cache-write tokens (provider-dependent)
tokens.total	number	Total tokens (input + output + cache)
timing.ttftMs	number | null	Time to first token in milliseconds
timing.totalMs	number	Total wall-clock time from request to completion
timing.generationMs	number	Streaming wall clock (message_start → message_end)
timing.streamMs	number | null	Inter-update span: first → last streaming update
timing.stallMs	number	Accumulated inference stall time in ms
timing.stallCount	number	Number of discrete stall events
timing.messageCount	number	Assistant messages in this turn
cost.input	number | null	Input token cost
cost.output	number | null	Output token cost
cost.cacheRead	number | null	Cache-read token cost
cost.cacheWrite	number | null	Cache-write token cost
cost.total	number | null	Total cost for this turn
timestamp	number	Unix timestamp (ms) when telemetry was computed

When cost is unavailable (provider doesn't report it), the entire cost object is null.

---

Testing

# Install dependencies
npm install

# Run tests
npm test

# Run tests with coverage
npm run test:coverage

# Type check
npm run typecheck

---

License

MIT