pi-perf

This is Philip's take on measuring time-to-first-token (TTFT) and inference rate (t/s) in Pi coding agent.

There are many such extensions, but this one is built to Philip's taste. Here it is in action:

install it

pi install npm:@pmw/pi-perf

status bar

perf prefill 982 1383 2336 ms · inf 127 177 219 t/s
         ^    ^   ^    ^         ^   ^   ^   ^ p90 inference rate
         |    |   |    |         |   |   \ live or p50 (median) inference rate
         |    |   |    |         |   \ p10 inference rate
         |    |   |    |         \ inference in progress, if highlighted
         |    |   |    |
         |    |   |    \ p90 prefill duration
         |    |    \ live or p50 (median) prefill duration
         |     \ p10 prefill duration
         \ prefill in progress, if highlighted

The status bar is the most visible part of this extension. During prefill, the live view is counting up time (ms or s). During inference, the live view is calculating tokens per second during this response. As the coding agent uses tools, the status bar toggles between prefill and inference, showing which phase we're in by a subtle highlighting.

When not active, both prefill and inference show the p10, p50, and p90 statistics for the current session. The bands are configurable; see "settings" section below.

If the prefill time climbs past the recent max, it turns red. If the inference rate drops past the recent min, it turns red.

The status bar shows metrics for the current model. So if you switch models, the numbers will change. Switching models does not lose any performance data; all data is retained and keyed by model ID.

/perf output

─── perf ────────────────────────────────────────────────────────── 49 requests

metric         mean      p50      p90      p99     min      max
───────────────────────────────────────────────────────────────
TTFT             2s       1s       2s       6s   464ms       7s
inference   182 t/s  176 t/s  224 t/s  504 t/s  41 t/s  553 t/s
generation       6s       2s      17s      32s   143ms      33s
total            8s       4s      19s      34s   666ms      35s
output tok     1.0k      411     3.0k     5.5k      18     5.9k
input tok      4.2k      520     5.0k    67.2k      76    70.2k

  by model
model                n  TTFT p50  TTFT p90  t/s p50  t/s p90
────────────────────────────────────────────────────────────
neuralwatt/glm-5.2  49        1s        2s  176 t/s  224 t/s

  last 6
time      model     TTFT      t/s   tok     stop
────────────────────────────────────────────────
23:10:41  glm-5.2     1s  183 t/s   106  toolUse
23:10:43  glm-5.2     2s  130 t/s    77  toolUse
23:10:45  glm-5.2     1s  140 t/s  2.2k     stop
23:15:22  glm-5.2     2s  214 t/s   468  toolUse
23:15:27  glm-5.2  996ms  206 t/s   976     stop
23:16:07  glm-5.2     2s  193 t/s  3.4k     stop

  Press q or Esc to close

This output shows more detailed metrics than the status bar. It shows data for all models you use during your session.

settings: /perf settings

 Perf settings

→ Status bar            on
  Status sample window  20
  Status band           p10–p90 · balanced

Status bar can be toggled on or off. If it's off, all the data is still collected; just not displayed in the status bar. /perf still shows it.

Status sample window is an integer sliding window of the last N data points that contribute to the status bar. Older data points are not used for the status bar, but still kept for /perf. A larger number gives a more stable view, at the expense of quickly reacting to a change in prefill time or inference speed.

Status band has four options:

• p25/p75 - tight
• p10/p90 - balanced (default)
• p5/p95 - wide
• p1/p99 - widest

tenets and matters of taste

This extension is calm. It uses color-coding judiciously, choosing to be understated and not draw attention to itself unnecessarily.

This extension is informative. It's the only one (to my knowledge) that shows percentiles in the status bar.

codebase details

To better understand how the code is structured, you can start with an LLM-maintained file://codebase.md .

credits

Thanks to GLM-5.2 for the coding help, and for Neuralwatt for providing models at both a standard and flex tier, so I could really exercise the prefill logic of this extension.