pi-bulk-read

Read many files — or one long file — in one budgeted call for pi. Ranges, globs, real line numbers, and an outline for whatever does not fit.

read stops at 2000 lines / 50KB and returns no line numbers, so a 3000-line file becomes three turns of paging with offset/limit, and every turn re-processes the whole context. Measured over 306 sessions: continued reads (the same file opened again with a larger offset) are 5.4% of reads under 8k, but 38.8% above 32k. The problem is not the file; it is the access pattern.

What it does

Line numbers are always printed: the only stable address for a file:A-B range and for edit. Runs of blank lines collapse and trailing whitespace is trimmed; printed numbers stay real, so any elision remains visible.

Install

pi install npm:pi-bulk-read

Or add "npm:pi-bulk-read" to the packages list in ~/.pi/agent/settings.json. The tool loads in every session; /reload picks it up without restarting.

Usage

bulk_read paths=["src/app.ts", "src/queue.py:120-260", "docs/**/*.md"]

Big file, first look — an outline with line spans (real output):

[bulk_read] 1 file(s) · mode=outline · budget 24000 chars
── extensions/bulk-read.ts · outline/symbols · 465 lines · 17.4KB ──
 1  // @ts-nocheck
 2  /**
 3   * bulk-read — read long files and many files in ONE budgeted call.
 ...
 ⋮
    43-58  export interface Unit {
    59-64  export function parseSpec(spec: string): { path: string; range?: [number, number?] } {
    65-69  function hasGlob(p: string): boolean {
   70-110  export function expandSpecs(specs: string[], cwd: string): { abs: string; range?: Unit["range"] }[] {
  111-119  function exists(p: string): boolean {
  120-125  function splitLines(text: string): string[] {
[full text: "extensions/bulk-read.ts:A-B" · e.g. "extensions/bulk-read.ts:43-58"]

Then the range you actually need:

bulk_read paths=["extensions/bulk-read.ts:43-58"]

Parameters

Environment

What it does not do

• No search mode. grep already does that, and a second implementation would drift. This tool only batches reads.
• The outline is heuristic (regex-based structure detection), not a language server. When no structure is found — plain text, logs — you get head + tail and are told to ask for a range.
• No silent loss. Missing, binary, empty, oversized (>5MB) and directory paths are reported as lines in the result, never dropped.

Tests

No model calls, no network — fixtures are built in a temp directory:

bun run test/test-bulk-read.ts

Covers range/glob parsing, outline spans for markdown/code/yaml, control flow being excluded from the symbol map, budget behaviour and continuation calls, error reporting, and the real execute path of the registered tool.

Releasing

ci.yml runs the test suite on every push and pull request. publish.yml publishes from GitHub with npm provenance (OIDC, no local OTP):

# bump "version" in package.json, commit, then
git tag v0.1.0 && git push origin v0.1.0

The workflow refuses a tag that does not match package.json, verifies the tests, skips a version that is already on npm, and publishes with --provenance (pre-release versions go to the next dist-tag). gh workflow run publish.yml --ref main publishes the current version without a tag.

The first publish of a new package name is still done locally (npm publish --access public), because npm trusted publishing can only be configured for a package that already exists. After that, add the repository as a trusted publisher in the package settings on npmjs.com.

License

MIT