pi-research-loop

pi-research-loop is a Pi Package plus standalone prl CLI for reproducible research code changes and experiments.

It provides:

• one Git branch/worktree per Agent Task;
• an immutable checkpoint commit per Run;
• a detached local runner that does not depend on Pi TUI lifetime;
• declarative process.exit, log.regex, timer.timeout, and file.created events;
• short event-driven Pi notifications instead of LLM log polling;
• Pi-session ownership so concurrent windows cannot consume or act on each other's Runs;
• optional MLflow REST tracking and W&B environment injection.

Requirements

• Node.js >= 22
• Git with git worktree
• Pi >= 0.82.1 for the Extension tools

Each server runs independently. PRL does not require a central service. MLflow and W&B are optional services for cross-machine tracking.

Installation after npm release

The CLI and Pi resources are installed separately:

# Installs the `prl` executable
npm install --global pi-research-loop@0.5.0

# Installs the Pi Extension, Skill, and Prompt
pi install npm:pi-research-loop@0.5.0

Verify:

prl --help
pi list

For upgrades, use the same versioned commands with a newer version, or:

npm install --global pi-research-loop@latest
pi update npm:pi-research-loop

Git installation

Install directly from the public repository and pin the release tag:

export PRL_REPO=https://github.com/kuan-er/pi-research-loop.git
export PRL_VERSION=v0.5.0

npm install --global "git+${PRL_REPO}#${PRL_VERSION}"
pi install "${PRL_REPO}@${PRL_VERSION}"

Local development installation

npm ci
npm run build
npm install --global .
pi install .

The complete multi-machine installation guide is in docs/INSTALL.zh-CN.md. The release procedure is in docs/RELEASING.md.

Quick start

Run PRL commands from the main worktree of a research Git repository:

cd /path/to/my-research-repo
prl init
prl doctor

prl task start --hypothesis H001 --name implementation-route
# Give the returned worktree path to the Pi Agent; reuse it across phases.

prl task checkpoint T-... --message "phase: geometry"
prl run launch --task T-... -- python train.py
prl run inspect R-...
prl run logs R-... --tail 100
prl run retry R-...
prl experiments rebuild
prl task finish T-...

prl init creates:

AGENTS.md
research/
├── PROJECT.md
├── STATE.yaml
├── DECISIONS.md
├── EXPERIMENTS.md
└── hypotheses/
.pi-research/
└── config.yaml
runs/

A Task owns one task_id, branch, and worktree for an implementation route. Repeated prl_task_start calls for an active hypothesis reuse that Task by default. A Task can have multiple phase commits and Runs. Use --new-worktree-reason stable_baseline|concurrent_agents|disposable_experiment only when a second worktree is intentional. PRL never pushes, merges, or removes a worktree automatically; review and merge the finished branch to main once at the end.

Events

Create events.yaml:

events:
  - id: cuda_oom
    type: log.regex
    pattern: "CUDA out of memory"
    source: both
    max_matches: 1
    actions: [terminate, wake_agent]

  - id: checkpoint_ready
    type: file.created
    path: outputs/best.ckpt
    actions: [record]

  - id: timeout
    type: timer.timeout
    after_seconds: 28800
    actions: [terminate, wake_agent]

Launch with an argv array; the command does not go through a shell:

prl run launch --task T-... --events events.yaml -- python train.py --seed 42

process.exit is always added automatically. Event records are persisted in run.yaml, notifications are saved under .pi-research/pending/, and Pi receives only a short summary. Pi can exit while the experiment continues.

Pi session isolation

Runs launched through the Pi Extension—or through prl from Pi's bash tool—record the current PI_SESSION_ID. Pending events, the monitor widget, prl_context.active_runs, prl_run_inspect, and prl_run_control are then scoped to that owning session.

Other Pi windows leave those pending events untouched. If the owner window is closed, its events remain on disk and are delivered/coalesced after that same Pi session is resumed. Starting /new creates a different owner; use /resume to continue an existing Run conversation and its iterative summaries.

Legacy Runs created before session ownership, or Runs launched from a normal terminal, are unbound and never auto-injected into an arbitrary Pi window. Adopt one explicitly from the intended window:

prl_run_claim {"run_id":"R-..."}

A Run already owned by another session cannot be claimed, inspected, terminated, or retried from the current window. Direct standalone CLI use outside Pi remains available for administrative inspection.

Dependent Runs and GPU handoff

For a GPU handoff, enqueue the successor before the parent finishes. The background worker then waits for the parent to succeed, validates the optional checkpoint, acquires a cooperative GPU lease, and launches without relying on an LLM wakeup:

prl run launch --task T-... --gpu 6 -- python train_long.py
prl run enqueue --task T-... --depends-on R-... \\
  --checkpoint-path /tmp/checkpoint.pt --gpu 6 --gpu-wait-seconds 86400 \\
  -- python train_next.py

The dependent Run remains queued while the parent or GPU is unavailable. nvidia-smi is checked immediately before launch, and PRL-managed Runs coordinate through .pi-research/runtime/gpu-leases/. This is a cooperative lease; unmanaged external processes can still race it. Use Slurm or another scheduler when hard reservation is required. If the parent is retried, it is marked superseded and stale dependent/wakeup actions do not launch new work.

Run inspection

prl run list
prl run inspect R-...
prl run inspect R-... --mode status
prl run inspect R-... --mode events
prl run inspect R-... --mode run_yaml
prl run inspect R-... --mode log_search --query "loss|accuracy"
prl run inspect R-... --mode git_diff
prl run inspect R-... --mode tracking
prl run logs R-... --tail 100

Each Run contains only:

runs/<run_id>/
├── run.yaml
├── console.log
└── result.md

console.log merges stdout and stderr with timestamps and source labels. result.md has an objective section generated by PRL and a preserved Agent analysis section.

Pi tools

The Extension registers:

• prl_context
• prl_task_start
• prl_run_launch
• prl_run_enqueue
• prl_run_claim
• prl_run_inspect
• prl_run_control
• prl_task_checkpoint
• prl_task_finish

A recommended Pi instruction is:

Call prl_context first; it contains only active Runs owned by this Pi session.
Start a Task with prl_task_start and modify code only in the returned worktree.
Launch experiments with prl_run_launch (or prl_run_enqueue for a dependent Run) and declare useful events.
Do not inspect, terminate, or retry Runs from another Pi session. Do not poll processes or logs.
Wait for [PRL EVENT], use prl_run_inspect in bounded mode, update the Run analysis/research state when useful, and decide whether to inspect, modify, retry, or stop.

The Skill is available at skills/prl-research-loop/SKILL.md.

MLflow and W&B

Enable MLflow in .pi-research/config.yaml:

tracking:
  mlflow:
    enabled: true
    tracking_uri_env: MLFLOW_TRACKING_URI
    experiment_name: my-project
    required: false

PRL records the PRL Run ID, Task ID, hypothesis, Git commit, branch, command, hostname, exit code, duration, and best-effort artifacts. If MLflow is unavailable and required: false, the local experiment continues.

PRL does not reimplement the W&B SDK. It injects:

PRL_RUN_ID
PRL_TASK_ID
PRL_HYPOTHESIS_ID
PRL_GIT_COMMIT
WANDB_RUN_ID
WANDB_NAME
WANDB_GROUP
WANDB_TAGS

See examples/wandb_minimal.py.

Safety

Automatic checkpoint commits refuse secrets, .env files, model weights, common credential files, experiment outputs, and files larger than the configured limit. Do not rely on this as a complete secret scanner; review diffs before committing or pushing.

Development and release

npm ci
npm run build
npm test
npm run release:check
npm pack

See docs/RELEASING.md before publishing. The GitHub workflows run tests on pull requests and can publish a GitHub Release to npm using the NPM_TOKEN secret.