pi-research-loop

pi-research-loop is a Pi Package plus standalone prl CLI for reproducible research code changes and experiments.

It provides:

• one Git branch/worktree per Agent Task;
• an immutable checkpoint commit per Run;
• a detached local runner that does not depend on Pi TUI lifetime;
• declarative process.exit, log.regex, timer.timeout, and file.created events;
• short event-driven Pi notifications instead of LLM log polling;
• optional MLflow REST tracking and W&B environment injection.

Requirements

• Node.js >= 22
• Git with git worktree
• Pi >= 0.82.1 for the Extension tools

Each server runs independently. PRL does not require a central service. MLflow and W&B are optional services for cross-machine tracking.

Installation after npm release

The CLI and Pi resources are installed separately:

# Installs the `prl` executable
npm install --global pi-research-loop@0.4.0

# Installs the Pi Extension, Skill, and Prompt
pi install npm:pi-research-loop@0.4.0

Verify:

prl --help
pi list

For upgrades, use the same versioned commands with a newer version, or:

npm install --global pi-research-loop@latest
pi update npm:pi-research-loop

Git installation

Install directly from the public repository and pin the release tag:

export PRL_REPO=https://github.com/kuan-er/pi-research-loop.git
export PRL_VERSION=v0.4.0

npm install --global "git+${PRL_REPO}#${PRL_VERSION}"
pi install "${PRL_REPO}@${PRL_VERSION}"

Local development installation

npm ci
npm run build
npm install --global .
pi install .

The complete multi-machine installation guide is in docs/INSTALL.zh-CN.md. The release procedure is in docs/RELEASING.md.

Quick start

Run PRL commands from the main worktree of a research Git repository:

cd /path/to/my-research-repo
prl init
prl doctor

prl task start --hypothesis H001 --name implementation-route
# Give the returned worktree path to the Pi Agent; reuse it across phases.

prl task checkpoint T-... --message "phase: geometry"
prl run launch --task T-... -- python train.py
prl run inspect R-...
prl run logs R-... --tail 100
prl run retry R-...
prl experiments rebuild
prl task finish T-...

prl init creates:

AGENTS.md
research/
├── PROJECT.md
├── STATE.yaml
├── DECISIONS.md
├── EXPERIMENTS.md
└── hypotheses/
.pi-research/
└── config.yaml
runs/

A Task owns one task_id, branch, and worktree for an implementation route. Repeated prl_task_start calls for an active hypothesis reuse that Task by default. A Task can have multiple phase commits and Runs. Use --new-worktree-reason stable_baseline|concurrent_agents|disposable_experiment only when a second worktree is intentional. PRL never pushes, merges, or removes a worktree automatically; review and merge the finished branch to main once at the end.

Events

Create events.yaml:

events:
  - id: cuda_oom
    type: log.regex
    pattern: "CUDA out of memory"
    source: both
    max_matches: 1
    actions: [terminate, wake_agent]

  - id: checkpoint_ready
    type: file.created
    path: outputs/best.ckpt
    actions: [record]

  - id: timeout
    type: timer.timeout
    after_seconds: 28800
    actions: [terminate, wake_agent]

Launch with an argv array; the command does not go through a shell:

prl run launch --task T-... --events events.yaml -- python train.py --seed 42

process.exit is always added automatically. Event records are persisted in run.yaml, notifications are saved under .pi-research/pending/, and Pi receives only a short summary. Pi can exit while the experiment continues.

Dependent Runs and GPU handoff

For a GPU handoff, enqueue the successor before the parent finishes. The background worker then waits for the parent to succeed, validates the optional checkpoint, acquires a cooperative GPU lease, and launches without relying on an LLM wakeup:

prl run launch --task T-... --gpu 6 -- python train_long.py
prl run enqueue --task T-... --depends-on R-... \\
  --checkpoint-path /tmp/checkpoint.pt --gpu 6 --gpu-wait-seconds 86400 \\
  -- python train_next.py

The dependent Run remains queued while the parent or GPU is unavailable. nvidia-smi is checked immediately before launch, and PRL-managed Runs coordinate through .pi-research/runtime/gpu-leases/. This is a cooperative lease; unmanaged external processes can still race it. Use Slurm or another scheduler when hard reservation is required. If the parent is retried, it is marked superseded and stale dependent/wakeup actions do not launch new work.

Run inspection

prl run list
prl run inspect R-...
prl run inspect R-... --mode status
prl run inspect R-... --mode events
prl run inspect R-... --mode run_yaml
prl run inspect R-... --mode log_search --query "loss|accuracy"
prl run inspect R-... --mode git_diff
prl run inspect R-... --mode tracking
prl run logs R-... --tail 100

Each Run contains only:

runs/<run_id>/
├── run.yaml
├── console.log
└── result.md

console.log merges stdout and stderr with timestamps and source labels. result.md has an objective section generated by PRL and a preserved Agent analysis section.

Pi tools

The Extension registers:

• prl_context
• prl_task_start
• prl_run_launch
• prl_run_inspect
• prl_run_control
• prl_task_checkpoint
• prl_task_finish

A recommended Pi instruction is:

Call prl_context first. Start a Task with prl_task_start.
Modify code only in the returned worktree.
Launch experiments with prl_run_launch and declare useful events.
Do not poll processes or logs. Wait for [PRL EVENT], then use prl_run_inspect
and decide whether to inspect, modify, retry, or stop.

The Skill is available at skills/prl-research-loop/SKILL.md.

MLflow and W&B

Enable MLflow in .pi-research/config.yaml:

tracking:
  mlflow:
    enabled: true
    tracking_uri_env: MLFLOW_TRACKING_URI
    experiment_name: my-project
    required: false

PRL records the PRL Run ID, Task ID, hypothesis, Git commit, branch, command, hostname, exit code, duration, and best-effort artifacts. If MLflow is unavailable and required: false, the local experiment continues.

PRL does not reimplement the W&B SDK. It injects:

PRL_RUN_ID
PRL_TASK_ID
PRL_HYPOTHESIS_ID
PRL_GIT_COMMIT
WANDB_RUN_ID
WANDB_NAME
WANDB_GROUP
WANDB_TAGS

See examples/wandb_minimal.py.

Safety

Automatic checkpoint commits refuse secrets, .env files, model weights, common credential files, experiment outputs, and files larger than the configured limit. Do not rely on this as a complete secret scanner; review diffs before committing or pushing.

Development and release

npm ci
npm run build
npm test
npm run release:check
npm pack

See docs/RELEASING.md before publishing. The GitHub workflows run tests on pull requests and can publish a GitHub Release to npm using the NPM_TOKEN secret.