rlm.pi PI plugin

pi-rlm — Large contexts on cheap models: Recursive Language Model (RLM) for Pi

Install

pi install npm:@hicaru/pi-rlm        # Pi
omp plugin install @hicaru/pi-rlm    # oh-my-pi

Then /reload (or restart) — /rlm, /rlm-config, /rlm-stop appear under [Extensions]. Toggle with Ctrl+Shift+R or /rlm.

What is pi-rlm?

A Pi plugin that turns your session into a Recursive Language Model (RLM): instead of stuffing a huge document into the prompt, the context lives in a Python REPL and your best model orchestrates it — searching, decomposing, and delegating leaf reads to cheap worker models, recursively. Same Pi session, same tools, same keys: /rlm and go. Reads .pdf .docx .xlsx .epub and more, works with any OpenRouter model, 100% local.

Benchmarks

The benchmark is OOLONG (oolong-synth): 24 tasks, context up to 65,536 tokens, temperature 0.

The model list is in bench/panel.json. The results are in bench/runs/<model>/. To run the benchmark, see the root README.

How it works

          ┌─────────────────────────┐
          │     Pi coding agent     │
          └────────────┬────────────┘
                       │  /rlm
                       ▼
          ┌─────────────────────────┐  spawns   ┌────────────────────┐
          │  Smart model (root)     │ ────────► │   Worker models    │
          │  drives a Python REPL   │ ◄──────── │   (cheap, fast)    │
          └────────────┬────────────┘  results  └────────────────────┘
                       │ recursion (depth-capped)
                       └────► child RLMs ────► (same loop)

   All local · one python3 process · no servers

• Smart model thinks and writes Python in a persistent REPL.
• Worker models do the heavy lifting — read, summarize, classify, search.
• Child RLMs recurse into hard sub-problems, inheriting the full context for free.
• Live tree shows every sub-call with model, cost, tokens, and duration.

Document format support

Drop ANY of these into add_context() — they auto-convert to Markdown and land in context:

add_context("report.pdf")   # → Markdown in context
add_context("data.xlsx")    # → Markdown in context
add_context("../some-lib")  # → entire directory packed
add_context("https://github.com/x/y.git")  # → shallow clone + pack

RECURSION — the core idea

A Recursive Language Model (RLM) replaces llm.completion(prompt) with rlm.completion(prompt). The prompt becomes a variable in a REPL. The model can launch sub-LLM and sub-RLM calls as ordinary Python functions — decomposing, delegating, and synthesizing across a tree of models, not a single context window.

Each step of a run is one validated state transition (paper §3.2, §5.7):

A_t = (P, Σ_t, O_t)          fixed prompt + execution state Σ_t + tool surface
ΔΣ_t = μ(A_t)                the model emits a repl() patch, not prose
V(ΔΣ_t, Σ_t)                 deterministic validator — no crash paths
Σ_{t+1} = Σ_t ⊕ ΔΣ_t         deep merge, null = delete

Commands

Settings (/rlm-config)

Sampling & reproducibility

The r3 bench showed the biggest capability lever is not the model — it is the sampling: temperature 0 took OOLONG from 71% pooled / 40% flips to 91.7–100% all-stable for $1.74 total. Those knobs are first-class in rlm.json (~/.pi/agent/rlm.json) and on the /rlm-config panel:

Reproducibility recipe (validated by r3):

{
  "config": {
    "rootSampling": { "maxTokens": 8192, "temperature": 0, "reasoning": "high" }
  }
}

Reasoning tokens share the completion budget with the answer — with thinking on, keep maxTokens generous (the bench doubles it to 8192; the engine warns once on turn 0 when it is tight).

Scope boundary: rlm.json sampling applies to RLM-mode runs, rlm() delegation, and child recursion at any depth (same engine function). The native Pi agent loop follows Pi's own session settings — rlm.json never touches it.

Model capability: reasoning requires a model whose registry entry has reasoning: true. Anything else has the level dropped before it reaches the provider (pi-ai clamps unsupported levels to off); capability comes from the registry, so OpenRouter hybrids like qwen/qwen3.8-27b just work.

Prompt Architecture

The system prompt follows a contract / routing / examples / rules pattern (api_v5), modeled on the best-performing arm from the RLM paper bake-off:

• <contract> — every heavy call returns a Task, only await_task returns content
• <routing> — decision tree: which tool for which job
• <examples> — concrete E1–E7 patterns with anti-patterns
• <rules> — locate-then-delegate, memoize, cap workers, author edits yourself

Key insight: children see Recursion depth: N and calibrate ambition — delegating only when their task genuinely decomposes further.

Security

• Key isolation — provider keys live in TypeScript only; sandbox receives prompts, returns text.
• Environment sanitization — sensitive env vars stripped before worker spawns.
• Restricted builtins — no eval/exec/compile/input in the sandbox.
• Per-block timeout — SIGALRM + parent watchdog (SIGKILL on hang).
• Trust — project-local install requires Pi project trust.

License

MIT — see LICENSE.