Toji

Toji is a Pi extension workspace with three packages:

• toji-mem: local code memory, search, graph traversal, and benchmarks
• toji-kompak: session compaction snapshots and recovery
• toji-agents: specialized, inspectable child agents forked from the current Pi session

toji-kompak augments Pi's default session compaction, whose behavior may change between Pi releases.

What toji-mem does

toji-mem indexes a repository into SQLite and answers common agent questions:

• where is this symbol?
• what file should I read?
• what related files/symbols are nearby?
• what might be affected if I change this symbol?

It uses:

• SQLite FTS5 for fuzzy search over files, symbols, docstrings, and paths
• B-tree indexes for exact file and symbol lookups
• Tree-sitter parsers for symbols, imports, and calls
• graph edges for blast-radius and related-file traversal

What toji-agents does

toji-agents lets the active Pi model delegate work to five fixed specialists. Every invocation creates a real child session from the current parent leaf, runs a separate Pi process, streams its activity in the parent tool row, and returns the child's final response.

• Doran validates backend code, tests, and edge cases.
• Oner scouts and summarizes codebases.
• Faker creates strict technical plans.
• Peyz handles frontend design and implementation.
• Keria designs cloud architecture from requirements and costs.

Full child transcripts remain available through the session path shown in the expanded tool result.

Install from Pi

After publishing to npm, install Toji like this:

pi install npm:@mr-jones123/toji

Try it for one Pi run without installing:

pi -e npm:@mr-jones123/toji

Local setup

npm install

Use with Pi

If installed with pi install, start Pi normally and all three extensions are discovered.

For local development, start Pi with one extension loaded:

pi -e ./packages/toji-mem/src/index.ts
pi -e ./packages/toji-agents/src/index.ts

toji-mem commands

/toji-index .
/toji-query indexProject
/toji-overview .
/toji-graph index project
/toji-blast indexProject
/toji-bench .

command	purpose
/toji-index [path]	index a project into Toji memory
/toji-query <query>	search indexed files, symbols, and standards
/toji-overview [path]	show compact project overview, expandable in Pi
/toji-graph <intent>	find related files/symbols from a natural-language intent
/toji-blast <symbol>	traverse likely impact radius for a symbol
/toji-bench [path]	run the operational benchmark from inside Pi

toji-agents

Ask naturally and the parent model calls toji_agent:

Ask Oner to scout the authentication flow.
Work with Faker on a plan for multi-tenant billing.
Have Doran validate the backend changes.
Ask Peyz to redesign this dashboard.
Have Keria compare AWS and Cloudflare architectures.

The tool accepts an agent name and concrete task. Child output is capped at Pi's 50KB tool limit; the full child session remains on disk.

Reproduce the operational benchmark

The benchmark measures Toji core directly, without LLM/session overhead:

• cold index
• hot re-index
• query p50
• project overview p50
• blast-radius p50
• indexed files/symbols/edges

1. Benchmark Toji itself

cd packages/toji-mem
npm run bench -- --repo . --json

2. Benchmark Flask

From the repository root:

mkdir -p benchmarks/repos
git clone --depth 1 https://github.com/pallets/flask.git benchmarks/repos/flask
cd packages/toji-mem
npm run bench -- --repo ../../benchmarks/repos/flask --json

If Flask is already cloned:

git -C benchmarks/repos/flask pull --ff-only
cd packages/toji-mem
npm run bench -- --repo ../../benchmarks/repos/flask --json

Current smoke scores

repo	cold index	hot index	query p50	overview p50	blast p50	files	symbols	edges
packages/toji-mem	679.02 ms	5.18 ms	0.68 ms	0.75 ms	1.71 ms	20	98	647
flask	1888.94 ms	12.85 ms	0.71 ms	9.67 ms	0.29 ms	83	1620	7658

RepoBench retrieval scores

Toji also has a local RepoBench Python v1.1 adapter for cross-file retrieval quality. It materializes synthetic repos grouped by repo_name, indexes them, then queries with the gold identifier plus repo/current-file context.

split	rows	files	Path Hit@1	Path Hit@5	Path MRR	Symbol Hit@1	Symbol Hit@5	Symbol MRR	p95 latency
cross_file_first	8,026	17,081	0.576	0.600	0.587	0.540	0.551	0.545	0.784 ms
cross_file_random	7,610	16,750	0.513	0.544	0.526	0.467	0.478	0.471	1.014 ms

Compared to the original name-only baseline, repo/import-aware reranking improves Path Hit@1 by about 13 points on both splits.

Benchmark notes

• CLI benchmark is the source of truth for stable operational numbers.
• /toji-bench uses the same benchmark engine from inside Pi.
• Cloned benchmark repos are scratch data and are not committed.
• Full RepoBench tables live in packages/toji-mem/README.md.