fff-subconscious-pi

A background memory layer for the pi coding agent. Before each LLM call it greps your local text corpora with fff, a fast local search engine, makes one cheap deepseek-v4-flash call to synthesize the hits into a short note, and injects that note into the prompt as a <fff_subconscious> block. The agent gets a reminder of what you worked on before, without being asked.

Install

pi install npm:fff-subconscious-pi

or straight from the repo:

pi install git:github.com/esinecan/fff-subconscious-pi

Requirements: python3 >= 3.10 on PATH and DEEPSEEK_API_KEY in the environment. On the first few context events after install, the extension bootstraps its own python venv in the background (.venv/ inside the package directory, with fff-search and openai, takes about a minute) and injects nothing until the .venv/.ready sentinel appears. From then on it is live. Unix-only for now, since the pipeline script uses fcntl and /tmp.

Point it at your own corpora

config.json in the package directory decides what gets searched:

• fff.transcripts_root defaults to ~/.claude/projects, which is where the Claude Code CLI keeps past session transcripts. If you use Claude Code, this works as is. If not, point it at any directory of text you want recalled.
• fff.canon_roots ships pointing at the author's private notes directories, which do not exist on your machine. Replace the entries with your own documentation or notes directories, or delete them.

A configured root that does not exist is named in /tmp/subconscious.log on every run and skipped, so a fresh install runs on whatever corpora it can find rather than failing.

What happens per prompt

pi.on("context", handler) fires before every LLM call with the full message history. The handler:

1. Guards: venv ready, mode, minimum messages, minimum input length, dedup.
2. Serializes the conversation into a temp transcript file.
3. Runs the vendored observe.py with the package venv's python. That script extracts search terms from the recent conversation, greps every configured corpus with fff, and makes one deepseek-v4-flash call to turn the hits into a short note, or NO_UPDATE when they are not worth injecting. The model may also escalate once, asking for a deep read of one promising file or a widened search, at the cost of one more flash call.
4. Reads the result from cache/ and injects it into the last user message as <fff_subconscious>...</fff_subconscious>.

A typical run takes about six seconds. Tool-call continuation turns re-inject the cached note without re-running the pipeline, and different user messages are spaced out by a cooldown. Any failure injects nothing.

Design notes

The pipeline script comes from a Claude Code hook setup by the same author and is vendored verbatim, so this package is self-contained. Claude Code runs it across two hooks: generation happens after each assistant reply, injection happens on the next prompt, and the injected note therefore always trails the conversation by one turn. Pi has no such split, only a single context event before each LLM call, so this extension runs the pipeline synchronously and injects a note built from the very prompt being answered. No lag, and a hard subprocess timeout (FFF_SUBCONSCIOUS_TIMEOUT_MS, default 60s) guarantees the LLM call never blocks on it.

One consequence of firing before the LLM call: on the first turn of a fresh session the transcript holds only the user's message, so the vendored config sets min_messages to 1. The note can then arrive on the opening question instead of waiting for turn two.

Configuration (env vars)

Variable	Default	Purpose
FFF_SUBCONSCIOUS_MODE	every	every, first, or off.
FFF_SUBCONSCIOUS_DEBUG	0	1 for stderr logs prefixed [fff-sub].
FFF_SUBCONSCIOUS_COOLDOWN_MS	120000	Min gap between pipeline runs across user messages.
FFF_SUBCONSCIOUS_MIN_MESSAGES	1	Skip if fewer messages.
FFF_SUBCONSCIOUS_MIN_CHARS	12	Skip injection for inputs shorter than this.
FFF_SUBCONSCIOUS_MAX_CHARS	4000	Hard cap on injected note length.
FFF_SUBCONSCIOUS_TIMEOUT_MS	60000	Hard cap on the pipeline subprocess.
DEEPSEEK_API_KEY	(required)	DeepSeek key for the flash calls.

Tuning of the retrieval itself (corpora roots, max search terms, escalation budgets) lives in config.json.

Cache and logs

Notes are cached in cache/ inside the package directory, one file per working directory, keyed by sha256(cwd). Temp transcripts are cleaned up after each run. Delete the cache directory to force fresh queries. The pipeline logs to /tmp/subconscious.log, with a structured version in /tmp/subconscious-diag.log.