pi-prompt-tracker

Track complete prompts sent to the LLM gateway — system prompt, tools, messages, responses, and usage — for debugging, caching analysis, and cost optimization.

Install

pi install git:github.com/<your-username>/pi-prompt-tracker

Or from a local path:

pi install /path/to/pi-prompt-tracker

Or try it once without installing:

pi -e /path/to/pi-prompt-tracker

What It Logs

For every LLM request, writes a structured Markdown log file containing:

Data	Detail
System prompt	Full system prompt text sent to the model
Tool definitions	Every tool's name, description, and JSON schema
All messages	User, assistant (text + tool calls + thinking), tool results
Full provider payload	Raw JSON of the complete request sent to the API
Response status	HTTP status code and response headers
Assistant response	Final output with token usage and cost

Log Files

Written to ~/.pi/agent/prompt-logs/ by default (configurable in source).

~/.pi/agent/prompt-logs/
├── prompt-log_2026-07-02--20-31-12_session-c0983527.md
├── prompt-log_2026-07-03--10-15-00_session-a1b2c3d4.md
└── ...

Each session gets one log file. Entries are appended as requests/responses happen.

Commands

Command	Description
/prompt-tracker-log	Open the latest log file in your editor
/prompt-tracker-dir	Show the log directory path and file count
/prompt-tracker-clear	Clear all logs (with confirmation)

Use Cases

• Debug what's actually sent — verify system prompt, tools, and message ordering
• Caching analysis — see which parts of the prompt are static vs dynamic to optimize prompt caching (Anthropic/OpenAI)
• Cost tracking — monitor token usage and cost per request
• Prompt engineering — inspect the final payload to refine system prompts and tool descriptions
• Provider comparison — log the exact payload format for different providers

Configuration

Edit the constants at the top of extensions/prompt-tracker.ts:

const LOG_DIR = join(homedir(), ".pi", "agent", "prompt-logs"); // Change log directory
const SHOW_NOTIFICATIONS = true;                                   // Toggle startup notification

Example Snippet

## Request #3 — Jul 02, 2026, 20:35:12

**Model:** deepseek-v4-flash-free

### System Prompt
\`\`\`
You are an expert coding assistant operating inside pi...
\`\`\`

### Tool Definitions
### 1. read
- **Description:** Read the contents of a file...
- **Input schema:** {...}

### Messages (42 total)
### user
can we create pi extension...

### assistant
Let me read the docs...

### Full Provider Payload
\`\`\`json
{ "model": "...", "messages": [...], "tools": [...] }
\`\`\`

Disclaimer

This extension was generated by Pi Coding Agent through vibe coding. It logs every prompt sent to the LLM, which may include sensitive data. Use with caution and review the code before deploying in production.

Requirements

• Pi coding agent
• No external npm dependencies (uses only pi's built-in packages and Node.js built-ins)

License

MIT