🔥 pi-firecrawl — Firecrawl Web Scraping Tools for Pi Agents

@narumitw/pi-firecrawl is a native Pi coding agent extension that exposes Firecrawl scraping, crawling, URL discovery, and search APIs as Pi tools.

Use it to give your AI coding agent reliable web research capabilities for documentation lookup, website audits, competitive research, content extraction, and retrieval-friendly markdown scraping.

✨ Features

• Scrape a single URL into markdown, HTML, raw HTML, links, screenshots, or JSON.
• Start Firecrawl crawl jobs from Pi.
• Check crawl job status and retrieve completed crawl data.
• Discover URLs with Firecrawl map.
• Search the web and optionally scrape search result pages.
• Supports Firecrawl API endpoint overrides.
• Shows statusline activity only while Firecrawl tools are running.
• Provides a /firecrawl menu with configuration help and tool controls.
• Uses @narumitw/pi-tui-kit for width-safe menus and individual tool selection.
• Persists the selected Firecrawl tools across Pi restarts.
• Bounds tool output to Pi's 50 KB or 2,000-line limit while preserving oversized responses in private temporary files.
• Never logs, displays, or stores your Firecrawl API key.

📦 Install

pi install npm:@narumitw/pi-firecrawl

Try without installing permanently:

FIRECRAWL_API_KEY=fc-... pi -e npm:@narumitw/pi-firecrawl

Try this package locally from the repository root:

FIRECRAWL_API_KEY=fc-... pi -e ./extensions/pi-firecrawl

⚙️ Configuration

Set a Firecrawl API key before running Pi:

export FIRECRAWL_API_KEY=fc-your-key

Optional API endpoint override:

export FIRECRAWL_API_URL=https://api.firecrawl.dev/v1

FIRECRAWL_BASE_URL is also accepted for compatibility. The extension never logs or displays the API key.

🛠️ Pi tools

• firecrawl_scrape — scrape a single URL and return requested formats such as markdown, HTML, links, screenshots, or JSON.
• firecrawl_crawl — start a site crawl job and return the Firecrawl job id.
• firecrawl_crawl_status — check a crawl job status and retrieve completed crawl data.
• firecrawl_map — discover URLs for a site.
• firecrawl_search — search the web through Firecrawl and optionally scrape result pages.

All tools fail with a clear configuration error when FIRECRAWL_API_KEY is missing.

Tool output is limited to 50 KB or 2,000 lines, whichever is reached first. When a response is truncated, the result reports the original and displayed sizes and the path to a complete temporary JSON file. These files use private permissions, remain available for the current session, and are removed during session shutdown or reload. Tool-result metadata contains only size and artifact information rather than a duplicate of the raw Firecrawl response. Oversized Firecrawl error bodies are bounded in the same way.

💬 Command

/firecrawl

Opens a menu with configuration quick start, command usage, tool status, controls for enabling or disabling all Firecrawl tools, and a selector for choosing individual tools.

Direct subcommands are also available:

/firecrawl help
/firecrawl config
/firecrawl quickstart
/firecrawl status
/firecrawl tools
/firecrawl toggle
/firecrawl enable
/firecrawl disable

• help shows command usage.
• config shows API-key presence and API URL without displaying the API key value.
• quickstart is an alias for config.
• status shows runtime tool state, persisted selection, settings file path, API-key presence, API URL, and active non-Firecrawl tool count.
• tools opens a width-safe selector for choosing individual firecrawl_* tools.
• toggle is an alias for tools.
• enable enables all firecrawl_* tools for future turns.
• disable disables all firecrawl_* tools for future turns. The slash command remains available.

The selected tool names are saved to:

${PI_CODING_AGENT_DIR:-~/.pi/agent}/pi-firecrawl.json

When the file is missing or invalid, the extension preserves Pi's current active-tool policy instead of enabling tools by itself. A valid saved selection is restored on Pi startup and /reload. A missing file is created by the first successful selection change. Within one Pi process, selection saves run in invocation order, reread the latest valid document, and preserve unknown fields. Malformed JSON or invalid recognized fields block the save without replacement; a failed save restores the prior Firecrawl tool selection while preserving other extensions' current tools. The settings file stores only tool names and a timestamp; it never stores FIRECRAWL_API_KEY, request headers, or other secrets.

Compatibility: older versions used pi-firecrawl-settings.json. A legacy-only file remains readable with a warning and is never modified automatically; rename it to pi-firecrawl.json. The first subsequent settings save writes the canonical file. If both files exist, pi-firecrawl.json wins and the legacy file is ignored. The legacy filename is deprecated and will be removed in a future major release.

🚀 Examples

Scrape a page as markdown:

{
  "url": "https://example.com",
  "formats": ["markdown"]
}

Map a small site:

{
  "url": "https://example.com",
  "limit": 20
}

Start a crawl with markdown extraction:

{
  "url": "https://example.com",
  "limit": 10,
  "scrapeOptions": {
    "formats": ["markdown"]
  }
}

🧠 Use cases

• Research documentation from inside Pi.
• Crawl websites for migration or audit tasks.
• Extract clean markdown for AI context.
• Discover URLs before scraping a site.
• Combine web search with coding-agent implementation work.

🗂️ Package layout

extensions/pi-firecrawl/
├── src/
│   ├── index.ts      # Pi package entrypoint
│   ├── firecrawl.ts  # Extension registration and command orchestration
│   └── *.ts          # Package-local client, settings, selector, and tool modules
├── README.md
├── LICENSE
├── tsconfig.json
└── package.json

index.ts is the Pi entrypoint and forwards to firecrawl.ts; the other source modules are internal.

🔎 Keywords

Pi extension, Pi coding agent, Firecrawl, web scraping, web crawling, URL discovery, web search, markdown extraction, AI research agent, TypeScript Pi tools.

📄 License

MIT. See LICENSE.