pi-wsearch: web tools for pi

Not the web tools you've used

WebSearch

One search, many judges. Most web tools hand you a single engine's ranking. WebSearch runs every engine you've configured in parallel and merges their results by reciprocal rank fusion (RRF). A hit only earns its place by surviving several independent judgments. The response shows the corroboration: ⇐ serper#1, exa#2, you#4 lists which indexes ranked that page and where each placed it. A hit returned by one engine alone shows just that engine and rank: ⇐ exa#3. Agreement is the spam defense; generative-SEO can stuff one index, not three.

Pick your engines. Nine are supported; three or four is plenty. Use whatever keys you have, even one:

• Rare strings match literally. pgvector, CVE numbers, error codes match as exact words, not fuzzy prose.
• Junk is downranked, not censored. Known junk domains and thin snippets lose points at scoring time.
• Same page, one entry. Tracking-param variants and near-duplicate titles collapse into one row.
• Two results per domain, max. One site's SEO army can't flood the answers.

Supported providers: Serper, Tavily, Exa, Brave, Jina, Kagi, You.com, Firecrawl, TinyFish

WebFetch

Fetch reads a map, not a dump. Most web tools hand you raw HTML and tell you to find the signal. WebFetch breaks any page into an outline of every section and answers by number: {url, sections: [0, 3]} fetches just two sections of the fetched page. The agent surveys the map, picks what matters, and reads only those: no full reads, no 40 KB dumps. Sections over 3,000 characters split into addressable continuation entries (Section (cont. 1)), so no tail is ever unreachable; overflowing pages are capped. Every cut is marked; nothing is silently lost. Codemode consumers get the same view via structuredContent: per-section {index, heading, text}. sections_truncated counts continuation cuts; sections_clipped lists local budget cuts.

Reads are free. Page parsing is built in, bills nothing per read, and needs no key. Real headings become the map, TOC echoes get dropped. Paid readers only step in for JavaScript-heavy pages or when the local read fails. Reader output is laundered: standalone photo credits (REUTERS/, AP Photo/, Getty), video durations, carousel markers and relative-time chips are stripped from news pages (whole-line rules only; prose is untouched), and a reader that returns a 404-page template is skipped so the chain moves on to the next backend.

Tracking links cache as one page. ?utm_source=… variants of the same URL share one 24 hour cache entry. Faster repeats, smaller disk.

Guarded against your localhost. A fetched page can't aim the agent at your own machine: private and loopback addresses (hex encoded, shorthand, and mapped IPv6 forms included) are rejected before any request is made.

Distrust is the default. The agent is prompted to treat fetched content as untrusted input: a claim only becomes a verdict after a second source corroborates it.

Optional readers: Firecrawl, Tavily, Exa, Jina, fallback only

Install

From npm:

pi install npm:pi-wsearch

Or from git:

pi install git:github.com/k3-2o/pi-wsearch

Either way pi picks up the package on next start and registers WebSearch, WebFetch, and the /websearch command. To try it once without installing:

pi -e npm:pi-wsearch

Keys

Search needs at least one provider key; fetch needs none. Two ways to set one:

• In your shell. Export it in the terminal session running pi, or park it in your shell config. The extension reads .zshrc, .zprofile, .bashrc, .profile, fish config, and nushell env based on your $SHELL at startup, so a key in any of those is discovered on its own.
• Via /websearch. Pick a provider from the list, paste the key. It's written to ~/.pi/agent/wsearch/env with 0600 permissions.

Either place accepts a !command value, so the key can come from Keychain or 1Password at use time and never sit on disk. Session env wins over the file, the file over shell config. /websearch shows which providers are armed and which are dormant. No keys yet? Tavily, Exa, Firecrawl, and TinyFish all have free tiers.

Commands

One command, five subcommands:

[!NOTE] Two knobs also live in wsearch/env (or your shell env, same precedence as keys). WSEARCH_ENGINES=tavily,exa makes search use only Tavily and Exa, with Tavily consulted first. WSEARCH_FETCH_CHAIN=tavily,jina makes failed reads try Tavily first, then Jina, skipping the rest.

Quick start

1. Get a key from any provider.

2. Set it one of three ways:

   /websearch login            # pick a provider, paste the key
   /websearch login tavily     # or name it directly
   export TAVILY_API_KEY=...   # or just the terminal session

3. Verify:

   /websearch

   Your provider should appear under armed:.

4. Ask the agent something current:

   What's the current Node.js LTS version?

   It will WebSearch the query, WebFetch the answer's page by section, and cite where each claim came from.

License

MIT