[!IMPORTANT] This extension is maintained by Wyna Informatik GmbH. The Wyna search API (searchProvider: "wyna") uses the same API key as your Wyna LLM gateway — no separate credential needed. See Quick start below.

pi-web-research

Token-saving web research for the pi coding agent. Not a bare web-search tool: an advanced pipeline — search (Kagi or Wyna) → parallel SSRF-guarded fetch → Readability → Markdown → isolated sub-agents — distills pages into cited briefings and targeted answers. Raw pages never enter your context; you spend tokens on findings, not noise.

What you get

Two tools, backed by specialized, isolated sub-agents that keep raw pages out of the main context:

• web_search — search results; in comprehensive mode a sub-agent reads the top N pages and returns one cited briefing.
• web_fetch — one page distilled into a summary (concise by default, or thorough/full) or, with a prompt, a targeted answer to just that question.

All research output is cited — every web-sourced claim carries its source as a visible, clickable URL with the publish date — and defended against content farms and fake news: no publisher dominates a search, same-publisher pages never count as corroboration, and uncorroborated single-source "events" are quarantined as unverified instead of reported as fact.

Privacy: only your chosen search provider sees your queries; pages are fetched and extracted locally and nothing is sent to any other third party. Fetched text is treated as untrusted, and all outbound fetches are SSRF-guarded. Bot-protected pages (Reddit, Cloudflare-challenged sites) can be handled by an opt-in, locally-run headless-browser fallback or a self-hosted reader — no third-party reader by default.

Quick start

pi install npm:pi-web-research

That's it — pi installs the package and its dependencies. Other methods:

• git: pi install git:github.com/wynainfo/pi-web-research
• local clone (development): pi install /path/to/pi-web-research
• manual: clone into ~/.pi/agent/extensions/ (auto-discovered) and run npm install there yourself

Then provide one search API key — either:

• Kagi: export KAGI_API_KEY="<key>" (key from kagi.com/api/keys), or
• Wyna: nothing to do if providers.wyna.apiKey is already in ~/.pi/agent/models.json — the LLM-gateway key doubles as the search key. Otherwise export WYNA_API_KEY="<key>".

Start pi and run /web-research-status to verify. If anything is missing, pi prints a single ⚠ web-research notice — just ask pi to finish the setup; it is briefed on exactly what's missing and where to store the key.

Optional: headless-browser fallback (Playwright)

Fetching bot-protected pages needs this optional dependency; everything else works without it. Install it up front — or simply ask pi to do it when a page comes back blocked:

cd ~/.pi/agent/git/github.com/wynainfo/pi-web-research
npm install playwright          # the Node client (also installed via optionalDependencies)
npx playwright install chromium # the Chromium browser binary (~once, ~150 MB)

The fallback activates automatically once installed (/web-research-status shows the state) and runs Chromium sandboxed in a separate process — details under Notes & limitations.

Choosing a provider

Provider	searchProvider value	Auth	Key source
Kagi v1 (default)	"kagi" / "kagi-v1"	Bearer token	KAGI_API_KEY env / kagiToken config
Kagi v0 (legacy/deprecated)	"kagi-v0" / "kagi-legacy"	Bot token	KAGI_API_KEY_V0 env / kagiV0Token config
Wyna	"wyna"	Wyna API key	WYNA_API_KEY env / Pi models.json (providers.wyna.apiKey) / wynaApiKey config

The default is Kagi v1; with only a Wyna key present, Wyna is auto-detected. To force a provider, set searchProvider in the global config.json (a security-sensitive, global-only key — see Configuration).

Disabling

• Per run: pi --no-extensions (disables all extensions).
• Uninstall: pi remove git:github.com/wynainfo/pi-web-research (or toggle it in pi config).

Development

Maintained at https://github.com/wynainfo/pi-web-research — pull requests and issues welcome. See CONTRIBUTING.md for the code layout, test suite, security invariants, and how to add a search backend.

---

Reference — details for pi (and power users)

Operational detail for pi and power users — not needed to get started. "The extension directory" below means wherever the package lives — under ~/.pi/agent/npm/ for the default npm install, ~/.pi/agent/git/github.com/wynainfo/pi-web-research/ for a git install (pi list shows installed sources).

Search keys: resolution details

Either key suffices — the provider auto-detects whichever is available (an explicit searchProvider in global config wins).

Kagi (default): the v1 key is read from, in order: KAGI_API_KEY env var → .env in this directory → kagiToken in the global config.json.

# alternative to the shell profile: a .env file in the extension directory (git-ignored)
echo 'KAGI_API_KEY=<your-kagi-v1-api-key>' > <extension-dir>/.env

Wyna (ai.wyna.info, Kagi upstream): resolution order: WYNA_API_KEY env var → ~/.pi/agent/models.json (providers.wyna.apiKey) → wynaApiKey in global config. API errors: 401 bad key · 402 top up at ai.wyna.info/account · 429 rate-limited (30/min) · 502 transient upstream (one retry) · 503 not configured server-side.

Legacy Kagi v0 (deprecated): set searchProvider: "kagi-v0" and supply its separate "Bot" token via KAGI_API_KEY_V0 (or legacy KAGI_TOKEN, or kagiV0Token). Migrating from v0? KAGI_API_KEY now holds the v1 key — move the old token to KAGI_API_KEY_V0.

Real env vars take precedence over .env. Global-config values support pi-style secret resolution: "$SOME_ENV" / "${SOME_ENV}" (env var), "!command" (command stdout, e.g. a secret manager), otherwise a literal token.

Security: secret resolution and the tokens/endpoints are honored only from the global config (or env / .env) — a project's .pi/web-research.json is restricted to a tuning-knob allowlist, so opening an untrusted repo cannot execute code, leak your token, or redirect it to another server. Full key list under Configuration.

Tools

Extraction chain: search provider → fetch (parallel, SSRF-guarded) → Mozilla Readability → HTML→Markdown → capped → sub-agent.

web_search(query, questions?, context?, count?, recency?, recency_strict?)

The default, context-cheap way to gather information. Searches the configured provider; the result shape depends on summaryMode (below). Raw pages never enter the calling agent's context.

• query — the search string (keywords).
• questions — optional; one or more specific questions the briefing must answer. Omit for a comprehensive digest of the top pages for the query.
• context — optional background on what you're researching, to orient the sub-agent.
• count — number of top pages to read, 1–10 (default 5).
• recency — day | week | month | year. Only read results newer than the window, newest first.
• recency_strict — default false. Only with recency: also drop undated (and suspect future-dated) results — may return fewer or zero results.

Summary mode (summaryMode):

• concise (default) — no sub-agent, no page reads: returns the ranked result list (title, snippet, date, URL) directly. The main agent then calls web_fetch(url, prompt) on whichever result it wants — that runs a sub-agent on the one fetched page (cached) and answers the specific question. Cheapest per search; detail is pulled only on demand.
• comprehensive — an isolated sub-agent reads the top N pages (markdown + llms.txt) in its own context and returns a thorough briefing (retains code, signatures, parameters, numbers, caveats) with inline [n] citations and a Sources list. Bounded multi-hop: if the top pages don't fully answer, the sub-agent may follow up to researchMaxHops cited links (default 3).

Either way, follow a citation with web_fetch (optionally with a prompt) when you need a source's full text or a targeted answer.

web_fetch(url, mode?, maxChars?, prompt?)

Fetch one page and get it back distilled, so the raw page never enters your context. A cheap, configurable model (fetchModel, falling back to subAgentModel, then the main model) does the distilling. Four behaviors:

• mode: "concise" (default) — page summarizer: a short summary of the page. Cheapest; best when you just need the gist.
• mode: "thorough" — page summarizer: a detailed summary that retains specifics, code, signatures, parameters, and caveats.
• mode: "full": the entire main content as clean markdown (+ the site's /llms.txt), no sub-agent. Use only when you need a source verbatim.
• prompt set — page researcher: answers just that question about the page (no whole-page summary); only the answer comes back. prompt overrides mode.

The default for the no-prompt case is set by fetchMode (default concise); a per-call mode always wins. If no sub-agent model is available, summary/research modes fall back to returning the full page.

Fetched+extracted pages are cached ~1h, so a follow-up web_fetch on an already-fetched URL is fast (no re-fetch; only the cheap model re-runs).

• maxChars — cap on page content read/returned (default perPageMaxChars).
• Only http/https URLs whose host resolves to a public address (see the SSRF note below).

Both tools return a clear error message (not a crash) if the token or a dependency is missing, or if a page can't be fetched/parsed.

Live UI: both tools stream their progress (searching → reading pages → following links → distilling → the streamed answer) with a transient status while they run. The TUI shows the call parameters, a header panel (provider, pages read, links followed / fetch mode), and the full output uncollapsed; without pi-tui, pi's default rendering applies.

Configuration (optional)

config.json in the extension directory (global) and/or <project>/.pi/web-research.json (project, merged over global):

{
  // --- sensitive: GLOBAL config only (ignored in project .pi/web-research.json) ---
  "searchProvider": "kagi",              // "kagi" (= v1, default) | "kagi-v0" (legacy) | "wyna"
  "kagiToken": "$KAGI_API_KEY",          // v1 Bearer key; optional (env / .env take precedence)
  "kagiEndpoint": "https://kagi.com/api/v1/search",  // v1 endpoint (POST); must be https
  "kagiV0Token": "$KAGI_API_KEY_V0",     // legacy v0 "Bot" token (only if using kagi-v0)
  "kagiV0Endpoint": "https://kagi.com/api/v0/search", // legacy v0 endpoint (GET)
  "wynaApiKey": "$WYNA_API_KEY",         // Wyna API key override (normally from models.json or env). Global-only.
  "wynaEndpoint": "https://ai.wyna.info/api/search", // Wyna search endpoint (GET/POST). Global-only.
  "allowPrivateNetwork": false,          // opt out of the SSRF private-IP guard

  // bot-protected-page fallbacks — sensitive, GLOBAL config only
  "browserFallbackEnabled": true,        // escalate bot-blocked fetches to headless Chromium (needs Playwright)
  "browserNoSandbox": false,             // RISKY escape hatch: drop Chromium's sandbox (only if user namespaces can't be enabled)
  "readerEndpoint": "",                  // optional self-hosted reader (Jina-Reader / FlareSolverr); no third-party default
  "readerMode": "auto",                  // "auto" | "jina" | "flaresolverr"

  // --- tuning knobs: may also be set per-project in .pi/web-research.json ---
  "browserTimeoutMs": 20000,             // navigation/challenge-settle budget for the headless render
  "acceptLanguage": "en-US,en;q=0.9",    // Accept-Language header for plain fetches
  "maxResults": 5,                       // default number of top pages web_search reads
  "perPageMaxChars": 12000,              // cap per fetched page
  "totalMaxChars": 40000,                // cap across all pages (sub-agent input budget)
  "fetchTimeoutMs": 12000,
  "concurrency": 5,                      // parallel page fetches
  "userAgent": "Mozilla/5.0 ...",

  // llms.txt (https://llmstxt.org/) injection
  "llmsTxtEnabled": true,                // probe /llms.txt from result domains
  "llmsTxtMaxChars": 8000,               // max chars of llms.txt content per origin
  "llmsTxtFetchFull": false,             // also try /llms-full.txt and walk parent dirs

  // web_search briefing sub-agent + multi-hop
  "summaryMode": "concise",              // "concise" (default: raw results, read on demand via web_fetch) | "comprehensive" (read pages, full briefing)
  "subAgentModel": "",                   // "provider/id" override; default: main model
  "subAgentThinking": "off",             // off|minimal|low|medium|high|xhigh
  "researchMaxHops": 3,                  // extra cited pages the sub-agent may fetch (0 disables; max 5) — comprehensive mode

  // web_fetch distilling sub-agent (page summarizer / page researcher)
  "fetchModel": "",                      // "provider/id" cheap model; falls back to subAgentModel/main
  "fetchMode": "concise"                 // default web_fetch depth w/o a prompt: "concise" | "thorough" | "full" (per-call `mode` overrides)
}

A project-level <project>/.pi/web-research.json is merged over the global config but is restricted to the tuning knobs (the second half of the block above); the keys marked GLOBAL config only are silently dropped from it, so an untrusted repo cannot redirect your queries, leak the token, relax the SSRF guard, spawn a browser, drop its sandbox, or send fetched URLs off-box.

Notes & limitations

• Endpoints: the active provider's endpoint must be https (the token rides in the Authorization header); override kagiEndpoint / kagiV0Endpoint / wynaEndpoint in global config if your account is provisioned differently.
• Bot-protected / JS-heavy pages: fetching escalates in tiers — plain fetch() with browser-like headers (Tier 0) → sandboxed headless Chromium that runs the page's JS (Tier 1, needs Playwright) → optional self-hosted reader (Tier 2, readerEndpoint). If all tiers fail, the host is recorded for the session and results carry [BLOCKED-DOMAIN] / [DO-NOT-RETRY] markers so the agent moves on (warn-only — a later call may retry, since the challenge may have cleared). If Playwright is missing, the failure text briefs the agent on the install steps and benefits, so you can just ask pi to set it up.
  • Isolation: the browser runs in a dedicated child process; page JS executes only inside Chromium's renderer sandbox and never in the pi process. The Chromium sandbox is kept on — on WSL2/containers it needs unprivileged user namespaces (sysctl -w kernel.unprivileged_userns_clone=1, or kernel.apparmor_restrict_unprivileged_userns=0 on newer kernels). If Chromium can't launch sandboxed it fails loudly with this guidance rather than silently disabling the sandbox; browserNoSandbox: true (global config) is the explicit, risk-accepting escape hatch. To relocate the browser binary, set PLAYWRIGHT_BROWSERS_PATH (for both the install and pi's environment).
  • Self-hosted reader (optional): set readerEndpoint (global config only) to a Jina-Reader-style or FlareSolverr endpoint. No third-party default; egress goes only where you point it.
• Page cache: extracted pages are cached in-process for ~1h (keyed by URL + char cap), so repeat reads — per-page RAG follow-ups, multi-hop, or the same URL across searches — skip the re-fetch. Failures aren't cached.
• Content-farm / fake-news defenses: at most 2 of the results read/shown per search come from the same publisher (a domain flooding the ranking can't dominate the research), and the sub-agent prompts enforce: same-publisher pages count as ONE source for corroboration; a page's self-description is never treated as credibility; high-salience events claimed by a single non-official source are verified against a primary source or quarantined as "Unverified — single source"; and the main agent is instructed to treat missing independent coverage as evidence against a claim.
• Citations: every result and source ships with a ready-made citation assembled by the extension — publish date, age, and the full URL as visible link text (2026-05-03 (2 months ago) [https://…](https://…)) — and the agent is instructed to copy it verbatim next to each claim it relays, so sources stay clickable and readable without clicking, and the format can't be mangled.
• Redundant-search guard: a near-duplicate web_search within a session (reworded query, or results fully covered by an earlier search) is short-circuited to a compact pointer at the prior findings; partially overlapping searches read only the genuinely-new pages. Refine the query to force a fresh search. (Token-set Jaccard ≥ 0.6, 30-min window.)
• SSRF guard: all outbound page/llms.txt fetches are restricted to http/ https, the host must resolve to a public address (loopback, RFC-1918, link-local 169.254.0.0/16 incl. cloud metadata, ULA, etc. are refused), and redirects are followed manually with each hop re-validated. This applies to every fetch, including the research sub-agent's multi-hop follow-ups. Set "allowPrivateNetwork": true in global config if you intentionally fetch localhost/internal docs. Response bodies are size-capped (≈3 MB per page, 256 KB per llms.txt) to bound memory.
• Prompt injection: fetched page text is untrusted. It normally flows into a sub-agent (the search summarizer, page summarizer, or page researcher), which is instructed to treat source text as data and ignore embedded instructions — so only the distilled output reaches the main agent. It enters the main agent's context directly only via web_fetch(url, mode:"full") (or a summary/research fallback when no sub-agent model is available). Treat any actions an agent takes off the back of fetched content with care.