Pi Web Core

A focused web search and page-fetching extension for the Pi coding agent.

Pi Web Core exposes two tools:

• web_search — search the web through a configured Provider.
• web_fetch — fetch HTTP(S) pages as readable Markdown or raw text.

Why Pi Web Core

• Minimal tool footprint — registers exactly two tools and no extra commands, shortcuts, browser UI, or workflow tools.
• Context efficient — small tool schemas reduce prompt and tool-selection overhead for the Agent.
• Focused API — search and page fetching stay predictable without media, PDF, repository-cloning, or content-cache APIs.
• Broad Provider support — keeps a compact public surface while supporting configurable search and fetch routing behind it.
• Secure fetching — includes SSRF protection, redirect validation, response limits, domain policies, and credential redaction.

Installation

pi install npm:pi-web-core

Pi Web Core requires Pi 0.37.3 or newer.

Tools

web_search

Search the web and return an answer with structured source citations when the selected Provider supplies them.

web_search({ query: "TypeScript 5.8 release notes" })

web_search({
  queries: [
    "React Server Components performance benchmarks",
    "React Server Components caching guidance",
  ],
  numResults: 10,
  recencyFilter: "month",
  domainFilter: ["react.dev", "github.com", "-example.com"],
})

Parameters:

Parameter	Description
query	A single search query.
queries	Multiple queries executed sequentially.
numResults	Requested results per query, from 1 to 20.
recencyFilter	day, week, month, or year.
domainFilter	Domains to include. Prefix a domain with - to exclude it.
proxy	Optional HTTP(S) proxy URL for this call.

Search Provider selection is configured in web-search.json rather than through the tool schema.

Supported search Providers include:

• OpenAI and Codex Hosted Search
• Exa
• Brave
• Parallel and Parallel MCP
• TinyFish
• Search1API
• Searchinfinity
• Querit
• Tavily
• Firecrawl
• Jina Search (jina, using https://s.jina.ai)
• SERPdive
• Kagi
• Bocha
• Ollama
• SearXNG
• DuckDuckGo
• Perplexity
• Gemini API and Gemini Web
• Kimi
• AnySearch
• XCrawl
• xAI
• Bright Data SERP
• SerpBase
• Serper
• Valyu

web_fetch

Fetch one or more HTTP(S) URLs.

web_fetch({ url: "https://example.com/article" })

web_fetch({
  urls: [
    "https://example.com/guide",
    "https://example.com/reference",
  ],
})

web_fetch({
  url: "https://example.com/api/status",
  mode: "raw",
})

Parameters:

Parameter	Description
url	A single absolute HTTP(S) URL.
urls	Up to 10 absolute HTTP(S) URLs.
mode	readable for Markdown extraction or raw for the exact textual response body.
proxy	Optional HTTP(S) proxy URL for this call.

Readable mode uses Readability and Turndown to convert HTML into Markdown. Text, JSON, XML, and Markdown responses are returned directly.

Fetch Provider routing supports:

• Direct HTTP
• Firecrawl
• Jina Reader
• TinyFish
• Search1API
• Querit
• Kagi
• Ollama
• Parallel and Parallel MCP
• Bright Data Web Unlocker
• Gemini URL Context and Gemini Web

Remote hosted Fetch Providers are disabled unless fetchRouting.allowRemoteHostedProviders is explicitly enabled.

Oversized Content

The default inline result limit is 30,000 characters. Configure it with maxInlineContentChars, up to 200,000 characters.

When a fetched page exceeds the limit, Pi Web Core returns a preview and writes the complete content to a restricted temporary file:

/tmp/pi-web-core-XXXXXX/content.md

The tool result includes the path in details.filePath for a single URL or details.tempFiles for multiple URLs. Temporary directories use mode 0700, files use mode 0600, and session shutdown triggers best-effort cleanup.

Configuration

The default configuration file is:

~/.pi/web-search.json

PI_CODING_AGENT_DIR takes precedence. When it is unset, XDG_CONFIG_HOME/pi is also supported.

Example:

{
  "provider": "openai",
  "openaiApiKey": "$OPENAI_API_KEY",
  "jinaApiKey": "$JINA_API_KEY",
  "openaiResponsesUrl": "https://api.openai.com/v1/responses",
  "searchRouting": {
    "providers": ["openai", "exa", "brave"],
    "useCurrentModel": false,
    "fallbackOn": [
      "transient",
      "quota",
      "network",
      "invalid-response",
      "unsupported"
    ]
  },
  "fetchRouting": {
    "providers": ["http", "firecrawl", "jina", "tinyfish"],
    "allowRemoteHostedProviders": false
  },
  "maxInlineContentChars": 30000,
  "proxy": "http://127.0.0.1:8080",
  "fetchContent": {
    "domainPolicy": {
      "allow": ["example.com"],
      "deny": ["blocked.example.com"]
    }
  },
  "ssrf": {
    "allowRanges": [],
    "trustEnvProxy": false
  }
}

Search Provider Selection

Use one of the following configuration forms.

A single Provider:

{
  "provider": "openai"
}

Concurrent aggregation:

{
  "provider": ["openai", "exa"]
}

All eligible Providers:

{
  "provider": "all"
}

Ordered fallback routing:

{
  "searchRouting": {
    "providers": ["openai", "exa", "brave"],
    "useCurrentModel": false,
    "fallbackOn": ["transient", "quota", "network", "invalid-response"]
  }
}

A top-level provider or searchProvider takes precedence over searchRouting.

Credentials

Provider credential fields support three forms:

{
  "braveApiKey": "literal-key",
  "exaApiKey": "$EXA_API_KEY",
  "openaiApiKey": "!/absolute/path/to/secret-command"
}

• A literal value is used directly.
• $NAME or ${NAME} reads a named environment variable.
• !command resolves the credential at request time.
• Prefix $$ or $! to escape a literal leading $ or !.

Common environment variables include:

OPENAI_API_KEY
BRAVE_API_KEY
EXA_API_KEY
PARALLEL_API_KEY
TINYFISH_API_KEY
SEARCH1API_KEY
SEARCHINFINITY_API_KEY
QUERIT_API_KEY
TAVILY_API_KEY
JINA_API_KEY
SERPDIVE_API_KEY
KAGI_API_KEY
OLLAMA_API_KEY
SEARXNG_BASE_URL
PERPLEXITY_API_KEY
GEMINI_API_KEY
XAI_API_KEY
BRIGHTDATA_API_KEY

Proxy Support

Set a global proxy in the configuration file:

{
  "proxy": "http://proxy.example.com:8080"
}

Or pass proxy to an individual tool call. An empty string forces a direct connection.

Proxy requests use a curl-backed transport. NO_PROXY is honored, and localhost is never sent through the configured proxy.

Security

Pi Web Core applies the following protections to page fetching:

• Only absolute HTTP(S) URLs are accepted.
• Loopback, private, link-local, metadata, and reserved addresses are blocked.
• DNS results and every redirect target are validated.
• fetchContent.domainPolicy can restrict allowed and denied hostnames.
• Responses are streamed with a 5 MiB limit.
• Hosted Fetch Providers require explicit opt-in.
• Credential headers are removed on cross-origin API redirects.
• Provider credentials are redacted from surfaced errors.
• Inline data: URIs are replaced with bounded omission markers.

Networks using TUN or fake-IP ranges can add a narrow exception:

{
  "ssrf": {
    "allowRanges": ["198.18.0.0/15"]
  }
}

Avoid broad CIDR exceptions.

Architecture

src/
├── index.ts
├── config/          shared config, paths, and credentials
├── tools/           web_search, web_fetch, and temporary output
├── search/          search types, routing, and lazy Provider registry
├── fetch/           fetching, HTML extraction, routing, and registry
├── providers/       Provider API adapters
├── net/             proxy, curl, API redirects, CIDR, and SSRF
└── observability/   request activity tracking

Search and Fetch Routers depend only on typed registries, not concrete Provider implementations. Provider adapters are loaded lazily and share one signature-aware configuration store.

Development

npm install
npm run typecheck
npm test

Additional checks used by the project:

npx tsc --noEmit --noUnusedLocals --noUnusedParameters
npx knip --include files,exports,dependencies
npm pack --dry-run

Fork and Acknowledgements

Pi Web Core is a fork of nicobailon/pi-web-access. The original project established the multi-Provider search and extraction foundation that this package builds on.

This fork is maintained as an independent package with a focused two-tool API and a configuration-controlled Provider model.

License

MIT