pi-camoufox-gateway

Give pi web search and page reading through a self-hosted Camoufox browser gateway. No API keys, no per-query cost, and JavaScript-rendered pages come back readable.

This repo is the pi extension. The gateway (camofox-browser) runs separately; this code just talks to it over HTTP.

Tools

camoufox_search(query, n?, engine?, resolve?)

Searches through the browser and returns ranked titles, urls and snippets.

• Google's results page is opened directly: POST /tabs/open at https://www.google.com/search?q=…, which is the same url the gateway's @google_search macro expands to. The macro went because the endpoint behind it cannot pass its own check — camofox-browser decides "did Google serve results" by requiring a heading link on a non-google host, and Google now serves organics as google.com/goto?url=…. Measured on the live gateway (2026-10-27): that call spent 31,285 ms and came back a DuckDuckGo page; opening the same url answered in 268 ms with 9–10 Google results on every query tried, cold and warm, including CJK. If Google walls us, DuckDuckGo answers, and the output lists which engines were tried.
• A search that is slow on one engine still gets its second engine. A timeout is not a verdict on the whole search; only an outcome the next engine would meet as well — gateway unreachable, credentials refused, the address guard refusing before anyone dialled — ends it. A hung snapshot costs one request deadline, not two. The chain has a budget of its own on top of that — two request deadlines, however many engines it walks — so a gateway that is up and answering nothing costs two deadlines rather than four, and the answer says which of the two ran out.
• The header names the engine that served the page, on every success and not only when something failed. "Google served this" and "DuckDuckGo served this after Google walled" are different claims about the same eight links, and a result set a model cannot attribute is one it cannot weigh. It was always in details.engine, but that is a field for a script, not text a model reads. Served is the exact word now, and not by convention: a page is only credited to the engine whose own host it sits on, so (via google) can no longer be printed over a DuckDuckGo page. That combination comes back as empty_results from google, duckduckgo is asked, and credits itself. Host it sits on is the gateway's answer and not our own guess: the rule reads the destination the gateway reports, and a gateway that reports none is answered as "owned" rather than refused on evidence nobody has. Under the shipped policy the address guard closes that door, refusing with destination_unreported before any of it is read — but turning the guard off entirely (allowPrivateHosts: true with no blockedHosts) opens it again, and a page then arrives credited with no substituted flag. That is pinned as a decision with a name in tests/snapshot.test.mjs rather than left a property of the code. Owned means the registrable domain, not the shape of the characters: google.co.uk, google.com.br and google.рф are Google's, while google.evil.com and www.google.com.evil.test — a host whose registrable domain is evil.test — are refused. What no suffix rule can do is tell an apex squatter from Google: whoever registers google under some TLD owns that apex, so https://google.xyz/ is credited to Google. Stated as a known limit rather than left a surprise; an allowlist of suffixes would close it and cost a search every time Google geo-redirects to a ccTLD the list does not name.
• A wall is where the page is, not what the url contains. The consent check used to be a substring test on the whole url, and the query is echoed into the search url — encodeURIComponent leaves dots alone — so camoufox_search("why does consent.google.com keep appearing") reached https://www.google.com/search?q=why%20does%20consent.google.com%20keep%20appearing&sei=…, matched it, and reported a healthy Google results page as a consent wall while DuckDuckGo answered an ordinary question. Measured live on 2026-10-28; no attacker was involved, though an attacker's ?next=https://consent.google.com/ walked through the same door. It matches the host now, normalised the same way every other host comparison in this repo is.
• Bing is not offered: there is no adapter for it. A Bing page that arrived anyway used to be read as whichever engine we had asked for — chrome is filtered by host, and no bing.com link is that engine's. It is now refused before it is parsed, by the same rule as above, so the mis-filter is not reachable and the engine that was asked never claims it. The one Bing case still pinned here is the harder failure: 378 chars of Camoufox's own Network Protocol Error, answered navigation_failed at a Bing url.
• engine is one of the two we ship, and the refusal says which way you missed. A name we have no adapter for comes back as "unknown engine", naming the ones we do have; a real engine you switched off under engines comes back as "disabled in your config". One of those is a typo and the other is a setting, so they are not merged into a single "not available".
• n is clamped to 10. That is Google's page width, and there is no pagination to ask for — which is why search.maxResults is refused above it rather than silently rounded down on every call. It has to be a whole number too: n: 2.5 used to slice to 2 rows and report nothing, because slice floors silently.
• The output counts what it threw away. A page that carried more than this call returned says so, and says which limit did it — your n, or the ceiling. Asking for 10 off a page of 12 used to print "10 result(s)" and read as the whole page, which is the output claiming more than it returned.
• resolve: true turns Google's google.com/goto?url=… wrappers into final urls. It costs one extra page load per result and runs at most three tabs at a time, so it is off by default.

camoufox_read(url, maxChars?, offset?, waitFor?, schema?, fresh?)

Loads a page in the browser and returns its text.

• Reports the final url after redirects.
• Defaults to 12,000 characters (~3K tokens) and tells you the offset to continue from. A full page from the gateway runs 20K+ tokens, which is the difference between reading a page and losing your context window.
• A larger maxChars is capped at read.maxCharsCeiling (40,000), not honoured. That default is the guard against one call swallowing the context window, and the caller it guards against is the one this tool is designed for — a model that guesses "more is better". The output says which of the two cut the page short, because truncated on its own cannot tell "the page is long" from "policy stopped me at the ceiling", and only the second one means a bigger request is pointless.
• Anything that is not a positive integer is refused, not capped. A ceiling enforced by comparing numbers has one value it cannot order, and NaN — which is what "1,000" becomes — is not above it, so the old guard let a malformed ask through and returned the entire page, saying nothing. Refusing names the argument instead of guessing at it: a caller that meant 1000 and typed "1,000" was not asking for the maximum. Measured on the budget actually spent, so a hand-built read.defaultMaxChars is refused on the same path and the message names which of the two is at fault.
• offset is checked the same way, with 0 as its floor — that is where a page starts. It ends up in (offset ?? 0) + sliced.length, and + concatenates: an offset of "100" used to come back as a nextOffset of "100100", rendered to the model as the way to continue and copied into a structuredContent.nextOffset the output schema declares a number.
• Waits for the page to render. openTab returning does not mean the content is there: a client-rendered site answers with its skeleton. A snapshot thinner than read.settleMinChars is re-taken until two consecutive snapshots come back identical — same text, same totalChars — up to read.waitForSettleMs. A page that never repeats (a clock, a carousel) therefore costs the whole budget, which is the point: a skeleton that swaps text at a constant width is not a finished page. If it never fills in, the output says so instead of letting the model conclude the site is empty.
• waitFor takes a CSS selector and asks the gateway to hold until it appears — use it when the settle heuristic cannot guess the content (waitFor: "#article"). A wait that cannot complete — no /wait route, or a gateway that ignores its own deadline until the client cuts it off at timeoutMs — costs the wait, not the page: the read continues and says which it was.
• schema accepts a JSON Schema whose properties carry x-ref hints; the tool snapshots the page first (refs only exist after a snapshot), then calls the gateway's extractor. The recorded contract requires type: "object", so an omitted or different type is refused rather than sent to a gateway that would reject it. Every top-level property has to carry an x-ref, and the check is all-or-nothing: a property without one is refused up front as unsupported, naming the key, and a schema that annotates only some of its properties is refused rather than answered with the subset it happens to address — as is an empty properties. Refs are read at the top level only, so a nested properties is refused too. All of it is checked before the navigation: a schema that cannot address anything is a typo, and the gateway's own answer to one costs a page load and arrives as gateway_error.
• Reads are cached for cache.pageMs (10 min) per pi session. A second read of the same url at the same offset comes back without opening a tab, and the output says it was served from cache. maxChars does not split the cache — it is a slice of a page already held — but offset does, because the gateway returns a window rather than a page. Requests carrying waitFor or schema always navigate: a stored snapshot can neither confirm a selector nor supply live refs.
• A cached page is re-checked on the way out, against the same destination rule that ran when it was stored. Narrowing blockedHosts or turning allowPrivateHosts off takes effect on the next read, not when the entry expires. Thin pages are never stored, so an unfinished render cannot become the answer for ten minutes. fresh: true skips the lookup and replaces the entry with what it fetches, so a plain read afterwards does not serve the older copy. One limit: a read already in flight carries no ordering against that replacement, so one that fetched earlier and lands later can still write its older page back.
• Refuses internal addresses, and keeps refusing after the browser moves. blockedHosts is applied, and so is a block on loopback, RFC1918, link-local (including the cloud metadata IP) and other non-global addresses — see network.allowPrivateHosts. Every destination gets checked, at tab open and on every snapshot, so a public url that redirects to 169.254.169.254 does not get read back — including by a gateway that reports no destination at open, which is the case a check at open alone would have missed.
• Refuses to return a page it cannot place. A snapshot that reports no url is checked against the tab list, and if nothing will say where the tab is, the read fails with destination_unreported rather than hand back unverified content under the url we asked for. Switching the guard off entirely (allowPrivateHosts: true, empty blockedHosts) drops this requirement too.

Install

Install it as a pi package and pi wires it into your settings:

pi install npm:pi-camoufox-gateway        # every session (~/.pi/agent/settings.json)
pi install -l npm:pi-camoufox-gateway     # this project only (.pi/settings.json)

The camoufox_search and camoufox_read tools are then available the next time pi starts. Try it for a single run without writing to settings:

pi -e npm:pi-camoufox-gateway

Manage it with pi list, pi remove npm:pi-camoufox-gateway (pi uninstall is an alias) and pi update.

Without npm, or straight from this checkout:

pi install git:github.com/odynvolk/pi-camoufox-gateway@v0.2.2   # pinned to a tag
pi install ./pi-camoufox-gateway                                 # a local checkout
pi -e ./src/camoufox-gateway.ts                                  # from this repo, nothing installed

You do not need to declare pi.extensions anywhere yourself — package.json points pi at src/camoufox-gateway.ts. Pointing the extension at your gateway is described below.

Requires Node 23+ and a running camofox-browser; the extension is shipped as TypeScript and pi loads it directly.

Configuration

Config resolves defaults → JSON file → environment variables.

File location: $CAMOUFOX_CONFIG_PATH, or $PI_CODING_AGENT_DIR/camoufox.json (default ~/.pi/agent/camoufox.json). See config.example.json.

Environment: CAMOUFOX_GATEWAY_URL, CAMOUFOX_API_KEY, CAMOUFOX_CONFIG_PATH, CAMOUFOX_ALLOW_PRIVATE_HOSTS.

Unknown keys are a hard error on purpose — a silently typo'd setting is worse than a loud one. A config file that exists but fails to load is not replaced with defaults either: every tool call returns config_error naming the file and the key, so a typo cannot quietly point the extension at the wrong gateway.

Values that are individually valid but cannot both be true are refused too, naming both keys: read.settlePollMs has to be below read.waitForSettleMs (at or above it the page is polled exactly once, and the budget stops meaning anything), read.waitForTimeoutMs below timeoutMs (the client aborts any request at timeoutMs, so a longer selector deadline could never be reached), and read.defaultMaxChars at or below read.maxCharsCeiling (above it, the default is capped on every read that omits maxChars — which is most of them, and none of them mentions a ceiling).

Structured results

Both tools declare an outputSchema and return a matching structuredContent. A model keeps getting content — the prose, with its header, its notes, and the offset to continue from — and a programmatic caller such as a codemode script gets the data instead of that prose to re-parse. A declared schema is also what lets pi treat the two as data rather than as text. pi does not validate the one against the other, so the shape below is the contract, and tests/camoufox-gateway.test.mjs checks both directions of it against the tools.

The keys a caller can rely on:

A failure returns the same shape rather than a different one: url is the url you asked for, text is empty, results is empty, and error carries a code from the Error codes section below. isError still tells pi which it is, so nothing has to guess from a blank page.

Everything else in details — settledInMs, cappedAt, fromCache, pageCarried, extracted, and the rest — is only filled in by a call that got that far: a failed one carries just the frame above and error, with a search's tried naming the engines it got through, plus gatewayHealth on the four codes that get a diagnosis (see Error codes).

A read also reports what it is doing while it does it: opening the tab, holding for a waitFor selector, and re-snapshotting a page that has not finished rendering. Those reach pi's tool updates, not the model.

Error codes

The tool output starts with the code so the model can react instead of guessing.

Four codes — gateway_error, timeout, navigation_failed, tab_not_found — also get a second line, because "the gateway failed" and "the gateway is answering, but nothing is behind it" are different instructions:

gateway_error: gateway error 500: snapshot exploded
Gateway state after the failure: ok=true, browserConnected=false, browserRunning=false, activeTabs=0, activeSessions=1, consecutiveFailures=3.

Only what that gateway actually reported is printed, so a 503 recovery answer comes back as its own two fields — ok=false, recovering=true — and a gateway that answered nothing answers as unknown — the gateway did not answer GET /health. The same facts, unnarrated, are on details.gatewayHealth: the message is for the model, details is for a script.

That second line is asked for only after a call has already failed — a working one costs nothing — never remembered between calls, since a stale count of live tabs is a wrong count, and bounded by the same 2s the liveness probe runs to. A gateway older than GET /health gets its failure back with nothing added. The liveness gate itself stays on GET /: a 503 means recovering, and refusing every call while the gateway says it is coming back would be a worse outage than the one being diagnosed.

What this does not do

• No browser or gateway code — that lives in camofox-browser.
• No clicking, typing or scrolling. Refs ([e12]) are only valid until the next navigation, which makes an interaction API hard to promise. The one exception is waitFor, which asks the gateway to hold for a selector rather than acting on the page.
• No defence against a hostname that resolves to an internal address (DNS rebinding, a localhost alias). The address guard reads the url and the redirect chain, not the DNS answer; egress policy belongs on the gateway.
• No LLM in the tool path. The tools return raw data; your model does the reading.
• Sessions: tabs live in a gateway session named after your pi session id and are torn down on session_shutdown. A lost POST /tabs/open response also triggers a sweep: the gateway may hold a tab we were never given an id for, so it is listed and closed rather than left behind.

Development

npm install
npm test          # unit + fixture tests, no network access
npm run typecheck
npm run test:live # live smoke test against the configured gateway
python3 scripts/record-fixtures.py  # re-record snapshots after a gateway or Google change

Requires Node ≥ 23: the tests import src/*.ts directly, which depends on Node's native type stripping. On 20 or 21 you get ERR_UNKNOWN_FILE_EXTENSION instead of a test failure.

The snapshot fixtures under tests/fixtures/ are real captures. If a parser test fails, re-record them before concluding the parser broke.