@underactive/pi-topping-web-tools

Web tools bundle for pi: keyless URL-to-markdown fetching, PDF text extraction, and headless Playwright browser automation, sharing one preapproved-host allowlist and permission model.

Install

pi install npm:@underactive/pi-topping-web-tools

The web_browser tool additionally requires Playwright's Chromium (~150MB):

npx playwright install chromium

Why this extension

Web work has two distinct needs: clean context from public sources and browser-level verification of the live experience. This bundle gives pi both, so an agent can move from a documentation question to an interactive check without stitching together unrelated extensions. Public-page retrieval is keyless, while browser automation remains available for the cases where a static fetch cannot answer the question.

• Keep research focused. fetch_markdown converts public documentation, READMEs, and articles into Markdown, extracting the primary article instead of a page's surrounding navigation. Its cache makes iterative questions against the same source faster.
• Reach PDF-only sources. pdf_extract reads the text layer of specifications, datasheets, and papers that markdown fetchers treat as undisplayable binary content, and that pi's read tool cannot parse.
• Validate the rendered product. web_browser runs a fresh headless Chromium context for client-rendered pages, local development servers, and user journeys that static HTML cannot represent. It can inspect the accessibility tree, interact with controls, and surface console and failed-network diagnostics.
• Use the appropriate tool, not the heavier tool by default. Fetch static public material first; move to browser automation when JavaScript, visual state, or user input is part of the question.
• Keep control over access. The tools share a host allowlist and confirmation workflow; web_browser also enforces approved URLs on HTTP(S) requests and WebSocket connections inside its isolated browser context. Fetched page text is explicitly marked as untrusted, and browser sessions do not reuse the system browser profile.

Examples

Give pi a task like the following; the extension supplies the web capabilities needed to complete it.

Research a dependency before changing code

Read https://react.dev/reference/react/useEffect. Summarize the cleanup behavior, identify the parts relevant to subscriptions, and cite the sections I should review before changing this component.

fetch_markdown returns the page as focused Markdown, making it useful for documentation research without spending context on site chrome. Repeated follow-up questions can reuse the cached response.

Verify a local user journey

Open http://localhost:3000, set a 390×844 viewport, add an item to the cart, and report any failed network requests or console errors. Save a screenshot of the final state.

web_browser can drive the local application as a user would, then collect the diagnostics that explain a failed interaction rather than only reporting that it failed.

Investigate a JavaScript-heavy page

Review the migration guide at https://example.com/migration. If the important content is not present in the static page, inspect the live page in the browser and return the breaking changes that affect this project.

Start with fetch_markdown for efficient static content; use web_browser when the page depends on client-side rendering or requires interaction. The two tools let the agent choose the lowest-cost path that still produces reliable evidence.

Tools

Tool	Description
fetch_markdown	Fetch a public URL and return its content as markdown (cached, keyless)
pdf_extract	Extract the text layer from a remote or local PDF, with page markers
web_browser	Headless Chromium automation — navigate, click, type, screenshot, evaluate, logs, cookies

fetch_markdown

Complements pi's built-in search-backed web_fetch with a lightweight, keyless fetch path for public documentation, READMEs, and articles:

• Native fetch — no API keys, no provider routing
• HTML → markdown — turndown conversion for readable agent context
• Readability extraction — @mozilla/readability (via linkedom) strips nav/footer/sidebar boilerplate from article pages by default; pass raw: true for the full page. Falls back to full-page conversion when a page doesn't look like an article
• 15-minute LRU cache with revalidation — faster repeat lookups; expired entries are revalidated with If-None-Match/If-Modified-Since when the server sent ETag/Last-Modified, so unchanged pages cost a 304 instead of a full download
• Strict redirect handling — redirects that change host, scheme, or port, or that leave a preapproved path, return the redirect URL instead of being followed; call again to follow (www. prefixes are treated as the same host)
• Host confirmation — prompts for non-preapproved hosts (session or durable allow/deny; see Saved permissions)

Usage notes:

• HTTP URLs are upgraded to HTTPS automatically
• Content is returned in 100K-character windows; truncated responses report the offset to pass for the next window
• For GitHub repo content, prefer web_fetch or the gh CLI
• JS-rendered SPAs may return empty markdown (static HTML only)
• Authenticated or private pages (Google Docs, Confluence, Jira) are not supported

Command: /clear-fetch-markdown-cache — clear the in-memory URL cache.

pdf_extract

PDFs are otherwise unreachable: fetch_markdown and pi's web_fetch classify application/pdf as binary, and the read tool cannot parse it. This tool extracts the text layer via unpdf (a serverless build of Mozilla's PDF.js, keyless and with no native dependencies).

• Remote or local — pass url for a remote PDF or path for an absolute or relative path, or a file:// URL; provide exactly one
• Page markers — output is delimited with --- Page N --- so pages can be cited
• Page selection — pages accepts "3", "1-5", or "1,4,7-9"; defaults to every page
• 15-minute LRU cache — the whole parse is cached, so a later pages or offset call reuses it; local files are re-read when their modification time changes
• Host and file confirmation — remote URLs use the shared allowlist; local files inside the working directory are read without prompting, anything outside prompts (session-scoped allow/deny for local files; session or durable for remote URLs)

Usage notes:

• Content is returned in 100K-character windows; truncated responses report the offset to pass for the next window
• Scanned or image-only PDFs have no text layer and report that explicitly — there is no OCR
• Limits: 25MB download, 2000 pages, and a 30s extraction timeout

Command: /clear-pdf-extract-cache — clear the in-memory PDF text cache.

web_browser

Single tool with an action parameter:

Action	Parameters	Description
navigate	url	Open a URL or file:// path (HTTP upgraded to HTTPS except localhost/.local/private IPs)
screenshot	fullPage?, selector?, toFile?	Capture PNG screenshot (base64 image, or temp file path when toFile=true)
click	selector	Click a CSS selector
type	selector, text, mode?	Fill (mode=fill, default) or key-by-key type (mode=press) into a selector
hover	selector	Hover over a CSS selector (tooltips, hover-menus)
press	key	Press a keyboard key globally (Enter, Escape, Tab, etc.)
select_option	selector, values	Select option(s) in a <select> dropdown (comma-separated values)
set_viewport	width, height	Resize the browser viewport (persists until changed or browser closes)
evaluate	script	Run JavaScript in the page context (re-checks permission for the current page URL)
get_content	—	Return raw page HTML
get_text	selector?	Return visible text (layout-aware innerText; default selector body)
get_markdown	—	Return page content as markdown (headings, lists, links)
wait_for	selector?, state?, networkidle?	Wait for selector state and/or network idle
get_accessibility_snapshot	selector?	Return ARIA accessibility tree (YAML)
get_console_logs	—	Drain captured console/page errors
get_network_logs	—	Drain captured egress blocks, failed requests, and 4xx/5xx responses
get_cookies	—	Return cookie metadata as JSON (name, domain, path, flags; values are redacted)
set_cookies	cookies	Set cookies (each needs url or domain+path)
go_back / go_forward / reload	—	Browser history navigation / reload
scroll	selector?, frame?, deltaX?, deltaY?	Scroll an element into view (selector) or scroll by pixel deltas (default deltaY = viewport height)
drag	selector, targetSelector, frame?	Drag-and-drop from selector to targetSelector
upload_file	selector, files, frame?	Set files on an <input type=file> (absolute or relative paths; prompts when a file is outside the working directory)
set_dialog_behavior	dialogAction?, promptText?	Configure how future JS dialogs (alert/confirm/prompt) are resolved for the rest of the session (default: dismiss)
get_dialog_logs	—	Drain captured dialogs (type, message, default value, resolution)
list_tabs	—	List open tabs (index, URL, title, active)
switch_tab	index	Switch the active tab (from list_tabs)
close	—	Close the browser

Optional timeout (ms, default 30000) applies to navigation and to selector-based actions (click, type, hover, select_option, screenshot, get_text, get_markdown, wait_for, get_accessibility_snapshot, scroll, drag, upload_file). Other actions use Playwright defaults.

Browser network egress

web_browser enforces its permissions inside every fresh browser context:

• Every HTTP(S) document, script, image, XHR/fetch, beacon, and other routed request is checked against the live preapproved, session, and durable permissions before the request is sent. Path-scoped preapprovals remain path-scoped; session and durable grants remain exact-origin grants except that bare and www. host variants are treated as equivalent.
• Redirect responses are inspected before Chromium receives them. Approved top-level redirects, including simple bare-domain ↔ www. redirects, are followed one gated hop at a time. During navigate, a redirect to an unapproved URL returns Blocked: redirect to <origin> requires approval; redirects triggered by later page interactions are blocked and reported in network logs. Redirected subresources are blocked rather than followed and also appear in network logs.
• WebSocket handshakes use the same live permission check. Service workers are disabled so they cannot bypass request routing.
• Client-side navigation and popup destinations are gated. A blocked popup is closed without becoming active, and a blocked same-tab navigation returns to the prior approved page.
• data: and blob: subresources are allowed because they do not create network egress, while top-level data:/blob: documents are denied. file: documents require approval for the target path; local-file subresources are allowed only from an approved local-file page.
• Decision errors fail closed. Route handlers never prompt; call navigate on a blocked target to use the normal approval prompt. Blocks are reported by get_network_logs as blocked by egress policy.

A navigation approved with Allow once is retained as a browser-session origin grant so the document and its later same-origin requests can continue to load under request-level enforcement. Choosing a durable duration still controls whether that origin is available in later sessions.

Strict egress can make an approved page incomplete when it depends on an unapproved CDN, API, font, or analytics origin. Navigate directly to that origin to approve it, then return to the page and reload.

Selectors are CSS selectors, or Playwright's role= and text= selector engines (e.g. role=button[name='Submit'], text=Sign in). Pass frame (a CSS selector for the containing <iframe>) to scope click/type/hover/select_option/screenshot/get_text/wait_for/get_accessibility_snapshot/scroll/upload_file/drag into that frame.

Opening a link with target=_blank or window.open opens a new tab whose destination is checked against live permissions. An approved popup becomes the active tab; a popup to an unapproved URL is closed without becoming active and the current tab stays active. Use list_tabs to see all open tabs.

JS dialogs (alert/confirm/prompt) are dismissed by default so they never block automation; call set_dialog_behavior before the action that triggers one if you need accept instead (and, for prompt, a promptText value).

Commands:

• /browser — show open/closed status, current URL, title, tab count, console/network/dialog counts, session-approved host count, and saved durable host count
• /browser-close — force close the browser
• /browser-screenshot — save a full-page screenshot to a temp PNG file

Status line: a footer segment (browser: example.com) appears while the browser is open.

Statusline feed

web_browser also publishes its current state as a pi custom session entry, so pi-topping-statusline can render it without depending on this package:

• Type: pi-topping-web-tools/browser
• label (text): current hostname, local file, ready, or an empty string when closed
• open (number): 1 when open, 0 when closed

For example, subscribe to label with prefix browser: and text format:

{
  "type": "pi-topping-web-tools/browser",
  "field": "label",
  "prefix": "browser: ",
  "format": "text"
}

Entries are published only when the state changes, except that the current state is republished immediately at session start and once after the first turn. Closing the browser publishes { "label": "", "open": 0 } so consumers can clear stale labels. Set PI_SUPPRESS_NOTIFICATIONS to 1, true, or yes (case-insensitive) to hide the native footer; feed publication continues.

Saved permissions

When the host confirmation prompt offers a durable option ("Allow for 1 day", "Allow for 1 week", "Allow for 30 days"), the grant is persisted to ~/.pi/agent/web-permissions.json (mode 0600) and applies across all sessions and projects.

• Per-tool scope. A fetch_markdown grant does not authorize web_browser, which executes JavaScript and drives interaction. Each tool scopes its grants independently.
• Exact origin. Grants are stored as scheme://host[:port] and do not cross ports or schemes. For web_browser egress, only the bare and www. variants of the same remote hostname are equivalent; no other subdomains inherit the grant.
• Max 30 days. All grants expire; there is no indefinite option. Expired entries are pruned automatically.
• Local files excluded. file:// prompts never offer durable options; local-file access needs a path-scope design that is out of scope here.
• Headless behavior. Active durable grants apply when there is no interactive UI (e.g. RPC or print mode). Without a grant, non-preapproved hosts remain blocked.

Commands:

• /web-permissions — list and revoke saved grants. Bundled preapproved hosts are not listed (they are code, not user grants).

Security model (shared)

• Preapproved hosts: common documentation and dev sites are allowed without prompting. The allowlist lives in src/permissions.ts and is shared by both tools.
• User confirmation: other hosts prompt Allow once / Allow for this session / Allow for 1 day / Allow for 1 week / Allow for 30 days / Deny. For web_browser navigation, Allow once becomes a browser-session grant because request-level enforcement must authorize the loaded document's later requests.
• URL validation: all tools reject embedded credentials and overlong URLs. fetch_markdown and pdf_extract additionally reject loopback, private, and link-local hosts and any non-HTTP(S) scheme on a url. web_browser accepts localhost, private IPs, and file:// paths so it can drive local dev servers and local HTML — each still prompts for confirmation unless session-approved or covered by a saved durable grant.
• Local file resolution: pdf_extract resolves a path through realpath before any check, so a symlink pointing outside the working directory is treated as outside and prompts. Directories, FIFOs, and devices are rejected, and the size limit is enforced from stat before any bytes are read. web_browser upload_file follows the same rule: files outside the working directory prompt for confirmation (session-scoped) before they are read.
• Isolated browser context and routed egress: each session uses a fresh Chromium context (no shared profile/cookies with your system browser). HTTP(S) requests, redirect targets, client navigations, popups, and WebSocket connections are checked against live web_browser permissions; service workers are disabled.
• Untrusted content boundary: fetch_markdown results are wrapped in <untrusted-content url="…"> tags, and pdf_extract results in <untrusted-content source="…">, so the model treats fetched page and document text as data, not instructions.
• PDF resource limits: extraction is bounded by download size, page count, and a timeout. Because the bundled PDF.js runs on the event loop rather than a worker, the timeout bounds extraction across its await points but cannot interrupt a fully synchronous parse — the size and page caps are the primary defence.

Limitations

• web_browser requires the Chromium download.
• Screenshots consume significant context on vision models — prefer get_text, get_markdown, or get_accessibility_snapshot.
• Network logs capture blocked/failed requests and 4xx/5xx responses, not successful traffic.
• Request routing buffers HTTP(S) responses and blocks redirected subresources, so streaming endpoints and resources that rely on redirects may not behave like an unrestricted browser.
• Playwright routing covers HTTP(S) and WebSockets, not browser transports such as WebRTC; OS-level egress sandboxing remains outside this extension's scope.
• pdf_extract reads only an existing text layer; scanned documents need OCR, which is out of scope.

Popups and navigation

Same-tab client navigation and popups are checked against the live web_browser permissions. Unapproved document requests are aborted before reaching the target; blocked popups are closed without becoming active, while a blocked same-tab navigation returns to the prior approved page. Approved popups become active only after their main-frame navigation commits.

Development

Requires Node >= 22.19 (tests run TypeScript directly via node --test).

npm install
npm run typecheck
npm test          # browser tests require Chromium