pi-agent-browser

English | 日本語

Typed browser automation tools for Pi, powered by agent-browser.

Use this package when Pi needs to operate real web pages, extract rendered context, keep login sessions, or capture visual evidence without falling back to ad-hoc shell commands.

What you get

• Rendered-page context extraction — read text from public pages or the active authenticated browser tab.
• Reliable interaction handles — use accessibility snapshots with stable @ref targets for clicks and form fills.
• Evidence capture — save screenshots and extracted text to predictable artifact paths.
• Session persistence — reuse login state with named agent-browser sessions and restore support.
• Safer browsing controls — constrain work with allowedDomains, maxOutput, and explicit state management.
• Typed Pi tools — each browser action has its own schema instead of one free-form command string.

Requirements

• Pi / Feynman runtime with package support
• agent-browser installed and available on PATH
• Chrome for Testing installed through agent-browser install

Check your local setup:

agent-browser doctor

If agent-browser is installed outside PATH, set:

export AGENT_BROWSER_BIN=/absolute/path/to/agent-browser

Installation

Install from npm:

pi install npm:@53able/pi-agent-browser

Install from GitHub:

pi install https://github.com/53able/pi-agent-browser

Install from a local checkout:

git clone https://github.com/53able/pi-agent-browser.git
pi install ./pi-agent-browser

Registered tools

• agent_browser_open — open a URL or launch about:blank
• agent_browser_read — extract text or JSON from a URL or active rendered tab
• agent_browser_snapshot — get accessibility tree refs for interaction
• agent_browser_click — click by @ref or CSS selector
• agent_browser_fill — fill an input by @ref or CSS selector
• agent_browser_screenshot — capture screenshots to disk
• agent_browser_eval — evaluate JavaScript in the active tab
• agent_browser_state — save, load, list, show, rename, clear, or clean saved state
• agent_browser_close — close a browser session
• agent_browser_handoff — pause and hand control to a human for 2FA, CAPTCHA, or identity verification
• agent_browser_login_handoff — spawn an unattended browser for interactive login flows (Google, OAuth)
• agent_browser_commit — confirmation gate for irreversible actions (payment, delete, publish, send)
• agent_browser_doctor — run agent-browser doctor

Common workflows

Read a page and save the extracted source

Ask Pi:

Read https://example.com through the browser and save the extracted text to outputs/browser/example.md.

Equivalent agent_browser_read arguments:

{
  "url": "https://example.com",
  "outputPath": "outputs/browser/example.md",
  "allowedDomains": ["example.com"]
}

Interact with a rendered page

1. Open the page.

{
  "url": "https://example.com",
  "session": "research-example",
  "allowedDomains": ["example.com"]
}

2. Call agent_browser_snapshot to get fresh @ref values.
3. Use agent_browser_click or agent_browser_fill with those refs.
4. Call agent_browser_read for text or agent_browser_screenshot for visual evidence.
5. Close the session with agent_browser_close when finished.

Keep login state across runs

Use a stable session name and restore: true:

{
  "url": "https://example.com",
  "session": "work-example",
  "restore": true,
  "headed": true
}

After logging in once, future opens with the same session and restore can reuse saved browser state.

Inspect or clear saved state

List saved states:

{
  "operation": "list"
}

Clear one saved session state:

{
  "operation": "clear",
  "sessionName": "work-example"
}

Clear all saved states only when intentionally resetting everything:

{
  "operation": "clear",
  "all": true
}

Safety guidance

• Pass allowedDomains whenever the browsing scope is known.
• Use outputPath when extracted content supports a report, audit, or later verification.
• Take a fresh snapshot after navigation, form submission, modal dismissal, or a failed click.
• Prefer agent_browser_read for textual evidence and agent_browser_screenshot for visual evidence.
• Use agent_browser_eval only when read or snapshot cannot expose the required state.
• Review saved browser state before clearing broadly.

Human handoff

Not every step should run unattended. Three tools hand control back to a human at the right moments:

• Safe boundaries — agent_browser_handoff pauses the agent for steps it cannot or should not perform itself: 2FA/OTP entry, CAPTCHA, identity verification. It captures a screenshot for context, then blocks on a real confirmation dialog until a human resumes or aborts. agent_browser_read and agent_browser_snapshot also run a lightweight keyword scan and flag likely checkpoints in their output as a nudge to call this tool.
• Interactive login flows — agent_browser_login_handoff solves the problem that Google, OAuth, and similar providers block sign-in when agent-browser is used, because automation detection happens during the login step itself. This tool spawns an unattended Chrome process (with no CDP attached) so the human can log in normally. After login succeeds, the still-running browser is live-attached to the named session and kept open as the permanent backing browser—it remains visible on screen with an open remote-debugging port until closed with agent_browser_close. Use this instead of agent_browser_open whenever a task is expected to hit an interactive Google or OAuth sign-in flow. Agents should call agent_browser_close as soon as the session is no longer needed, rather than leaving the debugging port open indefinitely.
• Stop before commit — agent_browser_commit is a dedicated gate for irreversible actions: payment, deletion, publishing, sending. It captures evidence, requires explicit human confirmation before clicking, and refuses outright (without clicking) if no interactive UI is available to ask. Use it instead of agent_browser_click for the final confirming step of any such action.

All three tools require an interactive session (TUI or RPC with UI support); in headless/unattended runs they refuse rather than proceed silently.

agent_browser_open remembers whether each named session was launched with headed: true. If agent_browser_handoff is called for a session that was not opened headed, it automatically warns the human and includes the exact agent_browser_open arguments (session name, headed: true, restore: true, and the last known URL) needed to reopen a visible window — no more guessing which command to rerun. Prefer headed: true up front for any task that might hit a login, 2FA, CAPTCHA, or identity-verification step, since some challenges are single-use and reopening mid-flow may require restarting the login. agent_browser_login_handoff records the resulting session as headed: true (since the login browser is visible and running); you don't need to reopen it to see it.

Troubleshooting

Run:

agent-browser doctor

Or ask Pi to run agent_browser_doctor. This verifies installation, Chrome, daemon, security, providers, network, and launch tests. It also reports any live login-handoff browsers left open and cleans up stale tracking entries.

If you have sessions created with agent_browser_login_handoff, agent_browser_doctor will report any that still have live login browsers running, showing the session name, process ID, debugging port, and last known URL. Call agent_browser_close for each session when done — these hold an open remote-debugging port until closed. Stale entries (tracking data for processes no longer running) are cleaned up automatically with no notification. agent_browser_login_handoff itself also names any other sessions still running a live login browser in its success message, so you notice without a separate agent_browser_doctor call.

Common fixes:

npm install -g agent-browser
agent-browser install

Project history

If you maintain or contribute to this package, read the project history in English or Japanese for the package-name decision, npm Trusted Publishing setup, and semantic-release workflow.

Release automation

Releases are automated with semantic-release. Maintainers merge Conventional Commits into main; GitHub Actions runs npx semantic-release@25, determines the next version, publishes to npm through Trusted Publishing, creates the Git tag, and writes the GitHub Release.

Commit message examples:

• fix: handle empty snapshot output → patch release
• feat: add browser state export tool → minor release
• feat!: rename tool arguments or BREAKING CHANGE: → major release

The npm Trusted Publisher should continue to point at .github/workflows/publish.yml. No long-lived NPM_TOKEN is required.

License

MIT