pi-browser-control 🛠️

Fix UI bugs by giving Pi Agent direct access to your browser.
Stop describing visual glitches to your AI. Pi opens your web app, visually spots layout issues, inspects computed CSS, tests interactive elements with mouse & keyboard, and fixes your frontend code live.

---

The Problem: Coding Agents Are Blind to UI Bugs

Normally, coding agents write frontend code blind. They cannot tell if:

• A div is not centered on screen.
• A button's text is getting chopped off or overflowing its container.
• A dropdown menu doesn't open when clicked.
• An element overlaps the navigation bar on smaller viewports.
• A component is silently throwing JavaScript errors in the console.

You have to take manual screenshots, paste them into chat, describe what's wrong, wait for a code change, and manually refresh the browser.

---

The Solution: Give Pi Direct Control of Your Browser

pi-browser-control gives Pi eyes, hands, and devtools inside your browser:

1. Pi Opens Your Web App Live: Launches your local browser (Brave, Chrome, Edge) right on your desktop so you can watch.
2. Spots Visual Defects with Set-of-Marks (SoM): Stamped badges ([1], [2], [3]...) let Pi see every button, input, and link with 100% click precision.
3. Diagnoses the Exact CSS Root Cause: Pi inspects computed styles (display, flex, margin, padding, box-sizing, overflow, z-index) to understand why text is cropped or an element is off-center.
4. Captures Console Logs: Pi immediately catches uncaught JavaScript crashes, React errors, or CSS 404s.
5. Fixes Code & Verifies Live: Pi edits your frontend files (CSS, HTML, React, Vue), reloads the browser, and visually verifies that the bug is resolved.
6. Auto-Closes When Done: As soon as the task is fixed, Pi cleanly closes the browser.

---

How Pi Diagnoses Common UI Bugs

1. Off-Center Divs & Misalignments

Pi compares the element's bounding rectangle against its parent:

Left Space  = Child.left - Parent.left
Right Space = Parent.right - Child.right

If Left Space !== Right Space, Pi calculates the exact pixel offset (e.g., "Shifted 48px to the left"), checks whether margin: 0 auto or justify-content: center is missing, and corrects your CSS.

2. Cropped & Overflowing Button Text

Pi compares scrollWidth (the actual width needed for text) against clientWidth (the visible box): If scrollWidth > clientWidth with overflow: hidden, Pi detects that text is being chopped off, identifies the constraining rule (max-width: 140px), and fixes it.

3. Non-Responsive Clicks & Broken Actions

Pi uses browser_click to test buttons and inspects browser_console_logs to catch uncaught runtime exceptions or missing event handlers.

---

Installation

Run this command in your terminal:

pi install npm:pi-browser-control

Or test it for a single session without installing:

pi -e npm:pi-browser-control

---

Example Prompts to Fix UI

Once installed, simply ask Pi to fix your interface:

• "Open http://localhost:3000, check why the hero header is not centered, and fix the CSS."
• "Test the signup form in the browser. Check if any button text gets truncated on mobile viewports."
• "Open the app, click the 'Submit' button, and inspect any console errors preventing the modal from opening."
• "Check why the navigation drawer overlaps the content and fix the z-index and flex layout."

---

The UI Fixing Loop

┌─────────────────────────────────────────────────────────────┐
│ 1. browser_open             → Opens web app on desktop      │
│ 2. browser_screenshot       → Sees UI with Set-of-Marks     │
│ 3. browser_inspect_element  → Diagnoses CSS & text clipping │
│ 4. browser_console_logs     → Checks JavaScript errors      │
│ 5. edit (Pi core tool)      → Modifies your CSS/JSX/HTML    │
│ 6. browser_navigate: reload → Visually confirms the fix     │
│ 7. browser_close            → Shuts down cleanly            │
└─────────────────────────────────────────────────────────────┘

---

Tools Provided to Pi

Tool	Purpose
browser_open	Opens a URL in your local browser and captures the initial labeled screenshot.
browser_screenshot	Captures a viewport screenshot with numbered Set-of-Marks badges.
browser_inspect_element	Inspects computed CSS layout styles, box dimensions, padding/margin, and text clipping.
browser_click	Clicks an element by badge ID (target: 1), coordinates (x, y), or CSS selector.
browser_type	Types text into inputs, search boxes, or textareas.
browser_hover	Hovers over an element to reveal hover states, tooltips, or dropdown menus.
browser_scroll	Scrolls the page up, down, left, or right to inspect below the fold.
browser_console_logs	Fetches browser console logs, warnings, and JavaScript runtime errors.
browser_navigate	Navigates to a URL, reloads (action: 'reload'), or goes back/forward.
browser_close	Closes the browser window and frees up resources.

---

Slash Commands (for Users)

In any interactive Pi session:

• /browser open <url> — Opens a URL in your browser.
• /browser screenshot — Captures a labeled screenshot.
• /browser logs — Displays recent browser console messages.
• /browser close — Closes the browser window.

---

Why It's Built on Your Local Browser

• Zero 300MB Downloads: Auto-detects Brave, Chrome, Chromium, or Edge already installed on your system.
• Real Desktop Experience: Runs headed so you can see your real website running on your real display server (X11 / Wayland).
• Supports Custom Paths & CDP: Set PI_BROWSER_PATH=/path/to/browser or connect to existing tabs via PI_BROWSER_CDP_URL=http://localhost:9222.

---

License

MIT © Gourav Raghuwanshi