sidehand

Sidehand lets Claude, Codex, Cursor or any MCP client read and operate Windows apps behind the window you are using. It types, clicks, picks from lists and reads the screen as text without moving your mouse or taking your keyboard.

Why Sidehand

• Stays out of your way. It works in windows behind yours. Nothing comes to the front, and your cursor never moves.
• Several agents at once. Each agent gets its own session, so three apps can be filled in at the same time.
• Cheap to read. One short line per control, 4 to 20 times fewer tokens than agent-desktop on the same windows.
• Small and local. No daemon, no telemetry, no downloaded binary. About 2,400 lines you can read in an afternoon.

Quick start

Add the MCP server to your client. Nothing else to install (Windows 10 or 11, Node 18.3+).

{
  "mcpServers": {
    "sidehand": { "command": "npx", "args": ["-y", "github:Blue-B/sidehand", "mcp"] }
  }
}

Not sure where your client keeps that file? Paste this to your AI agent and it sets itself up:

Install Sidehand (https://github.com/Blue-B/sidehand) for yourself: add an MCP server named "sidehand"
with command "npx" and args ["-y", "github:Blue-B/sidehand", "mcp"] to your own MCP settings.
If you are Claude Code, also run `npm install -g github:Blue-B/sidehand` and add the `sidehand guard`
PreToolUse hook from the README. Then tell me whether I need to restart you.

Prefer a CLI? npm install -g github:Blue-B/sidehand, then tell your agent to use sidehand tree and sidehand act (see Usage).

Keep a stuck agent off your mouse (Claude Code). When an action fails, an agent may write its own script that moves the real mouse or pulls the app forward. With the CLI installed, this hook in .claude/settings.json blocks such Bash commands and tells the agent to ask you instead (pi does this on its own and asks you first):

{
  "hooks": {
    "PreToolUse": [{ "matcher": "Bash", "hooks": [{ "type": "command", "command": "sidehand guard" }] }]
  }
}

How it compares

How each number was measured: docs/benchmark.md.

Usage

$ sidehand tree "Contoso Orders"
window: Contoso Orders  (18 elements)
  e4 Edit "Customer"
  e8 ComboBox "Region"
  e12 Button "Submit order"

$ sidehand act e4 type "Blue-B Studio"
$ sidehand act e8 select Jeju
$ sidehand act e12 click
$ sidehand wait "Contoso Orders" "Order submitted"

MCP and pi expose the same operations as tools (ui_tree, ui_act, capture, ...).

Limit what it can touch: set SIDEHAND_ALLOW="Contoso Orders;Notepad" and every other window disappears from its view.

What works in the background

WPF ignores mouse messages that do not come from the real mouse, and WPF and browser pages take keys only while focused, so those need --allow-foreground. Games and other apps that draw their own UI expose no controls (capture still works). Apps running as administrator need an elevated agent.

How it works

Sidehand is a UI Automation client, the same interface screen readers use, compiled on your PC on first use. Classic controls get plain window messages (WM_SETTEXT, BM_CLICK, ...), which never activate a window. Everything else gets a UI Automation call while the target window is marked no-activate, so the app cannot pull itself to the front. Clicks and keys are posted to the app's own message queue, never to your mouse.

License

MIT