@agimon-ai/po-computer-use
Session scoped semantic computer control through the Po Desktop capability.
Package details
Install @agimon-ai/po-computer-use from npm and Pi will load the resources declared by the package manifest.
$ pi install npm:@agimon-ai/po-computer-use- Package
@agimon-ai/po-computer-use- Version
0.0.0-alpha.183- Published
- Nov 22, 2026
- Downloads
- 31.3K/mo · 5,136/wk
- Author
- agiflow-ai
- License
- BUSL-1.1
- Types
- extension
- Size
- 1.1 MB
- Dependencies
- 8 dependencies · 4 peers
Security note
Pi packages can execute code and influence agent behavior. Review the source before installing third-party packages.
README
@agimon-ai/po-computer-use
Session-scoped semantic computer control through the Po Desktop capability.
Requirements
- Node.js 24 or newer
- A supported packaged macOS Po Desktop with native Computer Use permissions
- The Pi version declared by this package's peer dependencies
Use
In Desktop, enable Computer Use in host settings, then select the default-off Computer Use mode. When the agent needs control, it asks for a window or the whole desktop and a duration. Approve once from an owner-authenticated client. The computer-use Activity card monitors the live grant and offers Stop; there is no separate setup form. Only one session can hold a live Desktop run. Remote activation does not bypass Desktop opt-in or grant checks. To stop from another app on macOS, press Cmd+Shift+Esc. If macOS cannot register the shortcut, use Computer Use → Stop Computer Use in Po Desktop or stop from Activity.
The computer-use mode exposes Activate, Deactivate, and Doctor actions. /computer-use reports the current state.
While active, the package exposes three tools:
computer_stateobserves the authorized window and returns a semantic snapshot.saveScreenshot: truealso keeps the screenshot in session artifacts undercomputer-use/<run>/and returns itsartifact:path.computer_actionperforms one of the semantic actions advertised for an element in a current snapshot. Approved-window grants use the bundled, unmodified agent-desktop CLI and show a visual agent cursor. The cursor is not a separate keyboard or mouse. Whole-desktop grants continue to use the Swift backend for the existing cross-app, Dock and menu-bar behavior.computer_execexecutes a reusable JavaScript or erasable TypeScript function and its relative helpers in a restricted runtime, then returns a fresh semantic observation.
Po’s Swift helper records MP4 and audio. Finished recordings are copied into session artifacts under computer-use/ in bounded ranges. For a window grant, an approved request for extra trace capture also keeps bounded JSONL action metadata and PNGs of that window. The trace does not contain element refs, entered text, or raw CLI debug output. Recording and tracing stop before sensitive input can be sent. Treat these artifacts as private desktop content.
Generated functions receive the authorized program API, JSON input, bounded logging, and cancellation. They do not receive Node or filesystem access. Script-internal observations omit screenshots by default, and includeScreenshot: true requests an image explicitly. Full Node execution requires both trusted: true and an exact host-configured PO_COMPUTER_USE_SCRIPT_PATHS allowlist. Trusted scripts remain unsandboxed, including their imported dependencies. See Author computer functions for the contract, limits, trust boundary, and examples.
With code mode, computer.targets() lists candidate windows and computer.request() asks for human approval. Hosts without code mode use computer_targets and computer_request. The state, act, run and stop members are available only during an active grant. The codemode-how-to-computer-use guide is listed whenever the mode is selected.
Disabling the mode or host opt-in, stopping control, expiry, or Desktop disconnection stops further actions. An approval does not change who can access the session. The owner can monitor or stop control from the same signed-in browser that approved it.
Public API
import { ComputerScriptRunner, createComputerUseSessionClient } from '@agimon-ai/po-computer-use';
import type { ComputerScriptRun } from '@agimon-ai/po-computer-use';
The session-scoped server plugin owns the agent and hub routes. Their host-issued context tokens remain opaque and session-bound inside the request broker.
Development
pnpm fixcode
pnpm typecheck
pnpm test
pnpm build
License
Business Source License 1.1: free for personal use, organisations under USD 1M revenue, non-profits and education. See LICENSE and https://agimon.ai/licensing.