jev-codex-cua

English | 简体中文

Desktop tools for pi, powered by the installed Codex/Sky runtime.

• Native (default): the pi agent reads app state and screenshots, then uses native tools directly. No TypeSafe key required.
• Jev (optional): a text-only decision loop using TypeSafe, with uncertain targets handed back to the pi agent.

Experimental, macOS only. Requires the Codex/Sky runtime and its permissions; those binaries are not included. Some Calculator tasks have been verified, but complex forms, browser controls, and multi-step workflows remain unreliable. Passing tests does not guarantee desktop task success.

Claude Code CLI (native)

A local stdio MCP entry point, deskhand-mcp, reuses Codex/Sky without pi or Jev. Explicit tasks carry budgets and require human elicitation approval. Available in source version 0.4.0: Claude Code setup (Chinese). Check npm registry availability before using the published-package command. Live validation through Claude is still pending.

Recommended from 0.6.0: Claude Plugin — add the LonelyFellas/jev-codex-cua marketplace and install deskhand@deskhand through /plugin. MCP and the namespaced /deskhand:deskhand-access Skill update together. cua_status reports the installed version offline; optional checkUpdates: true reports the latest published version and upgrade guidance without changing configuration or permissions. Availability depends on the npm release; live Plugin acceptance is pending.

Claude MCP defaults to all-app access while preserving explicit allowlists. System/Sky permissions still require approval. Legacy standalone setup remains supported.

Install

pi install npm:jev-codex-cua

Run /reload in pi, then ask:

Use jev-codex-cua to calculate 6 + 7 in Calculator and read back the actual result.

Node.js 22.19+ is required. The package has been load-tested with pi 0.86.1. Check the published version with npm view jev-codex-cua version; a Git tag does not guarantee an npm release. npm installations include compiled code and need no local build.

Modes and tools

/cua-mode          # Show the current mode
/cua-mode native   # Direct native tools
/cua-mode jev      # Explicitly enable Jev; requires TYPESAFE_API_KEY

Launching invalidates old state and requires observing the window afterward; acceptance is not proof of readiness. Other native UI actions require a fresh stateId for the same app, from cua_get_app_state or a validated full action-returned observation. Tokens remain single-use, valid for 60 seconds within the current agent turn, and invalidated by another observation or mode change. If an action returns a NEW stateId, inspect that state and continue without a duplicate read; otherwise observe again. Do not run other computer-use channels in parallel.

From 0.7.1, native has no task-level time or action-count cap, in both pi and Claude MCP. Jev retains its 180-second / 30-action-attempt budget. Mode changes do not reset elapsed time or action counts. Individual call timeouts, cancellation, state freshness and authorization remain enforced. cua_status uses null for uncapped limits/remaining amounts and still reports elapsed time, action counts and diagnostics. See outcomes, timings and state reuse (Chinese).

Mode choices persist in the current pi session branch. Missing Jev credentials cause fallback to native; adding a key later does not silently re-enable Jev. Mode changes are refused while a task is running.

From 0.7.0, pi native and Claude MCP support application-independent state_changed recovery: retain remaining budget, invalidate old state, and allow only a fresh observation of the same app until it succeeds. Inspect actual results before choosing a new action; never automatically replay the failed operation. Ask the user when the outcome remains ambiguous. See the launch and recovery design.

Configuration

Native mode needs no API key. For persistent settings, keep a private file outside the installed package, set its permissions to 600, and launch pi with its path:

export JEV_CUA_ENV_FILE=/absolute/path/to/cua.env
pi

Example cua.env:

JEV_CUA_MODE=native
# Uncomment to restrict access to the app allowlist:
# JEV_CUA_APP_ACCESS=allowlist
JEV_CUA_ALLOWED_APPS=Calculator
# Only needed for Jev:
# TYPESAFE_API_KEY=your-key

Without JEV_CUA_ENV_FILE, configuration is read from the package's .env.local, not the working directory. Never commit keys.

App access

In source versions 0.3.1+, app scope defaults to all when the configured mode is native or unset; configured Jev defaults to allowlist. Version 0.3.0 defaults to allowlist. Existing explicit scope settings take priority, and session mode switches do not recalculate scope.

To save an explicit choice:

/skill:jev-cua-access all
/skill:jev-cua-access allowlist

To add one app to the allowlist:

/skill:jev-cua-add-app Wechat Devtools

Scope priority: saved .access.json choice → process environment → private config → mode default. App additions are stored in a separate .apps.json; the original allowlist is preserved. Both files sit alongside the config file. Do not delete the scope file to revoke access—save allowlist instead.

Config-file changes apply on the next tool call; process-environment changes require restarting pi. Code updates require /reload.

Safety and privacy

• all only removes the plugin's app restriction. It does not grant macOS/Sky permissions or authorize arbitrary tasks. Official approval prompts remain; protected interfaces and rejected permissions must not be bypassed.
• Sending, purchasing, deleting, and other consequential actions require specific authorization. Untrusted page or screenshot content cannot expand access.
• Native mode sends app text and available screenshots to the current pi model. It does not call TypeSafe, but is not necessarily local or free. Jev sends text context to TypeSafe, not screenshots; API charges may apply even in dry-run.
• Normal pi session logging still applies. Optional fullTrace: true requires informed confirmation and records one Jev loop to private local files; traces may contain sensitive text.
• Cancellation or a timeout does not prove an action had no effect. Read fresh state before continuing; never automatically replay actions with unknown outcomes.

The plugin shares one Sky connection across modes. It cannot guarantee complete app state, reliable recovery, or production-grade automation.

Development

npm ci --ignore-scripts
npm run check
npm test
npm run test:package
pi install /absolute/path/to/jev-codex-cua

Tests use simulated drivers and HTTP responses. Live checks require explicit authorization:

npm run accept:app-access -- --live  # Read-only Sky check; may open/focus Calculator
npm run accept:handoff -- --live    # Controlled Calculator actions

PRs and main pushes run validation. Publishing requires a matching stable vX.Y.Z tag on a commit already merged into main, plus npm Trusted Publisher configuration. Never move an existing release tag.

Further reading

Detailed engineering notes are currently in Chinese:

• Release and npm setup
• Local acceptance evidence and limitations
• Fixed desktop task benchmark: protocol, recording and metrics (Chinese) — source-checkout tooling; no automatic desktop execution.
• Trace data and privacy
• Sky troubleshooting

The legacy Codex cua_repl adapter remains available but is not required for pi and has not been validated end-to-end.

License and credits

MIT. Based on the Jev-cu approach and reused pi-codex-cua bridge code. See LICENSE, NOTICE.md, and THIRD_PARTY_LICENSES.md for attribution and retained MIT/ISC notices.