Cua Computer Server

Server component for the Computer-Use Interface (CUI) framework providing low-level computer control primitives.

Documentation - Installation, guides, and configuration.

Interfaces

The Computer Server exposes multiple interfaces simultaneously:

HTTP/WebSocket - REST API and WebSocket for programmatic access
MCP via HTTP - Model Context Protocol over streamable HTTP at /mcp endpoint

Android emulators also expose the emulator's built-in EmulatorController gRPC service on port 8554 (via -grpc 8554 launch flag). Local Android sandboxes (AndroidEmulatorRuntime) use GRPCEmulatorTransport to call getScreenshot() directly on this service, bypassing the computer-server HTTP layer entirely. Benchmark: p50 ~20ms, ~49 RPS vs ADB-backed HTTP p50 ~519ms, ~1.8 RPS.

Installation

# Basic installation (HTTP/WebSocket only)
pip install cua-computer-server

# With MCP support
pip install cua-computer-server[mcp]

# With the generated native Cua Driver SDK backend
pip install cua-computer-server[driver]

# With the VNC backend
pip install cua-computer-server[vnc]

Usage

# Start the server on default port 8000
python -m computer_server

# Or with custom port
python -m computer_server --port 8080

# Allow external access explicitly
python -m computer_server --host 0.0.0.0

# With resolution scaling (useful for Retina displays or VMs)
python -m computer_server --width 1512 --height 982

# Delegate portable whole-desktop actions to Cua Driver in this process
python -m computer_server --backend cua-driver --capture-scope desktop

# Or connect to an already-running Cua Driver daemon
python -m computer_server --backend cua-driver --driver-mode daemon

By default the server binds to 127.0.0.1. Deployments that need access from other hosts should pass --host 0.0.0.0 or another explicit interface.

This provides:

HTTP API at /ws, /cmd, /status endpoints
MCP server at /mcp endpoint (requires fastmcp package)

MCP clients can connect via streamable HTTP at http://localhost:8000/mcp.

VNC backend

--backend vnc drives a remote target over RFB instead of the local OS, so it covers screen, pointer, scroll, and keyboard only. Host-scoped surfaces (shell, files, PTY, browser, windows) are refused rather than silently executed against the server's own machine.

python -m computer_server --backend vnc \
  --vnc-host 127.0.0.1 --vnc-port 5900 --vnc-password secret

The equivalent environment variables are CUA_VNC_HOST, CUA_VNC_PORT, CUA_VNC_PASSWORD, and CUA_VNC_FORCE_CAPS.

Shift and non-compliant servers

RFB sends a keysym and leaves it to the server to work out which physical key and modifiers produce it, so a compliant server presses Shift itself when it receives underscore. QEMU only does that for uppercase letters: shifted punctuation arrives unshifted, and Hello_World (a>b) is typed as Hello-World 9a.b0.

Pass --vnc-force-caps (or set CUA_VNC_FORCE_CAPS=1) to send shift-<key> from the client instead. It stays off by default: vncdotool documents it as a workaround for non-compliant servers, and it changes what every uppercase and shifted-punctuation keystroke puts on the wire.

When the server is started for you by the Python SDK, pass it through Computer(...) instead — it is forwarded to the VM alongside the other VNC settings:

computer = Computer(
    backend="vnc",
    vnc_host="127.0.0.1",
    vnc_force_caps=True,
)

Cua Driver backend

--backend cua-driver keeps computer-server's HTTP/WebSocket, MCP, shell, file, PTY, browser, accessibility, and window-management surfaces while routing portable desktop capture, pointer, scroll, and keyboard actions through the generated cua-driver Python SDK. The default embedded mode loads the Rust runtime into computer-server and does not require a daemon. daemon mode is a compatibility option for deployments that already own a long-lived driver. The driver backend never falls back to OS-native input injection. Separate mouse_down, mouse_up, key_down, and key_up calls return an explicit unsupported-operation error; use drag, click, press_key, or hotkey instead.

Each computer-server process opens a distinct driver session. Its capture scope defaults to desktop, which enables the get_desktop_state command and screen-absolute actions. Select auto or window with --capture-scope when a stricter session policy is required. In auto, call escalate_capture_scope with the concrete ladder-exhaustion reason before using desktop-only actions; computer-server never escalates implicitly. Strict window sessions cannot escalate. The equivalent environment variables are CUA_DRIVER_MODE, CUA_DRIVER_SOCKET, CUA_DRIVER_SESSION_ID, and CUA_DRIVER_CAPTURE_SCOPE.

Resolution Scaling

When running on Retina displays or in VMs where the coordinate system may differ, use the --width and --height flags to specify the target resolution:

Screenshots will be resized to the target resolution
Click coordinates received will be scaled from target to actual screen coordinates
Cursor position will be reported in target coordinates

This ensures the AI model sees consistent coordinates between screenshots and mouse actions.

Claude Code Integration

Start the server (or run as a service/LaunchAgent):

python -m computer_server --port 8000

Add the MCP server URL to Claude Code:

claude mcp add cua-computer-server --transport http http://localhost:8000/mcp

Available MCP Tools

The MCP interface exposes 40+ tools for computer control:

Screen & Mouse

computer_screenshot - Capture current screen
computer_click - Click at coordinates
computer_double_click - Double-click
computer_move - Move cursor
computer_drag - Drag from start to end coordinates
computer_scroll - Scroll at position
computer_get_screen_size - Get screen dimensions
computer_get_cursor_position - Get cursor position

Keyboard

computer_type - Type text
computer_press_key - Press a single key
computer_hotkey - Press key combination (e.g., Ctrl+C)
computer_key_down / computer_key_up - Hold/release keys

Clipboard

computer_clipboard_get - Get clipboard content
computer_clipboard_set - Set clipboard content

Shell

computer_run_command - Execute shell command

File System

computer_file_read / computer_file_write - Read/write files
computer_file_exists / computer_directory_exists - Check existence
computer_list_directory - List directory contents
computer_create_directory - Create directory
computer_delete_file / computer_delete_directory - Delete files/directories

Window Management

computer_open - Open file or URL
computer_launch_app - Launch application
computer_get_active_window - Get active window
computer_activate_window - Focus a window
computer_minimize_window / computer_maximize_window - Window state
computer_close_window - Close window

Accessibility

computer_get_accessibility_tree - Get UI element tree
computer_find_element - Find UI element by role/title

cua-computer-server