For coding agents

Coding agents drive a Mac the expensive way: screenshot → vision reasoning → one click → screenshot again. Each step costs seconds and tokens. axstream gives the agent a set of on-device primitives that collapse that loop — and a macro layer that makes any task it has done before replay in seconds with no model in the loop.

Install

uv tool install axstream        # or: pip install axstream
axstream install

axstream install wires everything into every agent found on the machine, idempotently:

Claude Code — skill at ~/.claude/skills/axstream, MCP server registered via claude mcp add
Codex — skill at ~/.agents/skills/axstream (the Agent Skills open standard), MCP server in ~/.codex/config.toml

The executor daemon (cua-driver) is a separate one-line install — axstream --doctor checks for it and tells you how.

Sandboxed agents (Codex's default) can read macro files but cannot reach the driver's unix socket — replay then looks like "daemon not running" when the daemon is fine. Driving the UI inherently needs approved execution outside the workspace sandbox. The installed skill teaches the agent this.

The accelerator primitives and native capture bridge

The fast primitives complement the agent's native computer use — each one removes a model cost, not just a machine cost:

tool	what it replaces	cost
`screen_text`	a screenshot + vision pass — returns the window as text lines with coordinates	~200ms, ~2KB
`find`	visually scanning for one control — returns a click-ready target	~30–600ms
`act`	one model turn per step — executes a whole batch of verified actions in one call	one call
`check`	a verification screenshot — "is this text visible yet?"	~250ms
`begin_capture`	starts a bounded trace of Codex native `sky` calls	one setup call
`compile_capture`	translates the successful native trace into a slotted macro	one compile call

An act batch speaks the axstream spec — open, click, type, key, with assert guards between risky steps. Clicks resolve through a verified ladder (accessibility element → OCR text anchor → visual patch → window-relative pixels), and the batch stops at the first failure with an exact handoff point:

{"failed_at": 2, "op": {...}, "reason": "...", "completed": 2}

The macro flywheel

The loop the installed skill teaches:

Before any UI task: list_macros — replay if one matches.
Novel task, clear steps: do it with one assert-guarded act batch.
Novel task, native fallback: call begin_capture, run its node_setup, then drive Codex computer use normally. get_app_state immediately before an element_index action preserves the durable role/title behind the index; before a coordinate click/drag it records only the source image dimensions needed for Retina- and resize-aware replay.
It worked: write_macro the act ops, or compile_capture the native trace. Run the returned teardown so later native actions are not recorded.
When a re-run is safe: verify_macro with a different captured slot value — the gate rejects first-run values that stale UI could satisfy.
Forever after: replay_macro, seconds, zero model calls.

The verify gate

A macro can replay to 100% completion and still not do the task — stale coordinates click where a button used to be, and nothing complains. So axstream never trusts an unchecked macro:

Every macro should end with an assert that only passes when the outcome is real ({"op":"assert","target":{"text":"..."}} — on-device OCR, ~250ms).
verify_macro / axstream verify <name> replays it once, live; if the terminal assert passes, the macro is stamped verified (shown in list_macros). Editing the macro demotes the stamp to stale.
Verification re-executes the task, so the gate refuses macros carrying risk:"risky" ops — never auto-verify anything irreversible (sending, paying, deleting).
Saving a macro that does the same task as an existing one replaces it (the old file is archived) — the library converges instead of rotting.

This design follows PreAct (Computer-Using Agents that Get Faster on Repeated Tasks), whose central result is that cached-replay systems collapse without exactly this gate.

The CLI, for agents without MCP

Everything above also speaks plain CLI — which is all the installed skill needs:

axstream list --json                 # discover (includes verified state)
axstream replay <name> --dry         # inspect without executing
axstream replay <name> --slots '{"query":"..."}'
axstream verify <name>               # the gate
axstream bench <name> --warmup 1 --runs 5   # p50/p95 per op
axstream menu                         # human-visible trust, slots, timing

Replay emits one JSON progress object per action — via says how each click was verified (ax_element / ocr_anchor / patch / window_pixel), and the final summary lists unverified_steps so the agent knows when to double-check the outcome.

axstream menu is an optional observability and manual-replay surface over the exact same library. It does not make Codex's execution engine faster by itself; it makes the flywheel visible and easy to operate: verification state, parameter forms, live progress, measured outcomes, and a private frecency ranking all stay available from the macOS menu bar.