agent-desktop
agent-desktop is an open-source CLI and C library from Vercel Labs that reads the accessibility tree of desktop applications and exposes it as JSON with actionable element references. Agents use it to observe app state, execute interactions (click, keyboard, mouse, scroll, clipboard), and verify UI state changes via wait predicates. The tool works across macOS, Windows, and Linux, and integrates with agent-browser for combined desktop and web automation. It is designed for teams building AI agents that need to interact with desktop UIs without writing custom accessibility wrappers or platform-specific automation code.
agent-desktop is a desktop automation platform, integrating with agent-browser, Vercel and npm. InnovaAI rates it 4.1 of 10 for agency adoption, best for Technical Lead, Founder and QA Engineer roles.
Agency Audit
agent-desktop is a CLI and C library from Vercel Labs that exposes the accessibility tree of desktop applications as JSON, enabling AI agents to observe, interact with, and verify state changes across macOS, Windows, and Linux. Agencies building internal automation workflows, integrating agents with legacy desktop software, or automating repetitive UI-driven tasks benefit most from adopting it. The tool pairs with agent-browser for full-stack desktop and web automation, reducing manual testing and workflow scripting overhead for technical teams.
3recommended
36/mo
No paid plan published
Moderate
Illustrative scenario. Not a guarantee. Net capacity needs a verified paid base plan, and none is published for this service, so it is not modeled. Hours saved come from the service estimate; implementation, taxes, and unprovided usage charges are excluded.
- Technical Lead handling desktop application UI automation
- Founder handling legacy software integration testing
- QA Engineer handling repetitive data entry workflow automation
- Your agency does not have in-house engineering capacity to integrate agent-desktop into your agent stack or CI/CD pipelines.
- Your workflows are primarily web-based or SaaS-driven and do not involve desktop application automation.
- You are looking for a no-code RPA solution or a tool that non-technical team members can configure without developer involvement.
Internal Adoption Path
No paid plan published
36 hr/mo
3 seats × 12 hr each
$2,700/mo
modeled at $75/hr labor rate
No paid plan published
Illustrative scenario. Not a guarantee. No verified paid base plan is published for this service, so subscription cost and net capacity are not modeled. Implementation, taxes, and unprovided usage charges are excluded.
Platform Features
Core capabilities of agent-desktop
Accessibility tree JSON export
Reads the full accessibility tree of any desktop application and returns it as structured JSON with actionable element references. Enables agents to understand app layout and available interactions without manual UI mapping, saving QA and automation engineers 2-3 hours per app per week.
Cross-platform agent interaction
Executes click, keyboard, mouse, scroll, and clipboard actions on desktop apps via CLI or C library FFI. Works on macOS, Windows, and Linux without platform-specific scripting, reducing the complexity of multi-OS automation for technical teams.
UI state verification via wait predicates
Agents can verify that UI state changes occurred after an action before proceeding to the next step, eliminating flaky automation and manual verification checks. Compresses the observe-act-verify cycle that QA and automation engineers currently do manually.
Skeleton and drill-down navigation
Maps app UI with shallow skeleton snapshots and drills into specific regions on demand, reducing the overhead of parsing large or complex desktop interfaces. Helps automation engineers focus on relevant UI sections without loading the entire app state into memory.
Chromium and Electron DevTools integration
Exposes a DevTools port for web contents in Chromium and Electron apps, allowing agents to inspect and interact with embedded web UIs. Extends agent-desktop's reach to hybrid desktop applications without requiring separate web automation tooling.
CLI and C library dual interface
Offers both command-line and C library (FFI) access to desktop automation, enabling integration into agent frameworks, CI/CD pipelines, or compiled applications. Gives technical teams flexibility to embed automation in their existing tech stack.
What Makes agent-desktop Different
Unique advantages vs similar tools in this niche
Accessibility-tree snapshots with stable element refs
vs Pixel-matching desktop automation toolsIt reads accessibility trees and does not match pixels, giving agents deterministic refs to act on.
One consistent 60-command set across three OSes
vs Platform-specific automation scriptsThe CLI has the same 60 command names on every platform.
CLI plus in-process C library (FFI)
vs CLI-only automation wrappersIt ships as a CLI and as a C library with ad_* functions for in-process use.
Latest Updates
Recent releases and improvements for agent-desktop
1.0.1 Bug Fixes
Fix2026-10-27Remove the version line from skills (was showing 0.11.1 in 1.0.0); automate npm and GitHub releases when Cargo workspace version changes on main; remove custom Homebrew tap, Scoop bucket, and their package-update pull requests.
1.0.0 Release
New2026-10-27Add Linux support (x86_64 and aarch64, glibc 2.35+) via AT-SPI; release integrated macOS, Windows, and Linux CLI and C library as 1.0.0; bundle native CLI executables in npm installations; verify release archives and build provenance.
0.10.1 Bug Fixes
Fix2026-10-18Windows: let launch take a bare name without .exe.
Value Equation
Outcome-likelihood-time-effort assessment for agent-desktop
Value math requires real pricing
The Value Equation (dream outcome × likelihood ÷ time × effort) feeds directly into ROI math. agent-desktop has no published pricing, so we hold this section until real numbers are available.
Contact agent-desktopPricing
Pricing data not yet available for agent-desktop.
Reality Check
agent-desktop requires engineering effort to integrate into existing agent workflows and assumes your team is already building or deploying AI agents. It is not a no-code tool and does not reduce friction for non-technical roles. Payback depends on having 5+ hours per week of repetitive desktop automation work that justifies the integration cost.
High effort: requires technical configuration and team training
How This Accelerates White-Label Services
Who It's For
- ✓agencies-building-desktop-automation-workflows
- ✓developers-integrating-ai-agents-with-desktop-apps
- ✓teams-automating-legacy-desktop-software
Acceleration Steps
- 1Schedule onboarding with the vendor
- 2Configure read the accessibility tree of a desktop app and return it as json
- 3Connect agent-browser
- 4Launch your first client project
Academy for agent-desktop
Work through it in order: the course for this service first, then the modules behind it.
Course for this service
Agent Desktop Agency Implementation, Building Desktop Automation Services
Learn how to architect desktop automation services for agencies using Agent Desktop's accessibility tree JSON and cross-platform interaction model. This course covers client scoping, multi-OS deployment, accessibility permission workflows, and productizing desktop RPA as a retainer service that scales across Windows, macOS, and Linux environments.
Open the courseNo Academy modules are published for this service yet. Browse the full Academy
Why this category matters
The commercial case before the tooling.
Core concepts
The mental model you need to price and scope the work.
- Absorption HorizonConcept
Absorption Horizon is the expected lifespan of a narrow desktop utility before an operating system update or a broader automation platform reproduces its function for free. Agencies should price and scope desktop automation against that horizon, not against the tool's feature list. The category is thin and fragmented: rules-based file sorting, batch renaming, and local data entry are exactly the functions most likely to be bundled into Windows or Mac releases, or folded into agent builders that already sit inside a client's stack. A concrete case is EnlightPixel's SortSage, a Windows app that moves local files by name, extension, size, or age with a dry-run preview capped at 1,000 files. That is a real time saver for back-office delivery work, but it is one OS feature away from redundancy. Treat these tools as short-cycle internal efficiency purchases, keep client retainers off them, and avoid building a service line whose value proposition is a single local utility.
- Billable Hour Displacement RatioConcept
Billable Hour Displacement Ratio is the share of tracked back-office hours that desktop automation removes, weighed against what the agency does with the recovered time. A rules-based local utility that sorts and files a client's asset library might strip 6 to 10 hours a month from an account coordinator's week. Those hours become margin only if the retainer is fixed-fee and the work quietly disappears; on hourly billing they become a revenue hole the agency has to refill with higher-value delivery. The framework forces a decision before installation: is this workflow billed by time or by outcome? EnlightPixel's SortSage, with its dry-run preview capped at 1,000 files, is a clean example of a narrow utility whose value is entirely determined by how the agency prices the work it replaces. Measure the ratio per client, not per tool.
- Local Execution Trust BoundaryConcept
Local Execution Trust Boundary is the rule that desktop automation keeps client files and records on the machine where they already sit, so the risk question shifts from vendor security review to operator discipline. Agencies care because back-office file and data workflows still burn billable hours clients resist paying for, and a local utility converts that time into margin without a cloud data-processing agreement. The boundary is also the limit: a rules-based Windows sorter such as EnlightPixel's SortSage moves files by name, extension, size, or age with a dry-run preview capped at 1,000 items, which means the tool never sees content and cannot judge whether a file belongs in a client deliverable. Pair that with the churn risk of single-vendor dependency and the boundary becomes a scoping tool: local automation handles the deterministic 80 percent, humans and reviewable cloud workflows handle anything touching client confidentiality or judgment.
Decision and risk
How to judge the fit, and the ways it goes wrong.
- Desktop Automation Rule: Automate the Billable Hour, Not the Tool StackEvaluation Rule
Adopt a desktop utility only when the hours it recovers are already tracked and non-billable, and cap the commitment at internal efficiency until the same workflow proves large enough to migrate to a platform you can support.
- When a Desktop Utility Saves Under 5 Hours a Month, Keep It InternalEvaluation Rule
Run desktop utilities as internal margin tools and only promote one into a client-facing deliverable after you have logged the hours it saves across at least two months of retainer work.
- Why Desktop Automation Stalls at the Single-Utility StageFailure Pattern
- The Billable-Hour Mirage: Why Desktop Automation Never Reaches the RetainerFailure Pattern
8 modules selected for agent-desktop
Frequently Asked Questions
Answers about pricing, setup
agent-desktop reads the accessibility tree of desktop applications and returns it as JSON with actionable element references. AI agents use it to observe app state, execute interactions like clicks and keyboard input, and verify that UI changes occurred. It works via CLI and C library across macOS, Windows, and Linux, and integrates with agent-browser for full-stack automation of both desktop and web UIs.
agent-desktop is open-source and available on npm at no cost. There is no per-seat pricing or commercial license required for internal adoption.
Technical Leads and Founders building internal AI agents gain the most direct value, as agent-desktop provides the abstraction layer for agents to interact with desktop UIs. QA and Automation Engineers save time on manual testing and custom accessibility wrapper development. Operations teams automating legacy desktop software workflows reduce manual data entry and process verification overhead.
For a QA or Automation Engineer automating a single desktop application, expect 2-4 hours per week saved on UI mapping, interaction scripting, and manual verification checks. For Operations teams automating repetitive data entry or workflow tasks, savings range from 3-6 hours per week depending on task frequency and complexity. Payback is highest when the same workflow is repeated 5+ times per week.
No. agent-desktop exposes a CLI interface that works with shell scripts and a C library for compiled applications. If your team already uses JavaScript, Python, or Go for agent orchestration, you can invoke agent-desktop commands via subprocess calls or FFI bindings without learning new syntax.
agent-desktop is designed for desktop applications. For web automation, use agent-browser, which is built by the same team at Vercel Labs. Both tools share a similar observe-act-verify model, so teams can use them together for full-stack automation.