Letting AI use your desktop
Found this helpful? Share it:
Found this helpful? Share it:
Computer use lets an agent look at one computer's screen and move its mouse and keyboard, the same way a person would. It is powerful, so it is off until you turn it on, and you can stop it at any moment 🖱️
Free and upComputer use is available to every verified account, including the free tier. Anonymous visitors cannot use it.
Instead of running commands, the agent takes a screenshot, decides where to click or what to type, and acts. It works on apps and websites that have no command-line interface, and it can drive more than one monitor on the same computer.
The agent controls the real desktop. It can click buttons, submit forms, and change settings, just like you can. Only turn it on for computers and tasks where that is acceptable, and stay nearby while it works.
Computer use is granted per computer, with an Allow computer use consent toggle:
Cloud computers have it available by default, because you provisioned the machine for idapt to use.
Your own machines stay closed until you allow it on the machine itself. If the desktop app is running there, a native consent window pops up, and clicking Allow enables it. On a headless host, enable it from the daemon CLI instead.
Enabling the capability on the host is the real boundary. idapt can ask to control the screen, but a computer that has not consented simply refuses.
A desktop has a single mouse and keyboard, so only one chat can drive a given computer at once. The first on-screen action takes an exclusive control lease. A second chat that tries to drive the same computer is turned away until the first one finishes.
The lease releases on its own when the run ends, and the agent can release it early. If a run stalls, the lease also times out so the computer never stays locked.
While an agent has control, a border appears around the screen so it is obvious the computer is being driven, alongside a small always-on-top Stop button.
Press Stop to hand control back at once. It releases the lease and hides the border, so the agent can no longer act on the screen.
Pressing Stop in the chat cancels the whole run, including anything the agent was about to do.
There is also an automatic backstop: a single chat is capped at 400 on-screen actions in any 10-minute window, so a loop cannot run away unattended.
Every screenshot the agent takes is saved into the chat's own Drive folder, so the run leaves a visual trail you can scroll back through. You can open any screenshot later to see exactly what the agent saw when it acted.
Each computer has a Computer use tab that lists the sessions that drove it. Use it to review what happened, and when, after the fact.
Prefer commands when a task can be done from the terminal: they are faster and easier to audit. Reach for computer use when a task truly needs the graphical interface. See Computer commands and files.
Related articles
Computers
Pair your own machine or launch a cloud one, then let your AI run commands and work with files on it.
Computer commands and files
Run commands through the daemon and browse remote files with the built-in file manager.
Autonomy and confirmations
Decide how much an agent can do on its own, approve or deny individual actions, and stop a run at any time.
Was this helpful?