Computer Use

Computer use lets an AI system observe interfaces and select actions such as clicking or typing. It is one action path for an AI Agent; not every agent controls a screen. Anthropic's tool documentation ↗ separates observations and input controls from the application that executes them.^[1]

This is ORBiS Tree's design comparison. A structured API exposes explicit fields and results. Browser document access can address web elements; screen-based control interprets visible positions and state. Check API permissions, element access and screen/session access separately. None of these interfaces grants business authority by itself.

A fictional task saves ten public AI tools in a table file. An acknowledged click does not complete the task. Verify that the file exists, contains ten intended entries without duplicates, includes the requested columns and retains original URLs.

If layout or a popup changes, observe the current state before reusing coordinates. If saving might already have happened, inspect the file before retrying. Connect task identifiers and output locations with Data Provenance.

The official security considerations ↗ discuss constrained environments and managing sensitive data and network exposure.^[1] Websites and images can carry Prompt Injection. Enforce actual permissions in the application and retain outcomes for AI Agent Oversight. Check current documentation for tool versions and availability.

In a fictional ten-task evaluation, eight correct results, one incorrect save and one incomplete task give 8/10=80% task success. Ninety-five acknowledged actions out of one hundred do not establish 95% task success. Record time, retries and errors with their own denominators. These counts do not compare real products.

WebArena ↗ evaluates whether goals are functionally achieved in a reproducible web environment.^[2] Its section 3.2 helps distinguish outcome validation from matching an action sequence. Web results do not generalize to every desktop application or establish a current model ranking.