Anthropic Introduces “Computer Use” AI Tool
Anthropic, an artificial intelligence software company, has launched a new AI-powered tool named “Computer Use,” capable of controlling a user’s mouse cursor to perform basic tasks on a computer. This tool, available exclusively via API with the company’s mid-range 3.5 Sonnet model, was released alongside updates to Anthropic’s Claude and Haiku models. The tool enables users to provide multi-step instructions to execute tasks such as moving a cursor, clicking buttons, and typing text, potentially handling hundreds of steps.
Anthropic explained that the AI works by analyzing screenshots of what is visible to the user and calculating how many pixels the cursor needs to move to click in the correct location. Training Claude to count pixels was a crucial step for accurate mouse control, according to the company. While it can perform tasks like clicking, it still struggles with more complex actions such as drag-and-drop functions and operates based on rapid screengrabs rather than live video feeds, which may cause it to miss brief notifications or screen changes.
Despite these limitations, the tool is now in public beta testing after being trialed by employees from companies such as Amazon, Canva, Asana, and Notion. Anthropic acknowledged that the system remains “cumbersome and error-prone,” providing an amusing example of the AI abandoning a coding task to browse photos of Yellowstone National Park.
Competition and Industry Impact
The release of the “Computer Use” tool puts Anthropic ahead of its competitors, such as OpenAI, who have yet to publicly release similar capabilities. This development marks the beginning of what industry experts are calling an “arms race” in the AI sector, where companies are vying to develop tools that can automate menial office tasks. The potential for these tools to streamline repetitive activities in jobs like quality assurance and optimization work has garnered attention from developers.
Anthropic has long emphasized to investors that its AI models could perform certain office tasks more efficiently and affordably than human employees. The public release of the Computer Use feature brings the company one step closer to realizing that vision. However, the debate continues over whether such tools will merely assist workers or result in widespread job displacement across industries. Anthropic’s Computer Use could be part of this larger trend, though its full impact remains to be seen.
Safeguards and Future Concerns
While the potential of the tool is clear, there are significant concerns surrounding its misuse, particularly with sensitive applications like election interference. To address these issues, Anthropic has implemented several safeguards, including classifiers designed to flag potential abuse and limit the AI’s ability to interact with certain online platforms. The company is also on alert for any potential misuse of the tool during the upcoming US elections.
Anthropic stressed that the current version of Computer Use is not advanced enough to pose serious risks, but it remains vigilant in monitoring its use. The company continues to test and refine the tool with developers, seeking to enhance its functionality while preventing misuse. However, as with any evolving technology, the effectiveness of these safeguards will depend on ongoing research and improvements.
Anthropic’s Computer Use tool is now available for public testing, as the company works to address emerging challenges and explore new, positive uses for the technology.
