We’ve integrated Anthropic Computer Use into the Ui.Vision browser extension. This gives everyone the option to use/demo Anthropic Computer Use directly in their browser - no Docker required. You only need to install the open-source Ui.Vision browser extension for Chrome, Edge, or Firefox.

Step-by-Step Instructions

1. Install the Ui.Vision browser extension for Chrome, Edge, or Firefox
2. Install the XModule native app (available for Windows, Mac, and Linux). This app allows the browser extension to simulate mouse clicks and keyboard typing
3. Enter your Anthropic API key into the Ui.Vision extension. That’s all - you’re ready to go!

Computer Use Tutorial

To get started, we recommend trying one of the pre-made Computer Use macros. You’ll find them at the top of the macro list in Ui.Vision. Macros that use Computer Use start with “CU_”.

Computer Use (inside Ui.Vision) playing TicTacToe in the browser:

(Soon:) If you want to have a chat-like conversation with Claude Computer Use, open the sidebar and select the AI tab. Then start prompting.

read more

Update Dec 5, 2024: We integrated Anthropic Computer Use into Ui.Vision. So if you want to control your browser with AI, you can do it with Ui.Vision already now.

–

Rumours have it, that Google may release “Project Jarvis” (its reported codename) Chrome extension this month. It would perform tasks like research gathering, product purchasing, and flight booking.

read more

Anthropic just announced an exciting new capability for their AI assistant: Claude computer use, which allows it to interact with your computer by looking at your screen and controlling your mouse and keyboard.

How to use Claude computer use

Using Claude computer use is surprisingly straightforward: the system works by letting Claude see your screen through screenshots and then giving you instructions about what to do next.

Getting started is made easy with the Claude computer use demo, which Anthropic has provided on GitHub. While there isn’t yet an official Claude computer use tutorial, the Docker demo environment is helpful. The Claude computer use Docker setup is the quickest way to begin.

We tested the Anthropic Claude computer use features with tasks like opening a web browser, installing software, and editing computer code. So how good is it?

read more

Vision, the trade fair for machine vision, returns to in the Länd from 8-10 October, 2024.

read more

In 2022, we told our readers on our automation blog that iMacros, once the most used web automation tool in the world, would soon stop being available. Now, in January 2024, we see that the company owning iMacros has not only stopped updating and supporting it (which we expected) but also completely removed it from the Chrome and Firefox web stores.

read more