Ui.Vision integrates the power of large language models (LLM). The result is a hybrid approach to automation. This builds upon our existing foundation that combines traditional web automation with local (fast & free) computer vision/OCR. Now the toolbox also includes cloud-based computer vision and the "intelligence" that Large Language Models (LLM) offer. AI support is new and beta. Based on your feedback, we will improve and expand the integration with the next updates. Support for local LLM like Llama and Mistral is planed, too.
We started in version 9.3.6 with two AI commands: aiPrompt and aiScreenXY. With V9.3.8 we added the powerful Computer Use command. All use the Anthropic Claude API. So to use them, you need to generate your own API key (see below) and then enter the API key on the new "AI tab" in the Ui.Vision settings page.
New AI settings tab
Visit the Anthropic Console and sign up for an account.
The "Get API keys" button is directly on the dashboard. Here, you can generate a new API key by clicking on the 'Create API Key' button. Now enter the key into Ui.Vision.
Anthropic Dashboard
Note: There used to be a free $5 API credit after signing up & verifying your phone number. But it seems this offer is gone. As a cost benchmark, the Anthropic API cost of running the below four demo macros is 0.02 US$:
Anthropic API cost benchmark for aiPrompt and aiScreenXY. See also aiScreenXY vs aiComputerUse
There are many good video tutorials that show how to sign up for a Anthropic API key. This is one of them.
Test Computer Use and other AI commands with the demo macros.
...then please contact us.
