AI-powered Visual Web Automation

UI.Vision RPA's image and text recognition allow you to write automated visual tests with UI.Vision RPA - this makes UI.Vision RPA the first and only Chrome and Firefox extension (and Selenium IDE) that has "👁👁 eyes". For more information please see visual UI Testing and the XClick command.

AI-powered Visual Desktop Automation

UI.Vision RPA can not only see and automate everything inside the web browser. The same image and text recognition technology can automate your desktop as well (Robotic Process Automation, RPA). For more information please see Desktop Automation and the XClick command.

Classic Web Automation, fully Selenium IDE compatible

UI.Vision RPA is a open-source alternative to iMacros and Selenium IDE, and supports all important Selenium IDE commands. When you invest the time to learn UI.Vision RPA, you learn Selenium IDE at the same time. You can even import Selenium IDE test cases.

Automation by Example

Often it is best to learn from examples. UI.Vision RPA installs many ready-to-run demo macros that showcase its features and all important commands.

<em>UI.Vision RPA</em> Demo Macros

Command Reference

Image-driven Commands (AI powered Web and Desktop Automation)

Desktop Automation Commands

Screen Scraping Commands

Selenium IDE Commands (see also How is UI.Vision RPA related to Selenium IDE?)

Web Scraping with Selenium IDE

Command Line API Commands - control UI.Vision RPA from any program or script

Top Top

Flow Control

UI.Vision RPA was the first web macro recorder with built-in flow control commands like if/else/endif, while/endWhile or GotoIf. Follow the links for more details and examples.

Top

Loop button

The dropdown next to the Play button contains the loop feature - run macros as a loop. Looping a macro is useful for doing basic load testing with the Selenium IDE, for performance monitoring or to simply test the stability for the test case (macro) itself. The csvRead command to read a CSV file line by line makes also use of the LOOP button.

<em>UI.Vision RPA</em> for Chrome looping...

Top

Read and write CSV files

Comma-separated values (CSV) files are useful as data input for data-driven testing/automation or as data output for web scraping. In UI.Vision RPA for Chrome the csvRead command gives you access the data inside CSV files, and the csvSave command allows you to write test results or extracted values to a CSV file.

CSV Manager: Import (load) and export (save) CSV files from the <em>UI.Vision RPA</em> extension

The CSV manager tab (shown in the screenshot) allows you to import, export, view and delete CSV files to the local storage of the extension. So the CSV files are not directly accessed from the hard drive, but they are stored inside the web browser. The reason for this is that modern browser extension have no access to your hard drive. Just like with the macros, all CSV files are stored locally inside your browser (technically called local storage), nothing is uploaded to any kind of cloud service.

If you have the UI.Vision RPA FileAccess XModule installed, you can switch the macro storage mode to Hard-Drive Storage. This will also redirect all CSV read and write operation directly to the hard drive. By default, CSV files are then stored in the "kantu/datasources" folder.

Top

Automating File Upload

Update: UI.Vision RPA can automate file uploads in Chrome and Firefox. You can use XType to automate file uploads.

[This paragraph has another, second option, but it only works in Chrome versions before 72.0.3626.81]: To record file uploads, select the file with the file selection dialog as you normally would. UI.Vision RPA records it. Just note that for security reasons Chrome only returns the file name and not the file path to the extension. Therefore the file path is always recorded as "c:\fakepath". So after recording, you need to edit the macro and replace c:\fakepath with the real path.

Automating File Download

If you all need is a simple download, there is nothing special to do. If a click command triggers a download, UI.Vision RPA handles it automatically. By default, UI.Vision RPA does not stop and wait for the download to complete, the macro continues with the next command(s).

OnDownload | new file name | (wait for download to complete:) true/false

For more control over the download there is the OnDownload command. It allows you to overwrite the default file name with a custom name. And with "true" in the 3rd column you tell UI.Vision RPA to wait for a download to complete before continuing with the next command. UI.Vision RPA waits for max. timeout_download seconds. This option is great for checking the performance (download speed) of a file download. If you need to measure the exact download time, you can do that with the value of the !RUNTIME internal variable. The max. wait time between a mouse click on the link and the actual download start is limited by timeout_wait. OnDownload can be at any place in the macro as long as it is reached before the download is triggered.

Top

Internal variables

Some parameters can be controlled via Internal variables. All internal variables are valid only per macro run and they do not change the global default values. In other words, internal variables overwrite global default values inside the macro they are used.

  • !clipboard (read/write) - write to or read from the computer's clipboard (copy and paste).
  • ${!cmd_var1}, ${!cmd_var2}, ${!cmd_var2} (read) - the value of these three variables can be set via the command line with the cmd_var1/2/3 switches.
  • !col1, !COL2,... (read only) - Values read from CSV file with csvRead
  • !csvLine (read/write) - storing values to this variable appends them to the line used by csvSave. Use store | NULL | !csvLine to overwrite (not append) the current values.
  • !csvReadMaxRow (read) - In some cases you may need to now how many lines are in a CSV file, or you want to read the last (most recently added) line of a CSV file. In both cases the internal variable !csvReadMaxRow can help. After the first csvRead, this value is set the number of lines in the CSV (example here)
  • !csvReadStatus (read) - true if the last csvRead command was successful. Typically false if there are no more lines in the CSV file. So together with the while command, this can be used to read CSV files where the number of rows is unknown. Please see the DemoReadCSVwithWhile macro for a code example.
  • !csvReadLineNumber (read/write) - true sets the line number for the csvRead command to read. If not used, the value of !LOOP is used for the line number and !csvReadLineNumber is set to the same value.
  • !ErrorIgnore (read/write) - This is a very useful internal variable. If set to true with store | true | !errorIgnore the macro execution continues after an error. It is often used with flow control or to test/click links that sometimes exist and sometimes not. You can change back to the normal behavior at any point in the macro with store | false | !errorIgnore
  • !LastCommandOK (read only) - contains the status of the last executed command (true of successful, or false if the command encountered an error. Use with !ErrorIgnore set to true so the macro execution continues after an error. The ECHO command that does not influence the status, so it can be used for logging the value of !LastCommandOK without changing its value.
  • !statusOK (read/write) - contains the status of the macro execution. Once an error happens !statusOK is set to FALSE. Use with !ErrorIgnore set to true so the macro execution continues after an error. Only then !statusOK will be useful. Otherwise the macro stops at an error anyway. !statusOK is similar to !LastCommandOK but it does not get reset by a successful command. Once !statusOK is set to "false" by an error, it remains "false", even if the next commands succeed. But you can use store | true | !StatusOK to manually reset it.
  • !LOOP (read only) – current value of the loop counter (1,2,3… etc). Great for use with @POS=${!LOOP} to cycle through a list of links.
  • !MACRONAME (read) - Name of the macro this command is in
  • !REPLAYSPEED (read/write) - Set the replay speed to FAST (no delay between each step), MEDIUM (short, "human like" delay) or "SLOW" (2 seconds delay)
  • !RUNTIME (read) - Time elapsed since macro start. This value is very useful for performance testing and performance monitoring
  • !TIMEOUT_DOWNLOAD (read/write) - Time limit for file downloads if UI.Vision RPA is told to wait for a download to complete via OnDownload | file name | true
  • !TIMEOUT_MACRO (read/write) - Limit for the overall macro runtime. The default value is 300s (5 minutes).
  • !TIMEOUT_PAGELOAD (read/write) - How long to wait for page load
  • !TIMEOUT_WAIT (read/write) - How long to wait for an element to appear before reporting an element not found error (implicit waiting)
    Implicit waiting with timeout_wait in action
  • !URL (read only) - contains current browser URL that you see in the address bar. Previously, the storeLocation command was needed for this.
  • !WaitForVisible - by default the implicit waiting waits for an element to appear and become visible. If you do not want to wait for the element be visible (typically because it never becomes visible) use store | false | !WaitForVisible. Then waiting for visible is turned off for all following commands until you set it again to true.

Top

Source Code Tab

The source code tab is the useful to view, search, edit and copy & paste source code snippets:

Source Code Tab
You can use the source code tab to copy and paste code snippets to/from the UI.Vision RPA forum. If you paste code in the forum please remember use the code tag in the forum editor.

Jump to Source Code
In the table view you can use the "Jump to source code" context menu item to jump directly to the JSON code of the currently selected command. In other words, this feature opens the source code view tab and automatically scrolls down to the right position.

Performance Monitoring/Testing

The !RUNTIME internal variable contains the macro runtime and makes performance testing easy. In the example below we first click on the "Start OCR button " (OCR.space) and store the current run time in a variable. Then UI.Vision RPA waits for the result (= web element) to appear. Once the element is found and the macro execution continues we measure the time again by reading the current value of !RUNTIME. The difference is the runtime performance of this step. In a next step you can, for example, write the value of "TimeForThisStep" to a CSV file with the csvSave feature.

Performance Monitoring/Testing

The !RUNTIME internal variable contains the macro runtime and makes performance testing easy. In the example below we first click on the "Start OCR button " (OCR.space) and store the current run time in a variable. Then UI.Vision RPA waits for the result (= web element) to appear. Once the element is found and the macro execution continues we measure the time again by reading the current value of !RUNTIME. The difference is the runtime performance of this step. In a next step you can, for example, write the value of "TimeForThisStep" to a CSV file with the csvSave feature.

click link=Start OCR! Start the process to measure (in this case an OCR conversion)
store ${!RUNTIME} StartTime Store the start time
click //*[@id="sucOrErrMessage"]/strong Wait for the "Completed" message to appear...
store ${!RUNTIME} EndTime Log the current macro runtime
storeEval ${!EndTime}-${!StartTime} TimeforThisStep Calculate the runtime of the "yellow" step,
which is the difference between start and end time.
Top

Test status reporting

The background of each macro indicates its test status: White = not yet run, Green = last run was ok and red = last run had an error.

Test status indicator...

Setting the right macro replay speed

In (gear icon) > Replay Settings you can select the replay speed. FAST (no delay), MEDIUM (0.3s, default) and SLOW (2s). With FAST, UI.Vision RPA replays the macro as fast as it can. This is faster than any human could type, so on some websites that can cause problems, as they have not been tested for such a speed and sometimes Javascript code can not respond quickly enough. Typically MEDIUM is a good compromise between speed and being website-friendly. SLOW can be useful for testing and debugging.

You can change the default replay speed in the settings menu (see screenshot above) or inside a macro with store | SLOW/MEDIUM/FAST | !replayspeed.

Top

Position support in link text locator

With CLICK link=Download@POS=3 you instruct the IDE to use the n-th (here n=3) occurrence of a locator. This is useful, for example, if a have a website with many download links, and you want to click the n-th once. POS is only supported with the “normal text” locator link=... but not with XPATH and CSS selectors.

Top

More Example Macros (Test cases)

The UI.Vision RPA Selenium IDE github page contains the /test folder with plenty of test macros. You can download and import all macros/selenium IDE scripts at once. For this, use the "Gear icon => Import HTML" feature and then select all the macros/scripts that you want to import (we call the test cases often macros - this is the same. We use the word macros as some users use the extension not for web testing, but for web automation. There the word macro and macro recorder is very common.)

Import many macros at once! - and do not forget to BACKUP (export) your macros once in while

Top

Tip: Backup your macros

In the default storage mode macros, test suites and all UI.Vision RPA settings are stored inside your web browser in what is called technically HTML5 local storage. Nothing is copied or moved to the cloud. This means that once you uninstall the UI.Vision RPA extension, your macros are gone! You can use the "Export" feature to create backups of your important test cases, CSV data and images. The fastest way to export everything is to use the "Run backup now" button on the backup settings panel. It exports all data at once.

If you need to recover information from a backup, see this forum post on how to restore macros from a backup.

<em>UI.Vision RPA</em> Backup Reminder. Do not forget to BACKUP (export) your macros once in while

Top

Command Line API

Save the macro (= your Selenium IDE test case) in HTML format with the "Save HTML page" export feature. The export feature automatically adds some Javascript helper code to the exported HTML page that enables the autostart.

Create the HTML page for the command line autorun feature

Then load the website in the web browser - and the macro runs automatically! By default, the autorun feature is enabled for local websites and for the bookmark feature (see below), but not for starting macros from normal (internet-based) websites. To skip the "Do you want to run this macro?" question, add the "?direct=1" switch to your local file URL ("file:///...").

Important: Even so we mention the browser here, the command line feature works exactly the same in desktop automation (RPA) mode. The browser and the local website are just needed because the UI.Vision RPA core is technically implemented as browser extension. In other words, UI.Vision RPA can be used as part of a CI pipeline for web automation and for desktop automation.

Command Line Parameters

Note that in a strict technical sense these are not command line parameters, but "GET" parameters that are appended to the file URL of the autorun HTML page(s) you are loading into Chrome. But their purpose is just like that of real command line parameters, so we call them this way.

  • - direct=1 - Skip the "Do you want to run this macro?" question.
  • - closeBrowser=1 - Closes the Chrome browser when the macro is completed. The default is "0", the browser remains open.
  • - closeKantu=0 - Close UI.Vision RPA when the macro is completed. The default is "1", so UI.Vision RPA closes after the macro is done. Screencast of a PowerShell script that uses CloseKantu=1 and CloseBrowser=0
  • - continueInLastUsedTab=1 - If you want to combine a series of command line calls and want to make sure that the next macros continues in the tab where the last macro stopped (the active tab), then use this switch. Otherwise UI.Vision RPA opens a new tab for each new command line call. An example of when "continueInLastUsedTab=1" will be useful is if the first macro does the login, the second macro runs a test and third macro does the log off. Then you want to make sure all three macro work in the same browser tab.
  • - folder=foldername - exactly the same as the macro switch, but it runs all macros inside a folder. This means every folder can be used test suite.
  • - savelog=filename.txt - This saves the log (content of the Log tab) plus a header status header as a text file once the macro has completed. The log is saved whether the run was successful or not. The calling script (e. g. Batch file, Python or PowerShell) can check for the existence of this file to make sure the macro run completed). The first line of the log file is always the status of the macro run (error or success, and the error message, if any).
    A second way to save the log file is to use the command localStorageExport | log. But unlike the command line option, this method does not add a header with status information to the top of the log.
  • - storage=browser/xfile - This parameter tells UI.Vision RPA if the macro or testsuite folder to run can be found in the HTML5 browser storage or on the hard drive.
  • - macro=macroname - ignore the macro that is stored inside the generated HTML page, and run the specified macro. In other words, you only ever need to generate one autorun page and can then use it for all your macros.
  • - testsuite=testsuitename (deprecated) - please use the new folder command.
  • - cmd_var1=hello%21world, cmd_var2, cmd_var3 - Send values to a UI.Vision RPA macro from the command line. Inside the macro you can access the value with the internal variables. ${!cmd_var1}, ${!cmd_var2} and ${!cmd_var3}.
  • - Examples: All the above switches are used in the example PowerShell and Python scripts.

Run Selenium IDE from the command line

Allow UI.Vision RPA access to file URLs: Allow access to file URLs

Example: A Windows batch file to run a Selenium IDE/UI.Vision RPA macro from the command line on looks like this:
"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" "file:///D:/test/DemoAutofill1.html?direct=1&close=1&savelog=log1.txt"

Note that you must use file:/// for the file path. If you would just start with "C:\..." you will get a "file not found" error for the HTML file due to the appended "?direct=1&close=1&savelog=log1.txt" GET parameters. Windows is used as an example here, the command line feature also works on Mac and Linux as well. "demoautofill.html" is the exported UI.Vision RPA macro.

A longer command line, this time with Firefox:
"C:\Program Files\Mozilla Firefox\firefox.exe" "file:///D:/test/DemoAutofill1.html?macro=demoframes&cmd_var1=hello%21world&cmd_var2=123&cmd_var3=ABC&direct=1&savelog=logfirefox.txt"
In this example the content of the export macro is not used, the HTML file is merely needed to launch Firefox, and then the macro specified with macro= is run.

UI.Vision RPA as Automator alternative on Mac

Start <em>UI.Vision RPA</em> macros on Mac with AppleScript

On MacOs you need to use AppleScript to start Chrome/Firefox with the file url. If you would only use the OPEN command, then the GET parameters behind the ? are lost. You can get AppleScript executed from a bash script or function, or from a Terminal by just prefixing it with "osascript". So the correct command line syntax for Mac is
osascript -e 'tell application "Google Chrome" to open location "file:///Users/t/demoautofill.html?direct=1&close=1&savelog=demotestlog.txt"'
Thanks to user Shunt/Timo for the forum post Launching UI.Vision RPA from Command Line on a Mac. We used their feedback to update this section.

How to run UI.Vision RPA macros 24/7

Many applications of UI.Vision RPA require continuous operation of the software. Examples are the use of UI.Vision RPA as part of robotic process automation (RPA), extracting large volumes of information or web testing applications in general.

Problem: By design web browsers are not intended for 24x7 operations and running them repeatedly for several days can lead to undesirable effects, such as increased memory consumption ("memory leaks").

Solution: The UI.Vision RPA command line allows you to control the UI.Vision RPA operation. With the -savelog switch a calling script can easily check on the success of each macro run. The -closebrowser and -closekantu switches allow you close Chrome and Firefox periodically to avoid memory leaks. And you can add code to terminate the Chrome or Firefox instances if they hang (e. g. using proc.kill() in Python or #taskkill /F /IM chrome.exe /T in Powershell).

Examples: Demo scripts for running UI.Vision RPA 24/7 are available in Github for PowerShell and Python.

Important: How to avoid extension auto-updates during unattended operation.

Related: How to run UI.Vision RPA with the Windows Task Scheduler

Top

Running Concurrent Instances

You can easily run concurrent instances of UI.Vision RPA with Chrome and Firefox via the command line. To do so, you need to start each browser instance with its own profile. For Chrome the command line switch is -profile-directory="Profile 2" and for Firefox it is -p PROFILE_NAME.

Embed and run UI.Vision RPA macros in websites

You can embed macros directly into a website! This is a good option if you need to distribute your macros to a larger numbers of users. Since UI.Vision RPA macros are in standard JSON format, it is very easy to create them dynamically e. g. from info inside a database.

DEMO: This web page contains three embedded macros.

In order to run macros that are embedded in a website you have to first allow it . You do this by checking the "Run embedded macros from public websites" box. This is step 1 in the screenshot below. The default setting is OFF (do not run embedded macros). Once you allow such macros in general, you will see a dialog box asking for permission to run such macros. If you want to avoid this warning dialog for certain websites (for example your own internal website), then you can add this website to the website whitelist (step 2 in the screenshot below):

<em>UI.Vision RPA</em> URL Whitelist

The URLs in the white list are the websites that can contain embedded macros, and that you want to run without warning dialog.

Top

Run from bookmarks

In the macro context menu (right-click menu) select "Create bookmark". This adds a shortcut to the macro to your bookmarks. From now on you can just select the bookmark to run the macro. UI.Vision RPA will open to run the macro and then close again. But if the macro encounters an error, UI.Vision RPA stays open so you can see what went wrong. Add web macro shortcut to bookmarks bar

Technically UI.Vision RPA bookmarks are little Javascript snippets (bookmarklets) that start the UI.Vision RPA engine. And thus, like with all bookmarklets, they do not work on the Google Chrome and Firefox "New Tab" start page. For security reasons, the browser do not run any Javascript on this page. But whenever a "normal" web page is loaded, the kantu web imacro bookmarks work great. Bookmarklets need a normal web page loaded to work (= do not work on new tab page)

If you want to make sure Chrome and Firefox are in the foreground while the bookmark macro runs, add the BringBrowsertoForeground command to your macro.

If you want to trigger your macros with keyboard shortcuts you can combine Kantu's bookmarklets with the ShortKeys extensions.

You can change the "close UI.Vision RPA when macro is done" behavior to "keep UI.Vision RPA open when macro is done". To do so, edit the bookmark and change the flag keepKantuOpen: false to keepKantuOpen: true in the bookmarklet's Javascript code. Regardless of this flag, the UI.Vision RPA window stays open if you manually press STOP during the macro run or if the macro stops with an error.

Top

Password Encryption

UI.Vision RPA can store passwords encrypted. You can enable this feature and set a master password in the settings.

Enable password encryption

With the master password enabled, every website password that you enter during macro recording is not stored in plain text, but instead as an encrypted string. So instead of having a command like
"TYPE | id=password | your-password-here"
the recorded command is
"TYPE | id=password | __KANTU_ENCRYPTED__f8b53105ebb186d...".

Record encrypted passwords

UI.Vision RPA uses 256-bit AES encryption which is considered unbreakable by security experts. Note that if you change the master password later, you need to re-record the password field, so that a new "__KANTU_ENCRYPTED__..." string is created.

Top

Test Suites V2.0 (Folders as Testsuites)

Since UI.Vision RPA V5.0 test suites are created on the macros tab, and not the test suites tab. The new concept is straightforward: Every folder is a test suite. To play a test suite, right-click on a folder and select "Test suite: Play all macros in folder". To start the replay at a certain macro, right-click on the macro and select "Replay from here".

You can add or remove macros from a testsuite simply by adding or removing it from a folder. Macros in subfolders are ignored. The replay order is alphabetical, just like the display order.

To start a testsuite from the command line use the folder switch. This runs all macros in the folder.

New test suites
Every folder is now also a test suite. Just drag and drop macros in a folder to build a test suite.

Test Suites (Old-Style, Deprecated)

The old-style Test suites are created on the test suites tab (deprecated) by selecting the test cases (macros) from the drop-down. Click the "+Macro" button to add a new macro to the list. You can also specify how often each macro runs (loops). Test suites itself do not contain macro code, they contain the names of existing macros.

Old-style test suites
Old-style testsuites are still available for backward compatibility.

Once the test suite is completed, a test report summary is written to the log area. The success of each test case is also indicated visually.

To rearrange the order of the macros in the test suite and for other larger changes, we recommend to edit the JSON source directly. To edit it, either click the "hamburger icon" ☰ displayed to the right of the test suite name, or right-click the test suite name and then select "Edit source".
Edit the test suite JSON source
(DEPRECATED OLD-STYLE TESTSUITE) Once you save the edited source code, UI.Vision RPA runs a checks against all available macros to make sure there are no missing macros or typos.

Global variables: By default, each macro runs independent, no variables are shared. But you can create global variables that are available to every macro and test suite simply by prefixing them with "Global..." (e. g. "globalMyCounter" or "GLOBAL_username"). In other words, variables whose name starts with "Global..." are shared . Global variables keep their value even after the macro has been stopped. Their content is only lost once UI.Vision RPA is closed.

If you want your data to survive a closing of Kantu, then store ("persist") it in a CSV file with csvSave and csvRead:
Use global variables or CSV files to persist data

Top

Security and Privacy

The UI Vision RPA core is open-source and guarantees Enterprise-Grade Security. Your data never leaves your machine.

For developers

The source code of the UI.Vision RPA core is available on GitHub (License: GNU).

For the next update(s): Beta Testers wanted!

Join our <em>UI.Vision RPA</em> Beta Tester List!

We run our own QA, but our test automation can never cover the amazing variety of UI.Vision RPA use cases and ideas that UI.Vision RPA users have come up with. This beta program helps us to make sure that a new release does not inadvertently break existing macros. So please consider joining our beta version announcement list.

Top
Subscribe to the UI Vision RPA software newsletter . We'll send you updates on new releases that we're working on.