The uiv.* API: JS Script Macros Overview

Modern JavaScript Macros — API Overview

A Ui.Vision macro can be a modern JavaScript program: instead of a table of commands, you write real code against the uiv.* API. Everything JavaScript offers — loops, conditions, functions, try/catch, template literals — combines with everything Ui.Vision offers: DOM automation, computer vision, OCR, AI, desktop input. This page is the API overview. Coming from classic table macros? See the Selenium-derived commands vs uiv.* conversion table.

uiv.open('https://en.wikipedia.org');
uiv.page.type('id=searchInput', 'Solar cell');
uiv.browser.type('${KEY_ENTER}', {nav: true});
const h1 = uiv.$('css=#firstHeading');   // auto-waits, throws if it never appears
uiv.log(`Landed on: ${h1.text}`, 'green');

Two ideas carry the whole API: finders answer "where is it?" and return a match {x, y, rect, text, value, ...}; input tiers act on a match (or locator) and differ in how the input reaches the page. Every finder auto-waits up to !TIMEOUT_WAIT and then throws — finding an element IS the wait.

Finders — Where Is It?

Method	What it does	Example
`uiv.$(locator)`	First DOM match. Finds in ALL frames (even cross-origin) and open shadow roots — no selectFrame needed. Locators: css= id= name= link= xpath=	`uiv.$('css=#buy')`
`uiv.$$(locator)`	All DOM matches, as an array	`uiv.$$('css=tr').length`
`uiv.findImage(png, opts)`	First computer-vision match of an image on the visible page. Options: {minScore, area, scope}	`uiv.findImage('buy.png', {minScore: 0.8})`
`uiv.ocr.findText(text)`	First OCR match of rendered text — works on canvas, PDFs, images. ? and * wildcards per word	`uiv.ocr.findText('Check*ut')`
`uiv.ai.find(question)`	The configured AI locates the element from a screenshot and returns a match	`uiv.ai.find('the blue Buy button')`
`uiv.offset(match, dx, dy)`	A new match at a fixed offset from another — the composed form of the classic relative targets	`uiv.offset(uiv.ocr.findText('Email:'), 120, 0)`
`uiv.findElements / uiv.findImages / uiv.ocr.findTexts`	Long forms returning every match, with options like {required: false, timeout: 2} for optional elements (null/[] instead of throwing)	`uiv.findImages('row.png', {area: uiv.$('css=#list')})`

Acting — Three Input Tiers

Every input call names its tier, because how the input reaches the page decides whether it works. A visual click is always explicit: uiv.browser.click(uiv.findImage('buy.png')).

	uiv.page.*	uiv.browser.*	uiv.desktop.*
Methods	`click, type(locator, text), select`	`click, type(text), move, down, up`	`click, type(text), move`
Event type	Synthetic DOM events (like Selenium)	Trusted browser input via the debugger API (CDP)	Real OS input (XModule)
XModule needed	No	No	Yes
Coordinates	Viewport CSS pixels	Viewport CSS pixels	Screen pixels ({scope: 'desktop'} finders)
Browser in background OK	Yes	Yes	No — real mouse moves
Best for	Form filling, fast bulk work	Canvas apps, drag & drop, sites that ignore synthetic clicks	OS dialogs, other applications, desktop automation
Browsers	All	Chrome, Edge (not Firefox)	All (XModule: Windows, Mac, Linux)

Dragging is press, move, release: uiv.browser.down(start) holds the button, every uiv.browser.move while it is held drags, uiv.browser.up(end) releases — sliders, drag handles and canvas drawing all work this way. Keystrokes go to the focused element and support the ${KEY_...} codes: uiv.browser.type('${KEY_ENTER}', {nav: true}) submits and waits for the navigation it causes.

Page, Reading & OCR

Method	What it does	Example
`uiv.open(url)`	Navigate and wait for the page load	`uiv.open('https://ui.vision')`
`uiv.eval(code)`	Run JavaScript inside the website and return the result (the code must use return)	`uiv.eval('return document.title')`
`match.text / match.value`	Every DOM match carries its text and value — no separate store commands	`uiv.$('css=h1').text`
`uiv.ocr.read(opts)`	OCR the viewport, a region ({area}), a saved screenshot ({image}) or the desktop ({scope})	`uiv.ocr.read({area: myMatch})`

AI, Tabs, Data & Files

Method	What it does	Example
`uiv.ai.ask(prompt, opts)`	One round trip to the configured LLM; {images: [...]} attaches screenshots, {json: true} returns parsed JSON	`uiv.ai.ask('total?', {images: [uiv.shot.viewport()], json: true})`
`uiv.ai.computerUse(task)`	Hands the whole task to the AI computer-use agent; returns its report	`uiv.ai.computerUse('fill in this form')`
`uiv.tabs.select / open / close / list`	Tab control with absolute indexes (1..N); every call returns {index, title, url}	`uiv.tabs.select(2)`
`uiv.csv.read / write / append / list / exists`	CSV files as plain 2D arrays — no !csvLine, no line-number bookkeeping	`uiv.csv.append('log.csv', [ts, value])`
`uiv.download(source, opts)`	Download via link locator, URL or trigger function; renames ({as}), waits for completion, returns the on-disk name	`uiv.download('css=a.pdf', {as: 'report.pdf'})`
`uiv.shot.viewport / page / element / desktop / area`	Screenshots (page = whole page, scroll-stitched); each returns the file name, so shots pipe into OCR or AI	`uiv.ocr.read({image: uiv.shot.page()})`
`uiv.exportToDownloads(name)`	Copy a .png, .csv or 'log' from Ui.Vision storage to the browser's Downloads folder	`uiv.exportToDownloads('report.csv')`

Variables, Logging & Utilities

Method	What it does	Example
`uiv.getVar / uiv.setVar`	Read/write the same variable pool the classic commands use — internal !-variables included	`uiv.setVar('!TIMEOUT_WAIT', 20)`
`uiv.log(text, color)`	Write to the log (green/red/blue/..., '#shownotification' shows a browser notification)	`uiv.log('done', 'green')`
`uiv.banner(html, opts)`	On-page overlay message for the person watching — survives navigation, click-through, great for attended runs and human hand-offs	`uiv.banner('Your turn: solve the captcha')`
`uiv.sleep(ms)`	Fixed wait — last resort; finders auto-wait, so most macros need none	`uiv.sleep('2s')`
`uiv.run(command, target, value)`	The legacy bridge: run any classic command from inside a script	`uiv.run('setProxy', ...)`
`// @include` + `uiv.main`	Include another script file as a library; uiv.main is true only in the file that was started	`// @include Core/Sub/MyLib.js`

Error Handling Is Just JavaScript

A failed uiv call throws a real exception. try/catch handles retries and fallbacks, throw new Error('...') fails the macro deliberately, and optional elements are a {required: false} option away — no !errorignore, no !statusOK bookkeeping. A script should prove its own success: end it with a finder on something unique to the goal state, or compare a read value and throw on mismatch.

Where To Go Next

Selenium-derived commands vs uiv.* — the conversion table for everyone coming from classic table macros. Or just try it: the AI sidebar writes uiv.* scripts for you — and converts your existing classic macros on request. The preinstalled "Draw a cat🐱" macro is a fun working example of finders, trusted input and self-verification.

Anything wrong or missing on this page? Suggestions?

...then please contact us.

The uiv.* API: JS Script Macros