UI Action
Detect UI elements
Detect and identify interactive UI elements in the current screen. Note: This feature currently only supports element detection within a running browser. If the browser is not running, the Elements array will be empty.
post/boxes/{boxId}/actions/elements/detect
Path parameters
boxIdstring required
Example:c9bdc193-b54b-4ddb-a035-5ac0c598d32d
Box ID
Request body
Example request
{
"screenshot": {
"outputFormat": "base64",
"presignedExpiresIn": "30m"
}
}Response
Example response
{
"screenshot": {
"source": {
"uri": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA...",
"presignedUrl": "https://example.com/xxxxx/xxxxx/xxxxx"
},
"marked": {
"uri": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA...",
"presignedUrl": "https://example.com/xxxxx/xxxxx/xxxxx"
}
},
"elements": [
{
"id": "1",
"source": "chromium",
"type": "button",
"path": "#root > table > tbody > tr:nth-child(1) > td:nth-child(1) > div > button",
"width": 100,
"height": 50,
"x": 100,
"y": 100,
"centerX": 150,
"centerY": 125,
"label": "Click me"
}
]
}