v48

latestOpenAPI 3.0.0raw.githubusercontent.com2025-12-0297180624.8 KB

UI Action

Get screen layout

Get the current structured screen layout information. This endpoint returns detailed structural information about the UI elements currently displayed on the screen, which can be used for UI automation, element analysis, and accessibility purposes. The format varies by box type: Android boxes return XML format with detailed UI hierarchy information including element bounds, text content, resource IDs, and properties, while other box types may return different structured formats.

get/boxes/{boxId}/actions/screen-layout

Path parameters

boxIdstring required

Example:c9bdc193-b54b-4ddb-a035-5ac0c598d32d

Box ID

Response

Structured screen layout information (XML format for Android boxes)

contentstring required

Screen layout content.

Android boxes (XML):

<?xml version='1.0' encoding='UTF-8' standalone='yes'?>
<hierarchy rotation="0">
  <node ... />
</hierarchy>

Browser (Linux) boxes (HTML):

<html>
  <head><title>Example</title></head>
  <body>
    <h1>Hello World</h1>
  </body>
</html>

Example response

{
  "content": "<?xml version='1.0' encoding='UTF-8' standalone='yes' ?>\n<hierarchy rotation=\"0\">\n  <node index=\"0\" text=\"\" resource-id=\"\" class=\"android.widget.FrameLayout\" package=\"com.android.systemui\" content-desc=\"\" checkable=\"false\" checked=\"false\" clickable=\"false\" enabled=\"true\" focusable=\"false\" focused=\"false\" scrollable=\"false\" long-clickable=\"false\" password=\"false\" selected=\"false\" bounds=\"[0,0][1080,2316]\" displayed=\"true\">\n    <!-- UI elements -->\n  </node>\n</hierarchy>"
}