v1

latestOpenAPI 3.1.02026-07-137321.6 KB
AI

Extract structured data fields from a web page

Returns structured data fields extracted from the webpage using an LLM model. Proxies and Chromium JavaScript rendering are used for page retrieval and processing.

get/ai/fields

Query parameters

urlstring required

URL of the target page.

fieldsobject required

Object describing fields to extract from the page and their descriptions

headersobject

HTTP headers to pass to the target page. Can be specified either via a nested query parameter (...&headers[One]=value1&headers=[Another]=value2) or as a JSON encoded object (...&headers={"One": "value1", "Another": "value2"}).

timeoutinteger

Maximum web page retrieval time in ms. Increase it in case of timeout errors (10000 by default, maximum is 30000).

jsboolean

Execute on-page JavaScript using a headless browser (true by default).

js_timeoutinteger

Maximum JavaScript rendering time in ms. Increase it in case if you see a loading indicator instead of data on the target page.

wait_forstring

CSS selector to wait for before returning the page content. Useful for pages with dynamic content loading. Overrides js_timeout.

proxy'datacenter' | 'residential' | 'stealth'

Type of proxy. Use residential if your site restricts traffic from datacenters, or stealth for the most heavily protected sites with advanced anti-bot detection (datacenter by default). Residential and stealth proxy requests are more expensive than datacenter, see the pricing page for details.

country'us' | 'gb' | 'de' | 'it' | 'fr' | 'ca' | 'es' | 'ru' | 'jp' | 'kr' | 'in' | 'hk' | 'tr'

Country of the proxy to use (US by default).

custom_proxystring

Your own proxy URL to use instead of our built-in proxy pool in "http://user:password@host:port" format (<a target="_blank" href="https://webscraping.ai/proxies/smartproxy">Smartproxy</a> for example).

device'desktop' | 'mobile' | 'tablet'

Type of device emulation.

error_on_404boolean

Return error on 404 HTTP status on the target page (false by default).

error_on_redirectboolean

Return error on redirect on the target page (false by default).

js_scriptstring

Custom JavaScript code to execute on the target page.

Response

Success

object required