v1

latestOpenAPI 3.1.02026-07-137321.6 KB
Text

Page text by URL

Returns the visible text content of a webpage specified by the URL. Can be used to feed data to LLM models. The response can be in plain text, JSON, or XML format based on the text_format parameter. Proxies and Chromium JavaScript rendering are used for page retrieval and processing. Returns JSON on error.

get/text

Query parameters

text_format'plain' | 'xml' | 'json'

Format of the text response (plain by default). "plain" will return only the page body text. "json" and "xml" will return a json/xml with "title", "description" and "content" keys.

return_linksboolean

[Works only with text_format=json] Return links from the page body text (false by default). Useful for building web crawlers.

urlstring required

URL of the target page.

headersobject

HTTP headers to pass to the target page. Can be specified either via a nested query parameter (...&headers[One]=value1&headers=[Another]=value2) or as a JSON encoded object (...&headers={"One": "value1", "Another": "value2"}).

timeoutinteger

Maximum web page retrieval time in ms. Increase it in case of timeout errors (10000 by default, maximum is 30000).

jsboolean

Execute on-page JavaScript using a headless browser (true by default).

js_timeoutinteger

Maximum JavaScript rendering time in ms. Increase it in case if you see a loading indicator instead of data on the target page.

wait_forstring

CSS selector to wait for before returning the page content. Useful for pages with dynamic content loading. Overrides js_timeout.

proxy'datacenter' | 'residential' | 'stealth'

Type of proxy. Use residential if your site restricts traffic from datacenters, or stealth for the most heavily protected sites with advanced anti-bot detection (datacenter by default). Residential and stealth proxy requests are more expensive than datacenter, see the pricing page for details.

country'us' | 'gb' | 'de' | 'it' | 'fr' | 'ca' | 'es' | 'ru' | 'jp' | 'kr' | 'in' | 'hk' | 'tr'

Country of the proxy to use (US by default).

custom_proxystring

Your own proxy URL to use instead of our built-in proxy pool in "http://user:password@host:port" format (<a target="_blank" href="https://webscraping.ai/proxies/smartproxy">Smartproxy</a> for example).

device'desktop' | 'mobile' | 'tablet'

Type of device emulation.

error_on_404boolean

Return error on 404 HTTP status on the target page (false by default).

error_on_redirectboolean

Return error on redirect on the target page (false by default).

js_scriptstring

Custom JavaScript code to execute on the target page.

Response

Success

string required