Sign in

v1

latestOpenAPI 3.0.02026-07-133020.5 KB

/scrape: non-JS scraping

Use high-performance web scraping endpoint with Chrome browser TLS fingerprint, but without JavaScript execution and real browser overhead.

post/scrape

Request body

urlstring required

URL to scrape

headersstring[]

Custom headers to send with the request. By default, regular Chrome browser headers are sent to the target URL.

retryNuminteger

Amount of attempts.

geostring

Geo location for basic proxy pools (you can purchase premium ScrapeNinja proxies for wider country selection and higher proxy quality). Read more about ScrapeNinja proxy setup

proxystring

Premium or your own proxy URL (overrides geo field). Read more about ScrapeNinja proxy setup

followRedirectsinteger

Follow redirects.

timeoutinteger

Timeout per attempt, in seconds. Each retry will take [timeout] number of seconds.

textNotExpectedstring[]

Text which will trigger a retry from another proxy address.

statusNotExpectedinteger[]

HTTP response statuses which will trigger a retry from another proxy address.

extractorstring

Custom JS function to extract JSON values from scraped HTML. Write&test your own extractor on https://scrapeninja.net/cheerio-sandbox/

Example request

{
  "url": "https://example.com",
  "headers": [
    "X-Header: some-random-header"
  ],
  "retryNum": 1,
  "geo": "eu",
  "proxy": "http://user:pw@host:port",
  "timeout": 8,
  "textNotExpected": [
    "random-captcha-text-which-might-appear"
  ],
  "statusNotExpected": [
    403,
    429
  ],
  "extractor": "function(input, cheerio) {\n    let $ = cheerio.load(input);\n return $('h1.title').text(); } "
}

Response

OK

bodystring

HTML body of the rendered page.

Example response

{
  "info": {
    "statusCode": 200,
    "finalUrl": "https://example.com/url",
    "headers": [
      "content-type: xxx",
      "header2: val2"
    ]
  },
  "body": "<html><body><h1>Hello World!</h1></body></html>"
}