---
title: "Unlimited crawl"
method: POST
path: "/unlimited/crawl"
---

# Unlimited crawl

`POST /unlimited/crawl`

Requires a valid API key for an account with an active Unlimited subscription. Billing is based on purchased concurrent requests, not per-request credits. AI and LLM extraction parameters are not supported on this route.

## Request body

- RequestParams
  - `url` string
  - `proxy_enabled` boolean
  - `smart_mode` boolean
  - `webhook` Webhook
    - `destination` string
    - `on_credits_depleted` boolean
    - `on_credits_half_depleted` boolean
    - `on_website_status` boolean
    - `on_find` boolean
    - `on_find_metadata` boolean
  - `authorization` string
  - `cookies` string
  - `refresh_token` string
  - `hard_limit` integer
  - `cache` union — Use HTTP caching. Defaults to true. Standard routes skip browser when cached; AI routes always use browser.
    - boolean
    - object
      - `maxAge` integer — Freshness window in ms (default: 172800000 = 2 days)
      - `allowStale` boolean
      - `skipBrowser` boolean — Skip browser if cached HTML exists (default: true for standard routes, false for AI routes)
      - `period` string — RFC3339 timestamp cutoff
  - `budget` object
  - `store_data` boolean
  - `use_chrome` boolean
  - `root_selector` string
  - `exclude_selector` string
  - `run_in_background` boolean
  - `return_format` union
    - string
    - ReturnFormat[]
  - `request` 'http' | 'chrome' | 'SmartMode'
  - `depth` integer
  - `encoding` string
  - `locale` string
  - `request_timeout` integer
  - `delay` integer
  - `stealth` boolean
  - `headers` object
  - `viewport` object
  - `device` 'mobile' | 'tablet' | 'desktop'
  - `readability` boolean
  - `subdomains` boolean
  - `tld` boolean
  - `user_agent` string
  - `blacklist` string[]
  - `whitelist` string[]
  - `full_resources` boolean
  - `metadata` boolean
  - `fingerprint` boolean
  - `gpt_config` GPTConfigs
    - `prompt` string
    - `max_tokens` integer
    - `temperature` number
    - `user` string
    - `top_p` number
    - `prompt_url_map` object
    - `extra_ai_data` boolean
    - `paths_map` boolean
    - `screenshot` boolean
    - `api_key` string
    - `cache` object
    - `json_schema` object
  - `anti_bot` boolean
  - `respect_robots` boolean
  - `chunking_alg` 'No' | 'ByWords' | 'ByLines' | 'ByCharacterLength' | 'BySentence'
  - `skip_config_checks` boolean
  - `text` string
  - `website_limit` integer
  - `sitemap` boolean
  - `sitemap_path` string
  - `country_code` string
  - `wait_for` object
  - `execution_scripts` object
  - `disable_intercept` boolean
  - `disable_first_party_stylesheets` boolean — Disable the first-party stylesheet allow when `block_stylesheets` is on. Default `false` — first-party CSS still passes so SPAs hydrate. Set `true` for strict block-all-CSS bandwidth-minimal crawls.
  - `disable_first_party_javascript` boolean — Disable the first-party JavaScript allow through downstream blockers (intercept manager / adblock / blocklists). Default `false`. Set `true` to enforce blocklists strictly even on first-party scripts.
  - `disable_first_party_visuals` boolean — Disable the first-party visual allow when `block_visuals` is on. Default `false`. Set `true` for strictly bandwidth-minimal crawls that drop ALL visuals regardless of origin.
  - `external_domains` string[]
  - `css_extraction_map` object
  - `scroll` integer
  - `automation_scripts` object
  - `return_headers` boolean
  - `return_cookies` boolean
  - `return_page_links` boolean
  - `return_json_data` boolean
  - `return_embeddings` boolean
  - `virtual_display` boolean
  - `redirect_policy` 'Loose' | 'Strict'
  - `preserve_host` boolean
  - `filter_output_images` boolean
  - `filter_output_svg` boolean
  - `filter_output_main_only` boolean
  - `remote_proxy` string
  - `concurrency_limit` integer

## Response `200`

Successful crawl initiation (streaming supported)

- CrawlSuccessResponse
  - `result` string

## Other responses

- `400` — AI and LLM extraction parameters are not supported on Unlimited routes.
- `403` — An active Unlimited subscription is required for this route.
- `429` — All purchased Unlimited concurrency seats are active. The request was not queued; retry after the Retry-After interval.

---

[API](https://skmtc.net/spider/apis/spider-cloud-api.md) · [All operations](https://skmtc.net/spider/apis/spider-cloud-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/spider/spider-cloud-api/versions/278ac7e59f23/schema)