---
title: "Initiate a web crawl from a base URL"
method: POST
path: "/crawl"
---

# Initiate a web crawl from a base URL

`POST /crawl`

Tavily Crawl is a graph-based website traversal tool that can explore hundreds of paths in parallel with built-in extraction and intelligent discovery.

## Request body

- object
  - `url` string, required — The root URL to begin the crawl.
  - `instructions` string — Natural language instructions for the crawler. When specified, the mapping cost increases to 2 API credits per 10 successful pages instead of 1 API credit per 10 pages.
  - `chunks_per_source` integer — Chunks are short content snippets (maximum 500 characters each) pulled directly from the source. Use `chunks_per_source` to define the maximum number of relevant chunks returned per source and to control the `raw_content` length. Chunks will appear in the `raw_content` field as: `<chunk 1> [...] <chunk 2> [...] <chunk 3>`. Available only when `instructions` are provided. Must be between 1 and 5.
  - `max_depth` integer — Max depth of the crawl. Defines how far from the base URL the crawler can explore.
  - `max_breadth` integer — Max number of links to follow per level of the tree (i.e., per page).
  - `limit` integer — Total number of links the crawler will process before stopping.
  - `select_paths` string[] — Regex patterns to select only URLs with specific path patterns (e.g., `/docs/.*`, `/api/v1.*`).
  - `select_domains` string[] — Regex patterns to select crawling to specific domains or subdomains (e.g., `^docs\.example\.com$`).
  - `exclude_paths` string[] — Regex patterns to exclude URLs with specific path patterns (e.g., `/private/.*`, `/admin/.*`).
  - `exclude_domains` string[] — Regex patterns to exclude specific domains or subdomains from crawling (e.g., `^private\.example\.com$`).
  - `allow_external` boolean — Whether to include external domain links in the final results list.
  - `include_images` boolean — Whether to include images in the crawl results.
  - `extract_depth` 'basic' | 'advanced' — Advanced extraction retrieves more data, including tables and embedded content, with higher success but may increase latency. `basic` extraction costs 1 credit per 5 successful extractions, while `advanced` extraction costs 2 credits per 5 successful extractions.
  - `format` 'markdown' | 'text' — The format of the extracted web page content. `markdown` returns content in markdown format. `text` returns plain text and may increase latency.
  - `include_favicon` boolean — Whether to include the favicon URL for each result.
  - `timeout` number, float — Maximum time in seconds to wait for the crawl operation before timing out. Must be between 10 and 150 seconds.
  - `include_usage` boolean — Whether to include credit usage information in the response. `NOTE:`The value may be 0 if the total use of /extract and /map have not yet reached minimum requirements. See our [Credits & Pricing documentation](https://docs.tavily.com/documentation/api-credits) for details.

## Response `200`

Crawl results returned successfully

- object
  - `base_url` string — The base URL that was crawled.
  - `results` object[] — A list of extracted content from the crawled URLs.
    - `url` string — The URL that was crawled.
    - `raw_content` string — The full content extracted from the page. When `query` is provided, contains the top-ranked chunks joined by `[...]` separator.
    - `favicon` string — The favicon URL for the result.
  - `response_time` number, float — Time in seconds it took to complete the request.
  - `usage` object — Credit usage details for the request.
  - `request_id` string — A unique request identifier you can share with customer support to help resolve issues with specific requests.

## Other responses

- `400` — Bad Request - Your request is invalid.
- `401` — Unauthorized - Your API key is wrong or missing.
- `403` — Forbidden - URL is not supported.
- `429` — Too many requests - Rate limit exceeded
- `432` — Key limit or Plan Limit exceeded
- `433` — PayGo limit exceeded
- `500` — Internal Server Error - We had a problem with our server.

---

[API](https://skmtc.net/tavily/apis/tavily-search-and-extract-api.md) · [All operations](https://skmtc.net/tavily/apis/tavily-search-and-extract-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/tavily/tavily-search-and-extract-api/versions/9126e7d2df75/schema)