Sign in

v16

latestOpenAPI 3.1.0raw.githubusercontent.com2026-02-22152741.9 KB
crawling

Create Crawl Graph

Create a crawl graph by recursively crawling from a root URL

This endpoint crawls a website starting from the given URL up to the specified depth, extracting links and building a graph structure. Results are checkpointed.

Requires permission: crawl:write

post/crawl/graph

Request body

urlstring uri required

Root URL to start crawling from

enginesEngineType[] required

List of engines to use

crawl_idstring required

Unique identifier for this crawl

depthinteger nullable

Maximum crawl depth

absolute_onlyboolean

Only extract absolute URLs

keep_externalboolean

Keep external URLs in results

visit_externalboolean

Visit external URLs

max_workersinteger

Maximum concurrent workers

batch_sizeinteger

URLs per batch

max_urlsinteger

Maximum URLs to crawl

debugboolean

Enable debug information

Response

Successful Response

crawl_idstring required
root_urlstring required
total_pages_visitedinteger required
visited_urlsstring[] required
depth_reachedinteger required
max_depthinteger required
max_urlsinteger required
absolute_onlyboolean required
keep_externalboolean required
visit_externalboolean required
max_workersinteger required
batch_sizeinteger required
timestampstring required
debug_frameobject[] nullable