Sign in

v16

latestOpenAPI 3.1.0raw.githubusercontent.com2026-02-22152741.9 KB
downloading

Download Bulk

Bulk download multiple files with checkpointing

Requires permission: download:write

post/download/bulk

Request body

urlsstring[] required

List of URLs to download

crawl_idstring required

Unique identifier for this crawl

downloader_type'FLEET_ASYNC'

Available downloader types

max_workersinteger

Maximum concurrent workers

batch_sizeinteger

URLs per batch

wait_until'domcontentloaded' | 'load' | 'networkidle' | 'commit'

When to consider page load complete for web scraping operations

debugboolean

Enable debug information

Example request

{
  "crawl_id": "download-123",
  "downloader_type": "FLEET_ASYNC",
  "max_workers": 8,
  "urls": [
    "https://example.com/file1.pdf",
    "https://example.com/file2.pdf"
  ]
}

Response

Successful Response

crawl_idstring required
total_urlsinteger required
total_downloads_attemptedinteger required
success_countinteger required
failed_countinteger required
cached_countinteger required
save_failed_countinteger required
timestampstring required
debug_frameobject[] nullable