v1
latestOpenAPI 3.0.02026-07-263039205.4 KBRetrieve Crawl Job Data
Download the extracted results of a crawl job
get/crawl/data
Query parameters
namestring required
Name of the crawl job whose data you wish to download.
formatstring
Request format=csv to download the extracted data in CSV format (default: json). Note that CSV files will only contain top-level fields.
typestring
Request type=urls to retrieve the URL Report (CSV).
numstring
Pass an integer value (e.g. num=100) to request a subset of URLs, most recently processed first.
Response
200
Example response
[
{
"sentiment": -0.235,
"theCookie": "Global Rule cookie",
"humanLanguage": "en",
"diffbotUri": "article|3|1313073484",
"siteName": "example.com",
"pageUrl": "https://example.com/",
"html": "<p>This domain is for use in illustrative examples in documents. You may use this domain in literature without prior coordination or asking for permission.</p>\n<p><a href=\"https://www.iana.org/domains/example\">More information...</a></p>",
"text": "This domain is for use in illustrative examples in documents. You may use this domain in literature without prior coordination or asking for permission.\nMore information...",
"type": "article",
"title": "Example Domain",
"docId": 73824068375,
"fromSeedUrl": "https://example.com",
"seedUrlHash32": 3608703573,
"parentUrlDocId": 271353324342,
"gburl": "<REDACTED>",
"lastCrawlTimeUTC": 1726165563,
"timestamp": "Thu, 12 Sep 2024 18:26:03 GMT"
}
]