Sign in

v1

latestOpenAPI 3.0.32026-07-267015229.9 KB
Documents

Recrawl Document

Trigger a recrawl of a website document to fetch fresh content.

post/v2/documents/{document_id}/recrawl

Path parameters

document_idstring required

Unique id of the crawl-backed website document to refresh. Use when the source site changed, you want to refresh content on a schedule, or retry after crawl or processing errors.

The document must be in ready or error (otherwise 409). It must have been created with a crawl configuration unless you supply crawl in the request body for this call.

The same document cannot be recrawled more than once within each 1-hour cooldown (429 if invoked too soon).

Request body

Example request

{
  "crawl": {
    "depth": 2,
    "max_pages": 10
  }
}

Response

Recrawl initiated successfully

document_idstring

Unique identifier for the document

document_namestring

Name of the document

document_urlstring

URL of the document

status'started' | 'processing' | 'ready' | 'error' | 'recrawling'

After a successful recrawl request, typically recrawling until processing completes, then ready or error. Other values: started, processing.

progressinteger nullable

Processing progress as a percentage (0-100). Null when processing has not started or is complete.

error_messagestring nullable

Error code indicating why processing failed. Only present when status is error. Possible values include: file_download_failed, file_format_unsupported, file_size_too_large, file_empty, invalid_file_url, document_processing_failed, website_processing_failed, chunking_failed, embedding_failed, vector_store_failed, contact_support.

created_atstring

ISO 8601 timestamp of when the document was created

updated_atstring

ISO 8601 timestamp of when the document was last updated

callback_urlstring

If set on Create Document, Tavus POSTs status updates here while this recrawl runs through completion.

tagsstring[]

Array of document tags

crawled_urlsstring[] nullable

List of URLs from the previous crawl (will be updated when recrawl completes)

last_crawled_atstring nullable

ISO 8601 timestamp of the previous crawl

crawl_countinteger

Number of times the document has been crawled (will increment when recrawl completes)

Example response

{
  "document_id": "d8-5c71baca86fc",
  "document_name": "Company Website",
  "document_url": "https://example.com/",
  "status": "recrawling",
  "created_at": "2024-01-01T12:00:00Z",
  "updated_at": "2024-01-15T10:30:00Z",
  "callback_url": "https://your-server.com/webhook",
  "tags": [
    "website",
    "company"
  ],
  "crawl_config": {
    "depth": 2,
    "max_pages": 10
  },
  "crawled_urls": [
    "https://docs.example.com/",
    "https://docs.example.com/getting-started"
  ],
  "last_crawled_at": "2024-01-01T12:05:00Z",
  "crawl_count": 1
}