Sign in

v1

latestOpenAPI 3.0.0Proprietary2026-08-062711471.0 MB
Knowledge Bases

Add a website crawl source

Starts a website crawl that discovers and indexes pages under the given URL. Requires a USER API key.

post/public/v1/knowledge-bases/{knowledgeBaseId}/website-crawls

Path parameters

knowledgeBaseIdinteger required
Example:12

Request body

descriptionstring
sourceUrlstring uri required
websiteType'WEBSITE_CRAWL' | 'SITEMAP' required
maxDepthinteger
maxLinksinteger required
includePathsstring[]
excludePathsstring[]
autoSyncFrequency'NONE' | 'DAILY' | 'WEEKLY' | 'MONTHLY'

Example request

{
  "description": "example",
  "sourceUrl": "https://api.example.com/contacts/123",
  "websiteType": "WEBSITE_CRAWL",
  "maxDepth": 1,
  "maxLinks": 1,
  "includePaths": [
    "example"
  ],
  "excludePaths": [
    "example"
  ],
  "scrapeOptions": {
    "includeSelectors": [
      "example"
    ],
    "excludeSelectors": [
      "example"
    ]
  },
  "autoSyncFrequency": "NONE"
}

Response

Created website crawl source

idnumber required
namestring required
descriptionstring
sourceUrlstring uri required
websiteType'WEBSITE_CRAWL' | 'SITEMAP' required
maxDepthinteger
maxLinksinteger required
includePathsstring[]
excludePathsstring[]
autoSyncFrequency'NONE' | 'DAILY' | 'WEEKLY' | 'MONTHLY'

Example response

{
  "id": 1,
  "name": "example",
  "description": "example",
  "sourceUrl": "https://api.example.com/contacts/123",
  "websiteType": "WEBSITE_CRAWL",
  "maxDepth": 1,
  "maxLinks": 1,
  "includePaths": [
    "example"
  ],
  "excludePaths": [
    "example"
  ],
  "scrapeOptions": {
    "includeSelectors": [
      "example"
    ],
    "excludeSelectors": [
      "example"
    ]
  },
  "autoSyncFrequency": "NONE"
}