---
title: "crawl_website"
method: POST
path: "/v1/ingest/documents/website"
tags: ["Documents"]
---

# crawl_website

`POST /v1/ingest/documents/website`

Upload the content of a publicly accessible website for ingestion into a GroundX bucket. This is done by following links within a specified URL, recursively, up to a specified depth or number of pages.

Interact with the "Request Body" below to explore the arguments of this function. Enter your GroundX API key to send a request directly from this web page. Select your language of choice to structure a code snippet based on your specified arguments.

## Request body

- WebsiteCrawlRequest
  - `websites` object[], required
    - `bucketId` integer, required — the bucketId of the bucket which this website will be ingested to.
    - `cap` integer — The maximum number of pages to crawl
    - `depth` integer — The maximum depth of linked pages to follow from the sourceUrl
    - `searchData` object — Custom metadata which can be used to influence GroundX's search functionality. This data can be used to further hone GroundX search.
    - `sourceUrl` string, uri, required — The URL from which the crawl is initiated.

## Response `200`

Website successfully queued

- IngestResponse
  - `ingest` object, required
    - `processId` string, uuid, required
    - `status` 'queued' | 'processing' | 'error' | 'complete' | 'cancelled', required

## Other responses

- `400` — Invalid source URL
- `401` — Unauthorized to update bucket with given ID

---

[API](https://skmtc.net/groundxai/apis/groundx-apis.md) · [All operations](https://skmtc.net/groundxai/apis/groundx-apis/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/groundxai/groundx-apis/versions/1c1a5c17f52f/schema)