---
title: "Crawl a website for indexed search"
method: GET
path: "/integrations/web_crawler/index"
tags: ["Integrations", "Integrations"]
---

# Crawl a website for indexed search

`GET /integrations/web_crawler/index`

Recursively crawl a website to make it available for indexed search.

## Query parameters

- `url` string, required — The base URL of the website to crawl
- `max_depth` integer — Maximum depth of links to follow during crawling
- `limit` integer — Maximum number of pages to crawl in total

## Response `200`

Successful Response

- ResourceStatusResponse
  - `source` 'reddit' | 'notion' | 'slack' | 'google_calendar' | 'google_mail' | 'box' | 'dropbox' | 'github' | 'google_drive' | 'vault' | 'web_crawler' | 'trace' | 'microsoft_teams' | 'gmail_actions' | 'granola' | 'fathom' | 'fireflies' | 'linear' | 'hubspot' | 'salesforce' | 'coda' | 'lightfield' | 'gong', required
  - `resource_id` string, required
  - `status` 'pending' | 'processing' | 'completed' | 'failed' | 'pending_review' | 'skipped', required

## Other responses

- `422` — Validation Error

---

[API](https://skmtc.net/hyperspell/apis/hyperspell-api.md) · [All operations](https://skmtc.net/hyperspell/apis/hyperspell-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/hyperspell/hyperspell-api/revisions/5721acd588e6/schema)