---
title: "Crawl URLs"
method: POST
path: "/1/crawlers/{id}/urls/crawl"
tags: ["crawler"]
---

# Crawl URLs

`POST /1/crawlers/{id}/urls/crawl`

Crawls the specified URLs, extracts records from them, and adds them to the index.
If a crawl is running (the crawler's `reindexing` property is `true`),
the records are added to a temporary index.

This operation is rate-limited to 500 requests every 24 hours.

## Path parameters

- `id` string, required — Universally unique identifier (UUID) of the crawler.

## Request body

- object
  - `urls` string[], required — URLs to crawl.
  - `save` boolean — Whether the specified URLs should be added to the `extraURLs` property of the crawler configuration. If unspecified, the URLs are added to the `extraUrls` field only if they haven't been indexed during the last reindex.

## Response `200`

OK

- object
  - `taskId` string, required — Universally unique identifier (UUID) of the task.

## Other responses

- `400` — Invalid request.
- `401` — Authorization information is missing or invalid.
- `403` — The user doesn't have enough rights on the specified Crawler, or it doesn't exists.

---

[API](https://skmtc.net/algolia/apis/crawler-api.md) · [All operations](https://skmtc.net/algolia/apis/crawler-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/algolia/crawler-api/versions/7f1df3bfe8bd/schema)