v2

latestOpenAPI 3.1.02026-07-31125.9 KB

Returns the content of the web pages

Returns the HTML or Markdown of a target webpage.

post/v1/contents

Request body

urlsstring[]

Array of URLs to fetch the contents from.

formatsstring[]

Array of content formats to return. All included formats are returned in the response. Include "metadata" to get JSON-LD and OpenGraph information, if available.

crawl_timeoutinteger

Maximum time in seconds to wait for page content. Must be between 1 and 60 seconds. Default is 10 seconds.

max_ageinteger nullable

Maximum allowed age of cached content in seconds. When set, cached content older than this threshold is ignored and the page is re-fetched. Must be 0 or greater. Default: null (no age limit, cached content is returned regardless of age).

Example request

{
  "urls": [
    "https://www.you.com"
  ],
  "formats": [
    "html",
    "markdown"
  ],
  "crawl_timeout": 10,
  "max_age": 86400
}

Response

An array of JSON objects containing the page content of each web page

urlstring uri

The webpage URL whose content has been fetched.

titlestring

The title of the web page.

htmlstring nullable

The retrieved HTML content of the web page.

markdownstring nullable

The retrieved Markdown content of the web page.

Example response

[
  {
    "url": "https://www.you.com",
    "title": "The best website in the world",
    "metadata": {
      "site_name": "You.com",
      "favicon_url": "https://api.ydc-index.io/favicon?domain=you.com&size=128"
    }
  }
]

All 1 operations