v1

latestOpenAPI 3.0.02026-07-263039205.4 KB
Article

Article

Automatically extract clean article text and other data from news articles, blog posts and other text-heavy pages.

get/article

Query parameters

urlstring required

Target URL to extract

fields'links' | 'extlinks' | 'meta' | 'querystring' | 'breadcrumb' | 'quotes'

Specify optional fields to be returned from any fully-extracted pages (e.g. fields=querystring,links)

timeoutinteger

Sets a value in milliseconds to wait for the retrieval/fetch of content from the requested URL. The default timeout for the third-party response is 30 seconds (30000).

callbackstring

Use for jsonp requests. Needed for cross-domain ajax.

proxystring

Specify an IP address of a custom proxy that will be used to fetch the target page. (Ex: &proxy or &proxy=0.0.0.0)

proxyAuthstring

Used to specify the authentication parameters that will be used with a custom proxy specified in the &proxy parameter. (Ex: proxyAuth=username:password)

useProxystring

Set to default to use Diffbot's datacenter proxy for this request. none will instruct Extract to not use proxies, even if proxies have been enabled for this particular URL globally.

pagingboolean

Pass paging=false to disable automatic concatenation multiple-page articles.

maxTagsinteger

Set the maximum number of automatically-generated tags to return. (Default: 10)

tagConfidencenumber float

Set the minimum relevance score of tags to return, between 0.0 and 1.0. By default only tags with a score equal to or above 0.5 will be returned.

categoryConfidencenumber float

Set the minimum relevance score of categories to return, between 0.0 and 1.0. By default only categories with a score equal to or above 0.5 will be returned.

discussionboolean

Pass discussion=false to disable automatic extraction of article comments.

naturalLanguage'entities' | 'sentiment' | 'summary' | 'facts' | 'openFacts' | 'records' | 'categories' | 'sentences' | 'language'

Run extracted text and title through the Diffbot Natural Language API. Example: &naturalLanguage=entities,facts,categories,sentiment.

summaryNumSentencesinteger

Sets the maximum number of sentences for summary generation when using naturalLanguage=summary (Default: 3).

renderDelayinteger

Add additional time for rendering before the page is closed and the DOM is extracted. This can cause page timeouts, so a timeout parameter may be needed to extend the timeout. Note that the renderer closes automatically at 180 seconds.

scroll'slow' | 'fast'

Direct the browser to scroll down the page, to trigger lazy-loaded content.

Response

Successful API Response

humanLanguagestring
typestring
titlestring