Thunderbit Open API home page
SupportAPI Console
Distill

Batch Distill Multiple Pages

POST
/batch/distill
Batch Distill Multiple Pages

Distill multiple web pages simultaneously with asynchronous processing.

Use Cases:

  • Process entire website sections or categories
  • Batch import content into your knowledge base
  • Large-scale content migration
  • Periodic content updates from multiple sources

How It Works:

  1. Submit a batch job with up to 100 URLs
  2. Receive a job ID immediately
  3. Poll the status endpoint or receive webhook notification
  4. Retrieve all results when complete

Features:

  • Asynchronous processing for high throughput
  • Automatic retry on failures
  • Webhook notifications when complete
  • Detailed per-URL status and error reporting

Authorization

YOUR_API_KEY
AuthorizationBearer <token>

Enter your API key from the Thunderbit Dashboard. The header format will be: Authorization: Bearer YOUR_API_KEY

In: header

Body

application/json

TypeScript Definitions

Use the request body type in TypeScript.

urls*string[]required

List of URLs to distill, maximum 100

Example:
[
  "https://example.com/page1",
  "https://example.com/page2",
  "https://example.com/page3"
]
timeoutnumberdefault:30000

Timeout per request in milliseconds, default 30000

Required range:5000 <= x <= 60000
countryCodestringdefault:"US"

2-letter ISO country code for geo-targeting, applied to every URL (default: US)

Example:US
renderModestringdefault:"none"

JavaScript rendering mode applied to every URL: none (default), basic, or full

Value in"none" | "basic" | "full"
includeHtmlbooleandefault:false

Whether to include the original HTML for each URL (default: false). Prefer the include parameter.

extractLinksbooleandefault:false

Whether to extract links for each URL (default: false). Prefer the include parameter.

includestring[]

Optional fields to include for each URL. Available values: metadata, html, links.

Example:
[
  "metadata",
  "links"
]
webhookobject

Webhook callback configuration, notifies when task completes. See the Webhooks guide for the callback payload format, headers, and HMAC-SHA256 signature verification.

Show child attributes
urlstring

Webhook callback URL, must be HTTPS

Example:https://your-server.com/api/webhook/distill
secretstring

HMAC-SHA256 secret for signature verification. When set, each webhook request includes an X-Webhook-Signature header

Example:whsec_your_secret_key
headersobject

Custom headers (reserved for future use, not yet implemented by the server).

Example:
{
  "X-Custom-Auth": "your-token"
}

Response

application/json

Success response

successboolean
Example:
true
dataobject
Show child attributes
idstring

Batch task ID

Example:batch_abc123
statusstring
Value in"PENDING" | "PROCESSING" | "COMPLETED" | "FAILED" | "CANCELLED" | "UNKNOWN"
Example:PROCESSING
totalinteger
Example:
3
completedinteger
Example:
0
creditsUsedinteger
Example:
0
createdAtstring

Timestamp when the job was created

completedAtstring

Timestamp when the job completed (null if still processing)