批量蒸馏多个页面
通过异步处理同时蒸馏多个网页。
使用场景:
- 处理整个网站板块或分类
- 批量导入内容到知识库
- 大规模内容迁移
- 定期从多个来源更新内容
工作原理:
- 提交最多 100 个 URL 的批量任务
- 立即获得任务 ID
- 轮询状态端点或接收 Webhook 通知
- 完成后获取所有结果
功能特性:
- 异步处理实现高吞吐量
- 失败自动重试
- 完成时 Webhook 通知
- 详细的每 URL 状态和错误报告
授权
YOUR_API_KEY 在 Thunderbit Dashboard 获取 API key。请求头格式:Authorization: Bearer YOUR_API_KEY
位置: header
请求体
application/json
TypeScript Definitions
Use the request body type in TypeScript.
要蒸馏的 URL 列表,最多 100 个
[
"https://example.com/page1",
"https://example.com/page2",
"https://example.com/page3"
]30000每个请求的超时时间(毫秒),默认 30000
"US"用于地理定位的 2 字母 ISO 国家码,应用于每个 URL(默认 US)
US"none"应用于每个 URL 的 JavaScript 渲染模式:none(默认)、basic 或 full
false是否为每个 URL 包含原始 HTML(默认 false)。建议改用 include 参数。
false是否为每个 URL 提取链接(默认 false)。建议改用 include 参数。
每个 URL 可选包含的字段。可选值:metadata、html、links。
[
"metadata",
"links"
]Webhook 回调配置,任务完成时通知。查看 Webhooks 指南 了解回调负载、请求头和 HMAC-SHA256 签名验证。
展开子属性
Webhook callback URL, must be HTTPS
https://your-server.com/api/webhook/distillHMAC-SHA256 secret for signature verification. When set, each webhook request includes an X-Webhook-Signature header
whsec_your_secret_keyCustom headers (reserved for future use, not yet implemented by the server).
{
"X-Custom-Auth": "your-token"
}响应
application/jsonSuccess response
true展开子属性
批量任务 ID
batch_abc123PROCESSING300Timestamp when the job was created
Timestamp when the job completed (null if still processing)
无效的请求参数
false展开子属性
Canonical error codes returned in the error.code field (LEGACY) or error.status field (GOOGLE_RPC). Codes are grouped by HTTP status:
400 Bad Request
INVALID_URL— URL format is invalidINVALID_SCHEMA— JSON Schema is invalidINVALID_PARAMETER— One or more request parameters failed validationSCHEMA_OR_PROMPT_REQUIRED—schema(orprompt) is required for extractionSCHEMA_AND_PROMPT_EXCLUSIVE—schemaandpromptcannot both be providedBATCH_SIZE_EXCEEDED— Batch request exceeds the 100-URL limitMALFORMED_REQUEST_BODY— Request body is not valid JSON
401 Unauthorized
API_KEY_MISSING—Authorizationheader missingAPI_KEY_INVALID_FORMAT— API key format is invalidAPI_KEY_NOT_FOUND— API key not found in the systemAPI_KEY_REVOKED— API key has been revokedAPI_KEY_DISABLED— API key has been disabledAPI_KEY_EXPIRED— API key has expiredINVALID_API_KEY— (deprecated, use the specific API_KEY_ codes above)*
402 Payment Required
INSUFFICIENT_CREDITS— Not enough credits on the account
404 Not Found
JOB_NOT_FOUND— Batch job not foundRESOURCE_NOT_FOUND— Generic resource not found
408 Request Timeout
REQUEST_TIMEOUT— The API request timed outSCRAPE_TIMEOUT— The target page took too long to respond
422 Unprocessable Entity
SCRAPE_SSL_ERROR— Target site has SSL/TLS problemsSCRAPE_DNS_RESOLUTION_ERROR— Cannot resolve target hostnameSCRAPE_SITE_ERROR— Target site returned an errorSCRAPE_EMPTY_CONTENT— Target page returned empty contentSCRAPE_CONTENT_TOO_LARGE— Target page exceeds size limitsSCRAPE_TARGET_FORBIDDEN— Target site refused access (403)SCRAPE_TARGET_NOT_FOUND— Target URL returned 404SCRAPE_UNSUPPORTED_FILE— Target file type is not supported
429 Too Many Requests
RATE_LIMIT_EXCEEDED— Account rate limit triggeredSCRAPE_TARGET_RATE_LIMITED— Target site rate-limited our request
500 Internal Server Error
INTERNAL_ERROR— Generic internal errorDISTILL_FAILED— Distillation pipeline failedEXTRACT_FAILED— Extraction pipeline failedPIPELINE_ERROR— Pipeline step reported a failureAI_EXTRACTION_FAILED— AI extraction step failedMARKDOWN_CONVERSION_FAILED— HTML-to-Markdown conversion failed
502 Bad Gateway
SCRAPE_ALL_PROVIDERS_FAILED— All scraping providers failedUPSTREAM_BAD_GATEWAY— Upstream returned an invalid response
503 Service Unavailable
SCRAPE_PROVIDER_UNAVAILABLE— Scraping provider is unavailableAI_SERVICE_UNAVAILABLE— AI service is unavailableDOWNSTREAM_SERVICE_UNAVAILABLE— A downstream service is temporarily unavailable
504 Gateway Timeout
UPSTREAM_TIMEOUT— Upstream gateway timed outAI_TIMEOUT— AI service call timed out
HTTP status code (LEGACY format) or canonical error code (GOOGLE_RPC format). See the API_ERROR_FORMAT note in the Error schema description
Human-readable error message
Optional structured details — for example, field-level validation errors as { field: message } pairs
认证失败,无效的 API Key
false展开子属性
Canonical error codes returned in the error.code field (LEGACY) or error.status field (GOOGLE_RPC). Codes are grouped by HTTP status:
400 Bad Request
INVALID_URL— URL format is invalidINVALID_SCHEMA— JSON Schema is invalidINVALID_PARAMETER— One or more request parameters failed validationSCHEMA_OR_PROMPT_REQUIRED—schema(orprompt) is required for extractionSCHEMA_AND_PROMPT_EXCLUSIVE—schemaandpromptcannot both be providedBATCH_SIZE_EXCEEDED— Batch request exceeds the 100-URL limitMALFORMED_REQUEST_BODY— Request body is not valid JSON
401 Unauthorized
API_KEY_MISSING—Authorizationheader missingAPI_KEY_INVALID_FORMAT— API key format is invalidAPI_KEY_NOT_FOUND— API key not found in the systemAPI_KEY_REVOKED— API key has been revokedAPI_KEY_DISABLED— API key has been disabledAPI_KEY_EXPIRED— API key has expiredINVALID_API_KEY— (deprecated, use the specific API_KEY_ codes above)*
402 Payment Required
INSUFFICIENT_CREDITS— Not enough credits on the account
404 Not Found
JOB_NOT_FOUND— Batch job not foundRESOURCE_NOT_FOUND— Generic resource not found
408 Request Timeout
REQUEST_TIMEOUT— The API request timed outSCRAPE_TIMEOUT— The target page took too long to respond
422 Unprocessable Entity
SCRAPE_SSL_ERROR— Target site has SSL/TLS problemsSCRAPE_DNS_RESOLUTION_ERROR— Cannot resolve target hostnameSCRAPE_SITE_ERROR— Target site returned an errorSCRAPE_EMPTY_CONTENT— Target page returned empty contentSCRAPE_CONTENT_TOO_LARGE— Target page exceeds size limitsSCRAPE_TARGET_FORBIDDEN— Target site refused access (403)SCRAPE_TARGET_NOT_FOUND— Target URL returned 404SCRAPE_UNSUPPORTED_FILE— Target file type is not supported
429 Too Many Requests
RATE_LIMIT_EXCEEDED— Account rate limit triggeredSCRAPE_TARGET_RATE_LIMITED— Target site rate-limited our request
500 Internal Server Error
INTERNAL_ERROR— Generic internal errorDISTILL_FAILED— Distillation pipeline failedEXTRACT_FAILED— Extraction pipeline failedPIPELINE_ERROR— Pipeline step reported a failureAI_EXTRACTION_FAILED— AI extraction step failedMARKDOWN_CONVERSION_FAILED— HTML-to-Markdown conversion failed
502 Bad Gateway
SCRAPE_ALL_PROVIDERS_FAILED— All scraping providers failedUPSTREAM_BAD_GATEWAY— Upstream returned an invalid response
503 Service Unavailable
SCRAPE_PROVIDER_UNAVAILABLE— Scraping provider is unavailableAI_SERVICE_UNAVAILABLE— AI service is unavailableDOWNSTREAM_SERVICE_UNAVAILABLE— A downstream service is temporarily unavailable
504 Gateway Timeout
UPSTREAM_TIMEOUT— Upstream gateway timed outAI_TIMEOUT— AI service call timed out
HTTP status code (LEGACY format) or canonical error code (GOOGLE_RPC format). See the API_ERROR_FORMAT note in the Error schema description
Human-readable error message
Optional structured details — for example, field-level validation errors as { field: message } pairs
请求过多,触发速率限制
false展开子属性
Canonical error codes returned in the error.code field (LEGACY) or error.status field (GOOGLE_RPC). Codes are grouped by HTTP status:
400 Bad Request
INVALID_URL— URL format is invalidINVALID_SCHEMA— JSON Schema is invalidINVALID_PARAMETER— One or more request parameters failed validationSCHEMA_OR_PROMPT_REQUIRED—schema(orprompt) is required for extractionSCHEMA_AND_PROMPT_EXCLUSIVE—schemaandpromptcannot both be providedBATCH_SIZE_EXCEEDED— Batch request exceeds the 100-URL limitMALFORMED_REQUEST_BODY— Request body is not valid JSON
401 Unauthorized
API_KEY_MISSING—Authorizationheader missingAPI_KEY_INVALID_FORMAT— API key format is invalidAPI_KEY_NOT_FOUND— API key not found in the systemAPI_KEY_REVOKED— API key has been revokedAPI_KEY_DISABLED— API key has been disabledAPI_KEY_EXPIRED— API key has expiredINVALID_API_KEY— (deprecated, use the specific API_KEY_ codes above)*
402 Payment Required
INSUFFICIENT_CREDITS— Not enough credits on the account
404 Not Found
JOB_NOT_FOUND— Batch job not foundRESOURCE_NOT_FOUND— Generic resource not found
408 Request Timeout
REQUEST_TIMEOUT— The API request timed outSCRAPE_TIMEOUT— The target page took too long to respond
422 Unprocessable Entity
SCRAPE_SSL_ERROR— Target site has SSL/TLS problemsSCRAPE_DNS_RESOLUTION_ERROR— Cannot resolve target hostnameSCRAPE_SITE_ERROR— Target site returned an errorSCRAPE_EMPTY_CONTENT— Target page returned empty contentSCRAPE_CONTENT_TOO_LARGE— Target page exceeds size limitsSCRAPE_TARGET_FORBIDDEN— Target site refused access (403)SCRAPE_TARGET_NOT_FOUND— Target URL returned 404SCRAPE_UNSUPPORTED_FILE— Target file type is not supported
429 Too Many Requests
RATE_LIMIT_EXCEEDED— Account rate limit triggeredSCRAPE_TARGET_RATE_LIMITED— Target site rate-limited our request
500 Internal Server Error
INTERNAL_ERROR— Generic internal errorDISTILL_FAILED— Distillation pipeline failedEXTRACT_FAILED— Extraction pipeline failedPIPELINE_ERROR— Pipeline step reported a failureAI_EXTRACTION_FAILED— AI extraction step failedMARKDOWN_CONVERSION_FAILED— HTML-to-Markdown conversion failed
502 Bad Gateway
SCRAPE_ALL_PROVIDERS_FAILED— All scraping providers failedUPSTREAM_BAD_GATEWAY— Upstream returned an invalid response
503 Service Unavailable
SCRAPE_PROVIDER_UNAVAILABLE— Scraping provider is unavailableAI_SERVICE_UNAVAILABLE— AI service is unavailableDOWNSTREAM_SERVICE_UNAVAILABLE— A downstream service is temporarily unavailable
504 Gateway Timeout
UPSTREAM_TIMEOUT— Upstream gateway timed outAI_TIMEOUT— AI service call timed out
HTTP status code (LEGACY format) or canonical error code (GOOGLE_RPC format). See the API_ERROR_FORMAT note in the Error schema description
Human-readable error message
Optional structured details — for example, field-level validation errors as { field: message } pairs