Distillation par Lots de Plusieurs Pages
Distillez plusieurs pages web simultanément avec un traitement asynchrone.
Cas d'Utilisation :
- Traiter des sections ou catégories entières de sites web
- Importer du contenu en masse dans votre base de connaissances
- Migration de contenu à grande échelle
- Mises à jour périodiques de contenu depuis plusieurs sources
Comment Ça Fonctionne :
- Soumettez une tâche par lots avec jusqu'à 100 URLs
- Recevez immédiatement un ID de tâche
- Interrogez l'endpoint de statut ou recevez une notification webhook
- Récupérez tous les résultats une fois terminé
Fonctionnalités :
- Traitement asynchrone pour un haut débit
- Réessai automatique en cas d'échec
- Notifications webhook à la fin
- Rapports détaillés de statut et d'erreurs par URL
Autorisation
YOUR_API_KEY Récupère ta clé API depuis le Thunderbit Dashboard. Format d'en-tête : Authorization: Bearer YOUR_API_KEY
Dans: header
Corps
application/json
TypeScript Definitions
Use the request body type in TypeScript.
Liste des URLs à distiller, maximum 100
[
"https://example.com/page1",
"https://example.com/page2",
"https://example.com/page3"
]30000Délai d'expiration par requête en millisecondes, par défaut 30000
"US"Code pays ISO à 2 lettres pour le ciblage géographique, appliqué à chaque URL (par défaut : US)
US"none"Mode de rendu JavaScript appliqué à chaque URL : none (par défaut), basic ou full
falseIndique si le HTML original doit être inclus pour chaque URL (par défaut : false). Préférer le paramètre include.
falseIndique si les liens doivent être extraits pour chaque URL (par défaut : false). Préférer le paramètre include.
Champs optionnels à inclure pour chaque URL. Valeurs disponibles : metadata, html, links.
[
"metadata",
"links"
]Configuration du callback webhook, notifie lorsque la tâche est terminée. Consultez le guide Webhooks pour le format du payload de callback, les en-têtes et la vérification de signature HMAC-SHA256.
Afficher les attributs enfants
Webhook callback URL, must be HTTPS
https://your-server.com/api/webhook/distillHMAC-SHA256 secret for signature verification. When set, each webhook request includes an X-Webhook-Signature header
whsec_your_secret_keyCustom headers (reserved for future use, not yet implemented by the server).
{
"X-Custom-Auth": "your-token"
}Réponse
application/jsonSuccess response
trueAfficher les attributs enfants
ID de la tâche par lots
batch_abc123PROCESSING300Timestamp when the job was created
Timestamp when the job completed (null if still processing)
Paramètres de requête invalides
falseAfficher les attributs enfants
Canonical error codes returned in the error.code field (LEGACY) or error.status field (GOOGLE_RPC). Codes are grouped by HTTP status:
400 Bad Request
INVALID_URL— URL format is invalidINVALID_SCHEMA— JSON Schema is invalidINVALID_PARAMETER— One or more request parameters failed validationSCHEMA_OR_PROMPT_REQUIRED—schema(orprompt) is required for extractionSCHEMA_AND_PROMPT_EXCLUSIVE—schemaandpromptcannot both be providedBATCH_SIZE_EXCEEDED— Batch request exceeds the 100-URL limitMALFORMED_REQUEST_BODY— Request body is not valid JSON
401 Unauthorized
API_KEY_MISSING—Authorizationheader missingAPI_KEY_INVALID_FORMAT— API key format is invalidAPI_KEY_NOT_FOUND— API key not found in the systemAPI_KEY_REVOKED— API key has been revokedAPI_KEY_DISABLED— API key has been disabledAPI_KEY_EXPIRED— API key has expiredINVALID_API_KEY— (deprecated, use the specific API_KEY_ codes above)*
402 Payment Required
INSUFFICIENT_CREDITS— Not enough credits on the account
404 Not Found
JOB_NOT_FOUND— Batch job not foundRESOURCE_NOT_FOUND— Generic resource not found
408 Request Timeout
REQUEST_TIMEOUT— The API request timed outSCRAPE_TIMEOUT— The target page took too long to respond
422 Unprocessable Entity
SCRAPE_SSL_ERROR— Target site has SSL/TLS problemsSCRAPE_DNS_RESOLUTION_ERROR— Cannot resolve target hostnameSCRAPE_SITE_ERROR— Target site returned an errorSCRAPE_EMPTY_CONTENT— Target page returned empty contentSCRAPE_CONTENT_TOO_LARGE— Target page exceeds size limitsSCRAPE_TARGET_FORBIDDEN— Target site refused access (403)SCRAPE_TARGET_NOT_FOUND— Target URL returned 404SCRAPE_UNSUPPORTED_FILE— Target file type is not supported
429 Too Many Requests
RATE_LIMIT_EXCEEDED— Account rate limit triggeredSCRAPE_TARGET_RATE_LIMITED— Target site rate-limited our request
500 Internal Server Error
INTERNAL_ERROR— Generic internal errorDISTILL_FAILED— Distillation pipeline failedEXTRACT_FAILED— Extraction pipeline failedPIPELINE_ERROR— Pipeline step reported a failureAI_EXTRACTION_FAILED— AI extraction step failedMARKDOWN_CONVERSION_FAILED— HTML-to-Markdown conversion failed
502 Bad Gateway
SCRAPE_ALL_PROVIDERS_FAILED— All scraping providers failedUPSTREAM_BAD_GATEWAY— Upstream returned an invalid response
503 Service Unavailable
SCRAPE_PROVIDER_UNAVAILABLE— Scraping provider is unavailableAI_SERVICE_UNAVAILABLE— AI service is unavailableDOWNSTREAM_SERVICE_UNAVAILABLE— A downstream service is temporarily unavailable
504 Gateway Timeout
UPSTREAM_TIMEOUT— Upstream gateway timed outAI_TIMEOUT— AI service call timed out
HTTP status code (LEGACY format) or canonical error code (GOOGLE_RPC format). See the API_ERROR_FORMAT note in the Error schema description
Human-readable error message
Optional structured details — for example, field-level validation errors as { field: message } pairs
Échec de l'authentification, clé API invalide
falseAfficher les attributs enfants
Canonical error codes returned in the error.code field (LEGACY) or error.status field (GOOGLE_RPC). Codes are grouped by HTTP status:
400 Bad Request
INVALID_URL— URL format is invalidINVALID_SCHEMA— JSON Schema is invalidINVALID_PARAMETER— One or more request parameters failed validationSCHEMA_OR_PROMPT_REQUIRED—schema(orprompt) is required for extractionSCHEMA_AND_PROMPT_EXCLUSIVE—schemaandpromptcannot both be providedBATCH_SIZE_EXCEEDED— Batch request exceeds the 100-URL limitMALFORMED_REQUEST_BODY— Request body is not valid JSON
401 Unauthorized
API_KEY_MISSING—Authorizationheader missingAPI_KEY_INVALID_FORMAT— API key format is invalidAPI_KEY_NOT_FOUND— API key not found in the systemAPI_KEY_REVOKED— API key has been revokedAPI_KEY_DISABLED— API key has been disabledAPI_KEY_EXPIRED— API key has expiredINVALID_API_KEY— (deprecated, use the specific API_KEY_ codes above)*
402 Payment Required
INSUFFICIENT_CREDITS— Not enough credits on the account
404 Not Found
JOB_NOT_FOUND— Batch job not foundRESOURCE_NOT_FOUND— Generic resource not found
408 Request Timeout
REQUEST_TIMEOUT— The API request timed outSCRAPE_TIMEOUT— The target page took too long to respond
422 Unprocessable Entity
SCRAPE_SSL_ERROR— Target site has SSL/TLS problemsSCRAPE_DNS_RESOLUTION_ERROR— Cannot resolve target hostnameSCRAPE_SITE_ERROR— Target site returned an errorSCRAPE_EMPTY_CONTENT— Target page returned empty contentSCRAPE_CONTENT_TOO_LARGE— Target page exceeds size limitsSCRAPE_TARGET_FORBIDDEN— Target site refused access (403)SCRAPE_TARGET_NOT_FOUND— Target URL returned 404SCRAPE_UNSUPPORTED_FILE— Target file type is not supported
429 Too Many Requests
RATE_LIMIT_EXCEEDED— Account rate limit triggeredSCRAPE_TARGET_RATE_LIMITED— Target site rate-limited our request
500 Internal Server Error
INTERNAL_ERROR— Generic internal errorDISTILL_FAILED— Distillation pipeline failedEXTRACT_FAILED— Extraction pipeline failedPIPELINE_ERROR— Pipeline step reported a failureAI_EXTRACTION_FAILED— AI extraction step failedMARKDOWN_CONVERSION_FAILED— HTML-to-Markdown conversion failed
502 Bad Gateway
SCRAPE_ALL_PROVIDERS_FAILED— All scraping providers failedUPSTREAM_BAD_GATEWAY— Upstream returned an invalid response
503 Service Unavailable
SCRAPE_PROVIDER_UNAVAILABLE— Scraping provider is unavailableAI_SERVICE_UNAVAILABLE— AI service is unavailableDOWNSTREAM_SERVICE_UNAVAILABLE— A downstream service is temporarily unavailable
504 Gateway Timeout
UPSTREAM_TIMEOUT— Upstream gateway timed outAI_TIMEOUT— AI service call timed out
HTTP status code (LEGACY format) or canonical error code (GOOGLE_RPC format). See the API_ERROR_FORMAT note in the Error schema description
Human-readable error message
Optional structured details — for example, field-level validation errors as { field: message } pairs
Trop de requêtes, limite de débit atteinte
falseAfficher les attributs enfants
Canonical error codes returned in the error.code field (LEGACY) or error.status field (GOOGLE_RPC). Codes are grouped by HTTP status:
400 Bad Request
INVALID_URL— URL format is invalidINVALID_SCHEMA— JSON Schema is invalidINVALID_PARAMETER— One or more request parameters failed validationSCHEMA_OR_PROMPT_REQUIRED—schema(orprompt) is required for extractionSCHEMA_AND_PROMPT_EXCLUSIVE—schemaandpromptcannot both be providedBATCH_SIZE_EXCEEDED— Batch request exceeds the 100-URL limitMALFORMED_REQUEST_BODY— Request body is not valid JSON
401 Unauthorized
API_KEY_MISSING—Authorizationheader missingAPI_KEY_INVALID_FORMAT— API key format is invalidAPI_KEY_NOT_FOUND— API key not found in the systemAPI_KEY_REVOKED— API key has been revokedAPI_KEY_DISABLED— API key has been disabledAPI_KEY_EXPIRED— API key has expiredINVALID_API_KEY— (deprecated, use the specific API_KEY_ codes above)*
402 Payment Required
INSUFFICIENT_CREDITS— Not enough credits on the account
404 Not Found
JOB_NOT_FOUND— Batch job not foundRESOURCE_NOT_FOUND— Generic resource not found
408 Request Timeout
REQUEST_TIMEOUT— The API request timed outSCRAPE_TIMEOUT— The target page took too long to respond
422 Unprocessable Entity
SCRAPE_SSL_ERROR— Target site has SSL/TLS problemsSCRAPE_DNS_RESOLUTION_ERROR— Cannot resolve target hostnameSCRAPE_SITE_ERROR— Target site returned an errorSCRAPE_EMPTY_CONTENT— Target page returned empty contentSCRAPE_CONTENT_TOO_LARGE— Target page exceeds size limitsSCRAPE_TARGET_FORBIDDEN— Target site refused access (403)SCRAPE_TARGET_NOT_FOUND— Target URL returned 404SCRAPE_UNSUPPORTED_FILE— Target file type is not supported
429 Too Many Requests
RATE_LIMIT_EXCEEDED— Account rate limit triggeredSCRAPE_TARGET_RATE_LIMITED— Target site rate-limited our request
500 Internal Server Error
INTERNAL_ERROR— Generic internal errorDISTILL_FAILED— Distillation pipeline failedEXTRACT_FAILED— Extraction pipeline failedPIPELINE_ERROR— Pipeline step reported a failureAI_EXTRACTION_FAILED— AI extraction step failedMARKDOWN_CONVERSION_FAILED— HTML-to-Markdown conversion failed
502 Bad Gateway
SCRAPE_ALL_PROVIDERS_FAILED— All scraping providers failedUPSTREAM_BAD_GATEWAY— Upstream returned an invalid response
503 Service Unavailable
SCRAPE_PROVIDER_UNAVAILABLE— Scraping provider is unavailableAI_SERVICE_UNAVAILABLE— AI service is unavailableDOWNSTREAM_SERVICE_UNAVAILABLE— A downstream service is temporarily unavailable
504 Gateway Timeout
UPSTREAM_TIMEOUT— Upstream gateway timed outAI_TIMEOUT— AI service call timed out
HTTP status code (LEGACY format) or canonical error code (GOOGLE_RPC format). See the API_ERROR_FORMAT note in the Error schema description
Human-readable error message
Optional structured details — for example, field-level validation errors as { field: message } pairs