實戰範例
新聞聚合器
用兩段 batch 流程聚合多家新聞來源的頭條
兩段式設計：先 distill 每個來源的首頁找出文章 URL，再從這些 URL 擷取結構化的頭條紀錄。
流程
- 第一段 —— 對每個來源首頁跑
/batch/distill，加上include: ["links"] - 用 pattern 過濾抓到的連結（例如今天日期、文章路徑）
- 第二段 —— 對過濾後的 URL 跑
/batch/extract，套用頭條 schema - 兩段都跑完時 Webhook 觸發；把結果灌進你的儲存層
Schema
{
"type": "object",
"properties": {
"title": { "type": "string", "description": "The article headline" },
"url": { "type": "string", "description": "Canonical article URL" },
"publishedAt": { "type": "string", "description": "ISO-8601 publish timestamp" },
"summary": { "type": "string", "description": "1-2 sentence summary" }
},
"required": ["title", "url"]
}實作概述
import httpx
API = "https://openapi.thunderbit.com/openapi/v1"
H = {"Authorization": "Bearer YOUR_API_KEY"}
# Pass 1: distill homepages, collect article URLs
sources = ["https://news.example.com", "https://blog.example.org"]
home_job = httpx.post(f"{API}/batch/distill",
headers=H,
json={"urls": sources, "include": ["links"]}).json()
# (poll until COMPLETED — see Batch Job Lifecycle guide)
article_urls = []
for r in home_job["data"]["results"]:
if r["status"] == "SUCCEEDED":
article_urls += [u for u in r["links"] if "/article/" in u]
# Pass 2: extract structured headlines
extract_job = httpx.post(f"{API}/batch/extract",
headers=H,
json={"urls": article_urls, "schema": SCHEMA}).json()小技巧
- 首頁結果做 5–10 分鐘 TTL 快取，避免在同樣的頭條上重複燒額度
- 若來源會用 IP 地理化首頁，設定
countryCode - 用 canonical URL 或 content hash 去重 —— 同樣的頭條常常出現在多家來源
相關
本食譜將補充完整的兩段式編排器 —— 敬請期待。