Rust

用 reqwest + serde + tokio。預設非同步；搭配 futures::stream::iter 做散發。

Cargo.toml

[dependencies]
reqwest = { version = "0.12", features = ["json"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
tokio = { version = "1", features = ["full"] }

設定

use reqwest::Client;
use serde_json::json;

const API: &str = "https://openapi.thunderbit.com/openapi/v1";

fn client() -> Client {
    Client::builder()
        .timeout(std::time::Duration::from_secs(60))
        .build()
        .unwrap()
}

fn auth() -> String {
    format!("Bearer {}", std::env::var("THUNDERBIT_API_KEY").unwrap())
}

Distill 一個頁面

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let res: serde_json::Value = client()
        .post(format!("{API}/distill"))
        .header("Authorization", auth())
        .json(&json!({ "url": "https://thunderbit.com/playground" }))
        .send()
        .await?
        .error_for_status()?
        .json()
        .await?;
    println!("{}", res["data"]["markdown"]);
    Ok(())
}

Extract 結構化資料

let res: serde_json::Value = client()
    .post(format!("{API}/extract"))
    .header("Authorization", auth())
    .json(&json!({
        "url": "https://example.com/product/iphone-15-pro",
        "schema": {
            "type": "object",
            "properties": {
                "name":  { "type": "string" },
                "price": { "type": "number" }
            },
            "required": ["name", "price"]
        }
    }))
    .send().await?.error_for_status()?.json().await?;

批次散發

要用單個 URL 做高並發 distill（而不是走 batch endpoint），用一個有界的 JoinSet：

use tokio::task::JoinSet;

let mut set = JoinSet::new();
for url in urls {
    let c = client();
    set.spawn(async move {
        c.post(format!("{API}/distill"))
            .header("Authorization", auth())
            .json(&json!({ "url": url }))
            .send().await?.error_for_status()?
            .json::<serde_json::Value>().await
    });
}
while let Some(res) = set.join_next().await { /* … */ }

URL 數量達 10 個以上，優先用 /batch/distill，不要自己散發 —— 參見 Batch Job Lifecycle。

官方 Rust SDK 開發中 —— 敬請期待。

Rust

Cargo.toml

設定

Distill 一個頁面

Extract 結構化資料

批次散發

目錄