集成
CrewAI
给 CrewAI agent 配一个 Thunderbit 驱动的网页调研 Tool
CrewAI agent 需要新鲜、干净的网页内容做输入。把 /distill 包装成 CrewAI Tool，crew 里的任意 agent 都能按需读取 URL。
安装
pip install crewai httpx自定义 Tool
from crewai.tools import BaseTool
import httpx
API = "https://openapi.thunderbit.com/openapi/v1"
H = {"Authorization": "Bearer YOUR_API_KEY"}
class ReadUrlTool(BaseTool):
name: str = "read_url"
description: str = (
"Fetch a URL and return clean Markdown. Use for any web research task: "
"docs, articles, product pages, search results."
)
def _run(self, url: str) -> str:
resp = httpx.post(f"{API}/distill",
headers=H,
json={"url": url, "renderMode": "basic"},
timeout=60.0)
resp.raise_for_status()
return resp.json()["data"]["markdown"]接入 Crew
from crewai import Agent, Task, Crew
researcher = Agent(
role="Web Researcher",
goal="Gather authoritative information from public web pages",
backstory="Skilled at distilling long pages into key facts.",
tools=[ReadUrlTool()],
)
task = Task(
description="Research how vector databases compare in 2026.",
expected_output="A concise comparison table.",
agent=researcher,
)
Crew(agents=[researcher], tasks=[task]).kickoff()小贴士
- 多源调研时，再暴露一个
/batch/distill当第二个 Tool（read_urls），让 agent 并发抓取 - 把返回的 Markdown 截到 ~8k token 再给 agent —— 别把 context 撑爆
相关链接
这个集成正在补充多 agent crew 模板，敬请期待。