Integrations
CrewAI
Give CrewAI agents a Thunderbit-powered web research tool
CrewAI agents need fresh, clean web content as input. Wrap /distill as a CrewAI tool so any agent in the crew can read URLs on demand.
Install
pip install crewai httpxCustom tool
from crewai.tools import BaseTool
import httpx
API = "https://openapi.thunderbit.com/openapi/v1"
H = {"Authorization": "Bearer YOUR_API_KEY"}
class ReadUrlTool(BaseTool):
name: str = "read_url"
description: str = (
"Fetch a URL and return clean Markdown. Use for any web research task: "
"docs, articles, product pages, search results."
)
def _run(self, url: str) -> str:
resp = httpx.post(f"{API}/distill",
headers=H,
json={"url": url, "renderMode": "basic"},
timeout=60.0)
resp.raise_for_status()
return resp.json()["data"]["markdown"]Wire into a Crew
from crewai import Agent, Task, Crew
researcher = Agent(
role="Web Researcher",
goal="Gather authoritative information from public web pages",
backstory="Skilled at distilling long pages into key facts.",
tools=[ReadUrlTool()],
)
task = Task(
description="Research how vector databases compare in 2026.",
expected_output="A concise comparison table.",
agent=researcher,
)
Crew(agents=[researcher], tasks=[task]).kickoff()Tips
- For multi-source research, expose
/batch/distillas a second tool (read_urls) so the agent can fan out - Cap returned Markdown to ~8k tokens before handing it to the agent — avoid context bloat
Related
This integration is being expanded with multi-agent crew templates — check back soon.