CrewAI

Gli agent CrewAI hanno bisogno di contenuti web freschi e puliti come input. Incapsula /distill come Tool CrewAI così qualsiasi Agent della crew può leggere URL on demand.

Installazione

pip install crewai httpx

Tool custom

from crewai.tools import BaseTool
import httpx

API = "https://openapi.thunderbit.com/openapi/v1"
H = {"Authorization": "Bearer YOUR_API_KEY"}

class ReadUrlTool(BaseTool):
    name: str = "read_url"
    description: str = (
        "Fetch a URL and return clean Markdown. Use for any web research task: "
        "docs, articles, product pages, search results."
    )

    def _run(self, url: str) -> str:
        resp = httpx.post(f"{API}/distill",
                          headers=H,
                          json={"url": url, "renderMode": "basic"},
                          timeout=60.0)
        resp.raise_for_status()
        return resp.json()["data"]["markdown"]

Collega in una Crew

from crewai import Agent, Task, Crew

researcher = Agent(
    role="Web Researcher",
    goal="Gather authoritative information from public web pages",
    backstory="Skilled at distilling long pages into key facts.",
    tools=[ReadUrlTool()],
)

task = Task(
    description="Research how vector databases compare in 2026.",
    expected_output="A concise comparison table.",
    agent=researcher,
)

Crew(agents=[researcher], tasks=[task]).kickoff()

Suggerimenti

Per ricerche multi-source, esponi /batch/distill come secondo Tool (read_urls) così l'Agent può lanciare richieste in parallelo
Limita il Markdown restituito a ~8k token prima di passarlo all'Agent — evita il context bloat

Correlati

Questa integrazione sarà ampliata con template multi-agent crew — torna a controllare presto.