Search "Colly" and the first adjective is always the same one: fast. Fast Go crawler, fast because it compiles, fast because there's no browser in the way. Almost nobody attaches a number to it.
So I stopped taking the word on faith. I stood up a small fixture site, compiled Colly against it, and watched what the library actually did — recall on real pages, how it routed a failed request, how far a depth-limited crawl wandered. The short version, before the numbers: static extraction came back at full recall, a 500 landed exactly where it should, and a depth-capped crawl reached 17 pages out of a single static binary with no browser attached. The library also returned a clean zero on everything JavaScript-rendered — which happens to be the part the "fast" chorus tends to skip.
What Colly actually is (and isn't)
Colly bills itself as an "elegant scraper and crawler framework for Golang," and that one line carries more weight than it seems. This is a Go library — roughly ~25.4k stars as of 2026-07-09 on gocolly/colly, Apache-2.0 licensed. It is not a command-line tool you download and aim at a URL. You write Go, import the package, wire up a handful of callbacks, and compile the result into one executable.
The mental model is event-driven, and that trips up people coming from a request-and-parse habit. You don't iterate over a response and yank fields out line by line. You attach handlers to a Collector and let the library fire them as it walks pages. OnHTML runs your extraction code every time a matching CSS selector appears. OnResponse hands you the raw response body, which matters when the payload is JSON rather than HTML. OnError catches the requests that fall over. Crawling works the same way: inside a handler for links, you call Visit() on the URLs you find, Colly queues them, and MaxDepth decides how far the thing is allowed to roam. Callbacks, a visit queue, a depth limit, compiled static. No interpreter, no runtime, no headless Chrome sitting in memory.
The callback model, and why it changes how extraction feels
The callbacks are the whole personality of the tool, so they're worth slowing down on. Three of them carried every test I ran.
OnHTML(selector, handler) is the one you'll use most. Register it against .product or article p, and Colly calls your handler once per matched element while it parses the DOM. This is where structured extraction lives, and it reads well — you describe what you want, not the loop that fetches it.
OnResponse(handler) sits a level below and gives you the raw bytes off the wire. When a target returns JSON instead of markup, you never touch the DOM — you unmarshal the body yourself. That single callback is the reason Colly handled a JSON API in my run without parsing a scrap of HTML.
OnError(handler) is the callback everyone forgets until a scraper dies at 3am. It fires when a request fails and hands you the response, so you can read the status code and decide what happens next. A crawler that quietly swallows failures is worse than one that falls over loudly; Colly does neither, and that's a bigger deal than it sounds when the job runs unattended.
Two more features sit on top of those callbacks and matter operationally. MaxDepth caps the crawl, so a link-following collector stops two hops out instead of touring the open web. And the build output is a single static Go binary — compile once, get one file with no runtime dependencies, drop it on a server or into a CI job, run it. If you've ever lost an afternoon to a Python virtualenv on a fresh box, that deployment profile reads as a feature, not a footnote.
Setup — the Go toolchain nobody mentions
The dependency story is short, but it has one real catch, so here it is before you install a thing. The box I tested on had no Go on it at all, and Colly is a Go library, so step zero was putting a toolchain on the machine — I installed Go 1.26.5 through Homebrew. If your team doesn't already live in Go, that's the actual friction. Not the library. The language environment it needs before a single line will compile.
With Go in place, pulling Colly was clean. go get github.com/gocolly/colly/v2 resolved to v2.3.0 with no fuss — no browser, no headless anything, nothing beyond the compiled binary at the end. Set that against Python scrapers that install a parser and then fall apart on the first fetch over a chain of missing extras, and this was pleasantly dull. Dull is a compliment here.
One precision note, stated plainly, because it will absolutely confuse you if you go digging. The latest module on the Go proxy is v2.3.0, published December 2025. The newest tagged release on GitHub is v2.2.0, from March 2025. So the code I tested — v2.3.0 — is ahead of what the repository's Releases page shows. That's a quirk of how Go modules and GitHub tags drift apart over time, not a sign that anything's wrong. Just don't blink when go get and the Releases page tell you different numbers.
Hands-on — the numbers behind "fast"
I ran Colly against a self-contained fixture server built on Go's httptest, plus two public demo sites, so the behavior is reproducible instead of a story I'm telling you. Here's what came back.
| Test | Target | Result |
|---|---|---|
| Static catalog + pagination | local fixture | 12/12 products, recall 1.0 |
| Article extraction | local fixture | title + 3/3 paragraphs |
| Dynamic JSON API | local fixture | 8/8 items via OnResponse, recall 1.0 |
| HTTP 500 handling | local fixture | routed to OnError, status 500 |
Crawl graph (MaxDepth 2) | local fixture | 17 pages |
| Books to Scrape | public demo | 20 products |
| Dynamic page (no JS) | local fixture | 0 cards (expected) |
| Quotes JS (no render) | public demo | 0 (expected) |
Read that top to bottom and the picture holds together. Static extraction was clean — 12 of 12 products off the catalog, all three paragraphs off the article, every bit of it driven by OnHTML selectors. The JSON API test never opened an HTML parser: OnResponse handed over the body, I unmarshaled it, 8 of 8 items came back. The 500 test is the one I lean on hardest, because it's the line between a crawler you can leave running overnight and one you can't — Colly sent the failure to OnError and surfaced the status cleanly, with no crash and no silent drop. On the public Books to Scrape demo it pulled 20 products with no special handling.
The crawl result is the headline, and I want to phrase it carefully. A MaxDepth(2) collector, following links and resolving them to absolute URLs, reached 17 pages across my fixture graph. That's the "fast Go crawler" line finally pinned to an actual page count rather than a mood. Read the wording, though — 17 pages under a depth-2 crawl. The depth number there is my test harness's own counter, describing how I configured the run; I'm not claiming Colly internally guarantees "exactly depth 2, not one link further" as a contract. The honest, checkable statement is this: with depth capped at 2, the crawl traversed the graph and reached 17 pages.
Now the ceiling, which is where the "it's so fast" posts usually go quiet. Colly does not run JavaScript. I aimed it at a JavaScript-rendered fixture and got 0 cards back; I aimed it at the public Quotes to Scrape JS page and got 0 again. That isn't a bug and it isn't a knock. Colly is an HTTP crawler — it downloads and parses HTML, and it never boots a browser to run client-side scripts. Like Scrapy and the other HTTP-first crawlers, if the content you're after only exists after JavaScript executes, Colly hands you an empty result every time, and no amount of raw speed moves that line. Pair it with a renderer, or pick a tool that ships one.
I'll be equally straight about what I did not test, so nobody stretches my results past the evidence. I didn't push the async collector, the rate-limiting and politeness config, proxy rotation, or the queue and storage backends. Those exist in Colly. I ran the extraction-and-crawl core, not the scale-out plumbing. The README advertises throughput north of a thousand requests a second on a single core, but that's the project's own figure — I measured page counts and recall, not throughput, so when I say "fast" I mean the compiled-Go extraction path I actually clocked, not a benchmark against Scrapy I haven't run.
Pros and cons
Pros:
- Full recall on static extraction — 12/12 catalog products and 3/3 article paragraphs through
OnHTML. - Clean JSON handling via
OnResponse, no DOM parsing needed — 8/8 API items. - Correct failure routing — a 500 landed in
OnErrorwith the status exposed, no crash. - A depth-limited crawl reached 17 pages from a single collector.
- One static Go binary, zero runtime dependencies — an excellent deployment and ops profile.
- Permissive Apache-2.0 license.
Cons:
- No JavaScript execution — client-rendered content comes back as 0, full stop.
- Requires a Go toolchain; teams not already in Go pay that setup cost before writing any scraper.
- The latest module (
v2.3.0) sits ahead of the newest tagged release (v2.2.0), which will confuse anyone reading the Releases page. - Output is your own code — Colly gives you callbacks, not a built-in dataset or feed exporter the way Scrapy does.
- Async, rate-limiting, proxy, and queue backends exist but went untested here; "fast" is the extraction path I measured, not a head-to-head throughput number.
Who Colly is for — and who should skip it
Colly fits if you already write Go and you're crawling HTML- or JSON-backed sites at speed. If your definition of a clean deploy is copying one binary onto a box and running it — no interpreter, no virtualenv, no dependency roulette — the tool was built for precisely that disposition. The callback model earns its keep the moment your extraction stops being trivial: OnHTML for structure, OnResponse for raw payloads, OnError for the failures you'd otherwise never see. For a static or API-backed target you crawl on a schedule from CI, it's a strong, low-drama pick.
Skip it, or at least bolt on a second tool, when your targets lean on JavaScript. Colly returned 0 on every client-rendered page I put in front of it, and that's by design, not a setting you can toggle. Skip it too if your team doesn't touch Go and you'd rather not stand up a toolchain just to scrape a few sites — the language commitment is real and it's yours to maintain. And if you want structured data delivered to you rather than parsed by code you own, Colly's callbacks put that work squarely on your side of the fence.
Alternatives — where a managed AI scraping API fits
Colly is a free, open-source library you compile and run yourself. You own the Go code, the callbacks, the crawl logic, and the machine it runs on — and in return you pay nothing per request and keep the whole operation in-house. For a Go shop, that's a defensible answer, and the single-binary deploy is genuinely pleasant.
The two spots where it stops are the two worth comparing against something else. First, JavaScript — Colly won't render it, so anything client-side is off the table unless you bolt on a browser. Second, structure — Colly gives you callbacks and leaves the shaping of clean output to your own code. A managed AI scraping API answers both of those differently. Thunderbit's developer stack handles JS rendering and returns structured data server-side. POST /distill turns a page into clean, LLM-ready Markdown, with dynamic content and anti-bot handled for you. POST /extract returns structured JSON against a JSON Schema you define, with a renderMode you can dial up to full browser rendering when a page needs it. There's a Thunderbit MCP server for AI agents and coding assistants — thunderbit_suggest_fields is free, so you can probe what a page exposes before you commit — and a CLI you can run with npx @thunderbit/thunderbit-cli for the terminal, CI, and cron.
Try Thunderbit for Web Data Extraction
The trade-off isn't better versus worse. It's where the work lives. With Colly you keep rendering (none), parsing, and maintenance inside your own compiled binary, at zero per-call cost, and you babysit it when a site changes shape. With a managed API you hand off JS rendering, anti-bot, and structured output, and you pay per call for the privilege. Small, Go-native, HTML- or JSON-backed targets you're happy to own and maintain? Colly's control and speed win outright. JavaScript-heavy pages, or you'd simply rather receive schema-shaped JSON than write one more callback? That's the case for the managed route. If you want the wider landscape, the best web scraping tools and best web scraping GitHub projects roundups map out where a library like Colly sits next to the browser-based and managed options.
Verdict
Should you use Colly? Yes — if you write Go and you're crawling HTML or JSON at speed, it does what the "fast crawler" reputation promises, and now there are numbers sitting behind the reputation. Full recall on static extraction. Clean JSON through OnResponse. A 500 routed correctly to OnError instead of vanishing. A depth-2 crawl that reached 17 pages. All of it compiled into one static binary with no runtime dependencies, which is about the friendliest deploy story in this whole category.
Size the claims honestly, though. It renders no JavaScript — every client-side page in my run returned 0, and that's permanent, not a config you overlooked. It needs a Go toolchain, so non-Go teams pay a setup tax up front. The module you install (v2.3.0) runs ahead of the newest tagged release (v2.2.0), so don't panic when the pages disagree. And "fast" here means the extraction path I measured, not a throughput benchmark I haven't run. Inside those lines, Colly is a fast, dependable, genuinely deployable Go crawler — and it lives up to the reputation the moment you stop asking it to run JavaScript.
Try Thunderbit for Web Data Extraction Get Started Free
FAQs
Is Colly actually fast, and is there a number behind it? It's fast in the sense that matters for the core path I measured: compiled Go, full recall on static extraction (12/12 catalog products), clean JSON handling, and a depth-2 crawl that reached 17 pages — all out of a single static binary. What I did not run is a throughput benchmark against Scrapy, so treat "fast" as the measured extraction behavior, not a head-to-head speed score.
Can Colly scrape JavaScript-rendered pages? No. Colly is an HTTP crawler — it downloads and parses HTML but never runs a browser. A JavaScript-rendered fixture returned 0 cards, and the public Quotes JS page returned 0 as well. For client-side content you'll need to pair Colly with a renderer or use a tool that ships browser rendering built in.
Do I need to know Go to use Colly?
Yes. Colly is a Go library, not a standalone CLI — you import it, register callbacks (OnHTML, OnResponse, OnError), and compile. The machine I tested on had no Go on it, so setup began with installing a toolchain (1.26.5). If your team isn't already working in Go, that environment is the real setup cost.
Why does the version I install not match Colly's latest GitHub release?
Because the Go module and the GitHub release tag have drifted apart. The latest module on the Go proxy is v2.3.0 (December 2025), while the newest tagged release on GitHub is v2.2.0 (March 2025). I tested v2.3.0. It's a modules-versus-tags quirk, not a broken install.
Is Colly free for commercial use? It's Apache-2.0, which is permissive and commercially friendly. As always, confirm the current license on the repo before you build on it.
