Most people file Firecrawl next to the scraping libraries — the pip install, write-a-script, done crowd. That framing is wrong, and the difference matters before you type a single command. Firecrawl self-hosted isn't a library you import; it's a service you operate, and standing it up means running six Docker containers that talk to each other.
I ran the self-hosted stack on a Mac (arm64, Docker via colima) with no cloud key, pointed its /v1/scrape endpoint at a couple of scraping-friendly demo sites, and watched what came back. The short version: the core promise held up — a page went in, clean LLM-ready Markdown came out — but the setup was the heaviest of any tool I've put through this research base. This is a provisional look, not a final rating, and I'll be explicit about what I did and didn't test.
Firecrawl Is a Service, Not a Library
Here's the mental model to fix first. The scraping tools most developers reach for are libraries: you add a dependency, call a function, get HTML or parsed data back inside your own process. Firecrawl self-hosted is a different animal. It's a running platform with its own API, and you talk to it over HTTP.
The official positioning is "the API to search, scrape, and interact with the web at scale," and the product shape is exactly that — pages in, clean Markdown or structured data out. When you self-host, you're not linking against Firecrawl. You're bringing up a docker compose stack and hitting an endpoint, the same way you'd hit any internal microservice.
The stack I ran was six services:
- api — the HTTP surface you actually call
- playwright-service — a headless browser for JavaScript rendering
- redis — queue and cache
- rabbitmq — message broker
- nuq-postgres — a Postgres flavor for job state
- foundationdb — distributed key-value storage
That's a real backend, not a helper script. Redis, RabbitMQ, Postgres, and FoundationDB are all industrial-grade infrastructure in their own right. The payoff is that Firecrawl handles the messy parts of scraping — queueing, rendering, retries — behind one API call. The cost is that you're now operating those six containers. Hold that trade-off in mind; it's the throughline of this whole review.
For reference, I tested against the firecrawl-py 4.32.0 and firecrawl-js 4.30.0 SDKs, pulling the official prebuilt ghcr.io/firecrawl/firecrawl:latest image on 2026-07-09. The repo sits at roughly 148k stars as of that date (call it metadata, not a quality score), under an AGPL-3.0 license — a detail I'll come back to, because it changes the calculus for commercial use.
The Core Test: A Page Becomes Clean Markdown
The whole reason Firecrawl exists is to turn a web page into Markdown an LLM can actually read. So that's the first thing I checked.
I pointed /v1/scrape at books.toscrape.com, a static catalog built specifically for scraping practice. The result: 9,222 characters of clean, LLM-ready Markdown, with the page title All products | Books to Scrape parsed correctly. Not raw HTML dumped into a string — structured Markdown, with headings, links, and image references intact. The kind of output you could drop straight into a retrieval pipeline or feed to a model without a second cleaning pass.
This is Firecrawl's headline strength, and self-hosted delivered it without drama. If your job is "give me the readable substance of this page as Markdown," a static page came back exactly as advertised. That's a genuinely useful primitive, and it's the reason the tool has the following it does.
Worth being precise about scope: I exercised the single-page /v1/scrape path. I did not test /v1/crawl, the multi-page crawler that walks a whole site. That's a separate capability with its own failure modes, and I'm not going to claim it works when I didn't run it.
JavaScript Pages: The Bundled Browser Earns Its Container
A static page is the easy case. The harder question for any scraper is what happens when the content only appears after JavaScript runs — which, on the modern web, is most of the time.
This is where that playwright-service container stops being overhead and starts being the point. I pointed the scraper at quotes.toscrape.com/js/, a version of the demo site that renders its quotes client-side. If Firecrawl just fetched the raw HTML, the quotes wouldn't be there — they don't exist until the browser executes the page's script.
The scrape came back with 1,574 characters of Markdown, and the Einstein quote was in it. That quote is post-JavaScript content: its presence is proof the playwright-service actually rendered the page in a real browser engine before extracting text, rather than grabbing the empty pre-render shell.
So one of the six containers is a headless browser, and it does the job you'd hire it for. That's the concrete justification for the heavier architecture: you're not just paying for containers, you're paying for the ability to render JS-heavy pages without wiring up your own browser automation. For a lot of real-world targets, that's the difference between usable output and empty divs.
When the Target Is Bad: Structured Errors, No Crash
Scrapers spend a surprising amount of their lives pointed at things that don't work — dead hosts, typo'd URLs, servers that hang up. How a tool fails is as telling as how it succeeds.
I fed the API an invalid host on purpose. It returned a structured HTTP 500 and kept running — no stack trace vomited to the client, no container falling over, no hung process. The error came back as a clean response the caller can branch on.
That's the boring, correct behavior you want from something you'd put in a pipeline. A scraper that panics on a bad target is a scraper you can't automate around. This one handed back an error you can catch and move on from. I only tested a single error case, so read this as "handled the one failure I threw at it correctly," not an exhaustive resilience audit — but the one data point was the right result.
Setup Reality: The Heaviest Lift in the Base
Now the part nobody screenshots for the launch tweet. Firecrawl self-hosted was, without exaggeration, the most involved setup of any tool in this research base — and I've stood up a lot of them.
Six containers is the baseline cost. But I also hit two snags on the way up, and I want to be exact about whose fault they were — not Firecrawl's, as it turns out.
Snag one: the from-source build. Building the images from source failed inside my colima VM on a containerd snapshotter error. That's a known-flaky interaction between the build and colima's storage layer — an infrastructure hiccup in my environment, not a bug in Firecrawl. The compose file documents an alternative: use the official prebuilt ghcr.io/firecrawl/* images instead of building locally. I switched to those, and the whole stack came up cleanly. If you're on a standard Docker daemon rather than colima, you may never see this at all; I flag it as an environment caveat, and validating the contributor build on a clean daemon is on my gaps list.
Snag two: the SSRF guard. My first scrapes got blocked by Firecrawl's private-IP / SSRF protection. Why? colima's networking maps public hostnames to 198.18.x.x addresses, which live in a reserved range Firecrawl correctly treats as private — so its security layer did its job and refused to fetch what looked like an internal target. To get around this for local testing only, I set ALLOW_LOCAL_WEBHOOKS=true.
That flag gets copy-pasted into production and causes incidents, so be exact about what it is: the SSRF guard is a feature, not an obstacle. It's what stops a scraping service from being tricked into hitting your internal network. I disabled it because a quirk of colima's DNS made my legitimate public targets look private inside the VM. Do not turn off SSRF protection in a real deployment. If you take one operational note from this review, take that one.
Both snags, to say it plainly, were artifacts of running Docker through colima on a laptop — not defects in the software. On the flip side, the setup weight itself is real and it's Firecrawl's by design. This is not the tool you reach for when you want a quick local script; it's the tool you stand up when you want a rendering-capable scraping service and you're willing to run infrastructure for it.
What I Didn't Test, and What It Doesn't Do
Here's what I didn't cover, and what the tool doesn't give you.
Self-hosted has no Fire-engine. Firecrawl's cloud product includes Fire-engine, its proprietary anti-block layer for getting past bot defenses. Per the project's own SELF_HOST.md, self-hosted instances don't get it. So if you're picturing self-hosted Firecrawl punching through aggressive anti-bot systems out of the box, adjust the picture — that capability lives in the cloud tier, and it wasn't part of what I ran.
The cloud API is untested here. I had no cloud key, so everything above is the self-hosted stack only. The managed cloud service — with Fire-engine, hosted scaling, and the AI features — is a different product, and I'm not going to characterize its performance from the outside. Treat any cloud claim as out of scope for this review.
AI features need a key. The json structured-output format and the /extract endpoint lean on an LLM, which means bringing an OpenAI key or wiring up Ollama. That puts model choice into the bill of materials: before committing to a setup, compare the current API pricing of the providers you could bring. I didn't exercise those paths, so /extract and structured json output sit in the untested column too.
Proxies are a caveat, not a headline. Firecrawl supports proxy configuration, but I'm listing it as a footnote deliberately — it's a knob you can turn, not a reason to pick the tool, and self-hosted still lacks the cloud's anti-block layer regardless.
AGPL-3.0 is a real compliance decision. This one deserves its own beat.
The License: Read AGPL-3.0 Before You Ship
Firecrawl is licensed under AGPL-3.0. That's not a throwaway line at the bottom of a README — it's strong copyleft with a network-use clause, and it can directly affect whether you can build a commercial product on top of a self-hosted instance.
The short of it: standard GPL obligations trigger on distribution. AGPL goes further — the network-use provision means offering the software's functionality to users over a network can count as the kind of use that carries source-availability obligations. If you're embedding self-hosted Firecrawl inside a service your customers reach over the internet, that clause is squarely in scope, and "we never shipped a binary" is not the escape hatch people assume it is.
I'm not your lawyer, and license interpretation depends on how exactly you deploy. But for any commercial recommendation, AGPL-3.0 is a first-class consideration, not fine print. Loop in whoever owns licensing at your company before you build on it. Flagging this is not a knock on Firecrawl — plenty of excellent tools are AGPL — it's just a fact you need on the table early.
Where Thunderbit's Developer Stack Fits
Try Thunderbit for Web Data Extraction
If your actual goal is "page → LLM-ready Markdown" or "page → structured data," and the six-container operational tax plus the AGPL question aren't things you want to own, that's the exact gap Thunderbit's developer stack is built for. Same AI engine behind our 100,000+ extension users, exposed three ways for technical work — with the infrastructure kept on our side of the line.
- Open API (REST).
POST /distillturns a page into clean, LLM-ready Markdown;POST /extractreturns structured data against a JSON Schema you define. JS rendering, anti-bot handling, and dynamic content are handled server-side — no browser container for you to run. ArenderModeflag (none/basic/full) controls how hard it renders, and batch endpoints handle up to 100 URLs for distill. - MCP server. An official Model Context Protocol server, so an AI agent inside Claude or Cursor can scrape mid-task:
thunderbit_suggest_fieldsto plan an extraction (free),thunderbit_distillfor Markdown,thunderbit_extractfor structured data. The agent decides when to pull data without leaving its environment. - CLI.
npx -y @thunderbit/thunderbit-cliruns scrapes from the terminal, scripts, CI, or cron — no browser, no stack to babysit. Pipe it straight into other tools:thunderbit distill "$URL" -f markdown | claude -p "summarise".
The contrast with self-hosted Firecrawl is a clean one. Firecrawl self-hosted gives you full control and full operational ownership: six containers, the setup weight, the AGPL terms, and no Fire-engine for anti-block. Thunderbit's API/MCP/CLI trades that control for a hosted engine that returns schema-matched structured JSON — not just raw Markdown — with the containers, the anti-bot layer, and the copyleft obligations lifted off your plate. Different tools for different appetites for infrastructure.
Here's the trade-off in one view:
| Consideration | Firecrawl self-hosted | Thunderbit dev stack (API · MCP · CLI) |
|---|---|---|
| Deployment shape | Service you operate (6 containers) | Hosted API you call |
| To get running | docker compose up a 6-service stack | API key, then request |
| JS rendering | Bundled playwright-service (you run it) | Server-side, renderMode flag |
| Structured output | Needs LLM key (/extract, json) | POST /extract with JSON Schema |
| Anti-bot layer | None self-hosted (Fire-engine is cloud-only) | Handled server-side |
| License | AGPL-3.0 (network-use copyleft) | Commercial API, no copyleft on your code |
| Best when | You want full control and will run infra | You want Markdown/structured data without ops |
Neither is universally "better." If running the platform is the point for you — full data control, no external dependency, and AGPL fits your situation — self-hosted Firecrawl is a capable, actively maintained choice. If you'd rather make an API call and skip the six-container life, that's the pitch for the Thunderbit stack.
Who Should Actually Self-Host Firecrawl
Strip away the hype and the picture is clear enough to sort by need.
Self-host Firecrawl if you want full control over your scraping infrastructure, you're comfortable operating Redis / RabbitMQ / Postgres / FoundationDB in production, your rendering needs justify the playwright-service container, and AGPL-3.0 works for how you deploy. The core capability is real: I got clean, structured, LLM-ready Markdown out of both a static and a JS-rendered page, and the whole stack ran on prebuilt images.
Look elsewhere if you want a quick local script (this is the heaviest setup in the base, full stop), you need cloud-grade anti-block without operating it yourself (self-host has no Fire-engine), or the AGPL network-use clause collides with your commercial plans. For the "I just need Markdown or structured data from a URL, minus the ops" case, a hosted API like Thunderbit's /distill and /extract covers the same ground without the containers.
My provisional read: strong core, heavy operational commitment, and a license you must clear before building commercially. It earns its place for teams who want to own the whole pipeline — and it asks a lot of everyone else. I'll revisit this once I've run /v1/crawl, exercised /extract with an LLM key, and validated the from-source build on a non-colima daemon; those are the open questions between this and a final verdict.
Try Thunderbit for Web Data Extraction Get Started Free
FAQs
Is self-hosted Firecrawl the same as the cloud version?
No. Self-hosted gives you the core scrape-to-Markdown engine and JavaScript rendering via the bundled playwright-service, but it does not include Fire-engine, the cloud product's proprietary anti-block layer. AI features like the /extract endpoint and json output also require your own LLM key (OpenAI or Ollama). In this review I tested the self-hosted stack only; the cloud API was out of scope.
How many containers does self-hosted Firecrawl actually need? Six: api, playwright-service, redis, rabbitmq, nuq-postgres, and foundationdb. It's a full service stack, not a single binary — which is why it was the heaviest setup of any tool in this research base. Plan for the operational overhead of running message-broker, cache, and database infrastructure, not just a script.
Can Firecrawl handle JavaScript-heavy pages when self-hosted? Yes, in my testing. The bundled playwright-service renders pages in a real browser engine before extraction. I confirmed this on quotes.toscrape.com/js/, where the Einstein quote — content that only exists after JavaScript runs — showed up in the returned Markdown. That rendering capability is exactly why one of the six containers is a headless browser.
Does the AGPL-3.0 license affect commercial use? It can, and you should treat it as a first-order question. AGPL-3.0 is strong copyleft with a network-use clause, which means offering the software's functionality to users over a network can carry source-availability obligations — even if you never distribute a binary. If you plan to build a commercial product on a self-hosted instance, talk to whoever handles licensing at your company before you commit. This review flags the license; it isn't legal advice.
What's the difference between Firecrawl and Thunderbit's developer tools?
Firecrawl self-hosted is a service you operate — six containers you run yourself, with AGPL-3.0 terms and no built-in anti-block layer. Thunderbit's developer stack (Open API, MCP server, CLI) is a hosted engine you call: POST /distill for Markdown, POST /extract for JSON-Schema-structured data, with JS rendering and anti-bot handling server-side and no copyleft obligation on your own code. Firecrawl suits teams who want full infrastructure control; Thunderbit suits those who want the output without the operational load.
