Failed Scrapes and Your Bill: What ScraperAPI, Firecrawl and Bright Data Document

Originally published at AgentStack Notes. Last fact-checked: November 10, 2026.

Disclosure: This article was drafted with AI assistance and checked against each vendor's official documentation. We may receive a commission if you sign up through the Bright Data †, ScraperAPI † or Firecrawl † links marked † in this article, at no extra cost to you. We have not run production workloads on any of the three. See our Advertising Disclosure.

Your agent works through a long list of URLs. Some are dead, some block it, some time out. Do those requests still use up credits? The answer differs by vendor, and in several cases the docs do not say. Here is what each one publishes.

Note on experience: we have not tested billing behaviour ourselves. Pages were read through a web-fetch tool that summarises them, on November 10, 2026. Check the live docs before relying on any rule here. † marks a referral link; "not stated" means the page we read did not say.

Side by side

ScraperAPI

The credits page says: "We only charge for successful requests (200 and 404 status codes)." The status-code page says a 500 is not charged, and that requests failing after 70 seconds of retrying are not charged. It gives no billing wording for 403, 429, 400 or 401, and does not list 410. The docs recommend a connection timeout of at least 60 seconds (70 recommended) so ScraperAPI can retry through different proxies. Cancelling a request yourself after 70 seconds is charged.

Who it suits, per the docs: people who want a clear 200/404 rule and a response header (sa-credit-cost) that reports each request's cost. Where the docs stop: billing for blocked-style responses (403 and others) is not stated. ScraperAPI link †

Firecrawl

Billing turns on whether Firecrawl returned a document. Error-status pages "such as 403 Forbidden or 404 Not Found" still cost 1 credit, and "Firecrawl returned no document: 0 credits." Some operations charge even without a document: a threat-protection scan (2 credits per scanned URL), the prompt-injection check (5 credits once it has run), a lockdown cache miss (1 credit), and browser sessions billed per minute with a one-minute minimum. An escalated retry on the enhanced proxy is not charged separately. Credits are billed as each page of a batch or crawl finishes, so charges can show up after submission.

Who it suits, per the docs: anyone who wants the blocked-page case spelled out, since a 403 that returns a document is documented as 1 credit. Where it needs care: the docs advise checking metadata.statusCode and not retrying URLs that are consistently blocked. Firecrawl link †

Bright Data

The Web Unlocker introduction says you are "charged only for successful requests to your target domain." It does not define what counts as successful and does not mention HTTP status codes, so whether a 404 or a 403 is billable is not stated there. We did not find billing wording for failed requests on the MCP pages we read.

Who it suits, per the docs: the stated principle is favourable (successful requests only), but with no definition of success you would have to measure it yourself. Bright Data link †

What to do with this

• A 404 is not free on the two vendors that say so. If your agent walks a list of URLs that may be dead, ScraperAPI and Firecrawl both document charging for it. Filter or validate URLs first.
• Blocked is the ambiguous case. Firecrawl documents a charge when a document returns with a 403; ScraperAPI and Bright Data do not say. Test a small batch and compare your dashboard to your logs before scaling.
• Do not retry blindly. Both Firecrawl's docs and ScraperAPI's retry design suggest the vendor already retries; a second retry loop of your own can multiply cost on pages that never succeed.
• Check the per-request cost signal. ScraperAPI exposes sa-credit-cost in a response header; Firecrawl exposes metadata.statusCode. Log both.

What we could not confirm

• ScraperAPI's billing for 403, 429, 400, 401 and 410 responses.
• Bright Data's definition of a "successful request" and treatment of 404s, plus billing rules specific to the Web MCP free tier.
• Whether the vendors' dashboards match these rules in practice. We have not measured it.
• Success rates and speed of any of the three.

Sources (checked November 10, 2026): ScraperAPI Credits and Requests Costs, ScraperAPI API Status Codes, ScraperAPI FAQ: Low Success Rates, Firecrawl billing, Bright Data Web Unlocker introduction.

More: the original article.