Web Data Frontier Benchmark

Compare web access APIs (web unblockers / scrape APIs) head-to-head against a fixed suite of 99 real-world, bot-protected URLs spanning 16 industries (Amazon, Walmart, Zillow, Cloudflare/PerimeterX-guarded retail and travel sites, etc. — see Target sites for the full list). Each provider is sent the same URLs; a request passes when the API returns a 2xx status and the response body contains the page's expected text.

Read the full write-up: The Web Scraping Benchmark Problem.

Results

Official run: August 11, 2026 (99 targets × 5 attempts × 15 providers = 7,425 requests; raw data in official_results/benchmark-2026-08-11T22-44-25-322Z.json).

These numbers predate the current adapter configurations. Six adapters ran below the strongest proxy tier their API offers when this run was collected. ScrapingBee is the one to discount hardest: it was fetched from the default datacenter pool with no premium_proxy or stealth_proxy, which is the weakest anti-bot posture of any provider in the table. The adapters have since been corrected; the table has not been re-collected. Treat the ranking as stale until a re-run replaces it.

Rank	Provider	Success rate	Latency score	Passed
1	string	97.0%	9.98s	480/495
2	scrapfly	82.0%	18.35s	406/495
3	context_dev	79.2%	12.68s	392/495
4	firecrawl	78.6%	9.21s	389/495
5	bright	78.0%	26.14s	386/495
6	oxylabs	76.8%	14.67s	380/495
7	zyte	72.7%	14.85s	360/495
8	decodo	70.7%	22.63s	350/495
9	nimble	66.3%	18.41s	328/495
10	scraperapi	64.2%	13.65s	318/495
11	scrapingdog	54.3%	12.51s	269/495
12	browserbase	42.2%	14.19s	209/495
13	zenrows	34.3%	17.60s	170/495
14	scrapingant	30.7%	16.08s	152/495
15	scrapingbee	29.7%	16.97s	147/495

Latency scoring

Latency is an equal-weighted average across target URLs. For a provider that returns verified content, the target score is the nearest-rank 75th percentile of its successful attempt latencies; failed attempts do not contribute a fast response time. For a provider with no verified-content response for a target, the score is the nearest-rank 75th percentile of the successful providers' target scores. If nobody succeeds on a target, the score is the benchmark's 90-second timeout. Nearest-rank p75 uses ceil(0.75 × n): one successful value uses that value, two use the slower value, and five use the fourth value after sorting fastest to slowest.

The CLI's per-provider, per-target output labels the value included in the leaderboard as Resolved latency and identifies whether it came from successful attempts, successful providers, or the timeout.

How it works

• Test suite (src/tests.const.ts): each fixture is { name, url, antibot?, industry?, containsText? }. See Target sites.
• Providers (src/providers/*.ts): one thin adapter per service that fetches a URL through that provider's API and returns { body, statusCode }.
• Runner (src/runner.ts): runs every test × N attempts per provider with bounded concurrency, then aggregates success rate and latency.
• A provider only runs when all of its API keys are present in .env.

Quick start

# 1. Install
npm install            # or: bun install

# 2. Configure keys
cp .env.example .env   # then fill in keys for the providers you want to test

# 3. Run
npm run benchmark      # Node (tsx)
# or
bun run src/cli.ts     # Bun

# Recalculate a report from stored results without sending provider requests
npm run analyze -- --in official_results/benchmark-2026-08-11T22-44-25-322Z.json --out results/recalculated-latency.txt

A smoke test against a single provider and a single fixture:

npm run benchmark -- --providers scrapfly --tests amazon --attempts 1

Benchmark CLI options

Option	Description
--providers <a,b>	Only run these providers (default: all with keys set)
--tests <a,b>	Only run these fixtures by name (default: all)
--attempts <n>	Attempts per test (default: 5)
--concurrency <n>	Parallel requests per provider (default: 2)
--provider-concurrency <n>	Providers to benchmark at once (default: 15)
--out <file>	Results JSON path (default: results/benchmark-<timestamp>.json)
--report-out <file>	Rendered report path (default: results path with .txt)
-h, --help	Show help

Each benchmark run writes the full structured results (every attempt) to JSON and its comparison leaderboard plus per-provider, per-test breakdown to a companion text report. Use --report-out to choose another path.

Results analysis

Recalculate the current scoring from a stored benchmark JSON file without calling providers:

npm run analyze -- --in official_results/benchmark-2026-08-11T22-44-25-322Z.json --out results/recalculated-latency.txt

Option	Description
--in, --input <file>	Benchmark results JSON to analyze
--out <file>	Rendered report path (default: results/analysis-<timestamp>.txt)
-h, --help	Show help

Supported providers

Provider	Required .env var(s)
bright (Bright Data Web Unblocker)	BRIGHT_API_KEY + BRIGHT_ZONE
zyte	ZYTE_API_KEY
scrapfly	SCRAPFLY_API_KEY
scraperapi	SCRAPERAPI_API_KEY
scrapingant	SCRAPINGANT_API_KEY
scrapingbee	SCRAPINGBEE_API_KEY
scrapingdog	SCRAPINGDOG_API_KEY
zenrows	ZENROWS_API_KEY
oxylabs	OXYLABS_WEB_UNBLOCKER_CREDENTIALS (username:password)
firecrawl	FIRECRAWL_API_KEY
massive	MASSIVE_API_KEY
decodo	DECODO_API_KEY
nimble	NIMBLE_API_KEY
context_dev	CONTEXT_DEV_API_KEY
browserbase	BROWSERBASE_API_KEY
string	STRING_API_KEY

Programmatic use

import {
  PROVIDERS,
  makeExecutor,
  runWebAccessBenchmarkSuite,
  WEB_ACCESS_ALL_TESTS,
  WEB_ACCESS_BENCHMARK_CONFIG,
} from "web-data-frontier-benchmark";

const scrapfly = PROVIDERS.find((p) => p.name === "scrapfly")!;
const result = await runWebAccessBenchmarkSuite(WEB_ACCESS_ALL_TESTS, makeExecutor(scrapfly), WEB_ACCESS_BENCHMARK_CONFIG, "scrapfly");
console.log(result.overallSuccessRate);

Contributing

Read CONTRIBUTING.md before opening a pull request. Maintainers approve provider configuration changes from external contributors only after verifying current affiliation with the affected provider's company, unless a repository maintainer explicitly takes responsibility for the change.

Adding a provider

1. Create src/providers/<name>.ts exporting a Provider (name, envKeys, fetch).
2. Register it in the PROVIDERS array in src/providers/index.ts.
3. Document its key in .env.example and the table above.

Notes

• All requests carry a per-attempt timeout (default 90s). The runner enforces it with an AbortController; adapters built on an SDK that takes no signal pass the same value as a client-side socket deadline instead.
• Each provider is sent the strongest anti-bot configuration its public API offers: the best proxy pool the vendor sells, plus any explicit stealth or bypass switch. JS rendering is a separate axis and is not part of the bypass, so it stays off wherever the vendor allows both and every provider is compared on the same artifact. ScrapingBee uses Auto Mode with max_cost: 75, letting the vendor choose the proxy pool and rendering; ZenRows and Decodo render because their adapters predate this note.
• Rendering is mostly not the bypass. Among the seven caller-selected providers — the ones whose adapter sets the flag, so we know what was requested — 12 of the 99 targets were passed by an unrendered provider alone, against 2 passed by a rendering provider alone. Requiring a majority of attempts makes it 15 against 4; requiring all five, 20 against 6. So rendering earns its place on a handful of targets and costs on most. Bot protection keys on IP reputation and TLS fingerprint before it looks at content, so a headless browser mostly buys cost, latency and one more thing to fingerprint — and latency is a scored column here. A more expensive tier is not automatically a stronger one. Providers that choose rendering server-side are excluded from this comparison; they cannot be sorted into either column without guessing.
• In the recorded run, providers fell into two groups, and the distinction matters when reading the numbers:
  • Caller-selected tier. ScrapingBee, ScraperAPI, ScrapingAnt, Scrapingdog, ZenRows, Decodo and Scrapfly expose the proxy pool as a request parameter. Each adapter pinned the top pool, so every attempt started there rather than escalating into it after a block. ScrapingBee now uses Auto Mode and belongs to the server-side escalation group for future runs.
  • Server-side escalation. Zyte, Nimble, Firecrawl, Bright Data, Oxylabs, Context.dev, Browserbase and String decide the bypass strategy themselves. The adapter asks for the strongest mode it can name and the vendor picks the rest, so the configuration is not fully observable from this repo. Because String is in the second group, its result reflects our own server-side routing and is not parameter-comparable with a pinned-tier provider. Read it as the service's default behaviour.
• See the per-file comments for the exact request shape and the credit cost it implies.
• Running the full suite across many providers makes real, billable API calls. Start with --attempts 1 and a small --tests subset.
• Yeah much vibecoding (so catch the AI-isms) but we did read all of the code + verify :)

Target sites

The suite is 99 domains, one target URL each, defined in src/tests.const.ts. Every provider is sent this same list, in the order below. Site is the fixture name --tests accepts. Industry is the fixture's industry field, an editorial grouping of the suite by the kind of data behind the URL; every target carries exactly one. Anti-bot is the fixture's antibot field; Not classified means the fixture carries no antibot value, not that the site is unprotected.

Vendor spread: Akamai Bot Manager 20, DataDome 20, Cloudflare 15, PerimeterX / HUMAN 9, In-house / custom 9, AWS WAF 7, Kasada 3, Fastly Bot Management 1, Imperva Incapsula 1, Temu (in-house) 1, Ticketmaster (in-house) 1, Not classified 12.

Industry spread: Retail & ecommerce 16, Fashion & luxury 11, Marketplaces & classifieds 10, Travel 9, News & finance 9, Social media 8, Real estate 7, Reviews & local 6, Jobs & hiring 5, Grocery & food 5, Tickets & events 4, Developer & research 3, Search engines 2, Gaming & betting 2, Health & pharmacy 1, Government 1.

#	Site	Domain	Target URL	Industry	Anti-bot
1	allegro	allegro.pl	link	Marketplaces & classifieds	DataDome
2	Amazon	www.amazon.com	link	Retail & ecommerce	AWS WAF
3	canadagoose	www.canadagoose.com	link	Fashion & luxury	Kasada
4	indeed	www.indeed.com	link	Jobs & hiring	Cloudflare
5	lowes	www.lowes.com	link	Retail & ecommerce	Akamai Bot Manager
6	safeway	www.safeway.com	link	Grocery & food	Imperva Incapsula
7	walmart	www.walmart.com	link	Retail & ecommerce	PerimeterX / HUMAN
8	zillow	www.zillow.com	link	Real estate	PerimeterX / HUMAN
9	saksfifthavenue	www.saksfifthavenue.com	link	Fashion & luxury	DataDome
10	neimanmarcus	www.neimanmarcus.com	link	Fashion & luxury	DataDome
11	g2	www.g2.com	link	Reviews & local	DataDome
12	aa	www.aa.com	link	Travel	Akamai Bot Manager
13	asda	www.asda.com	link	Grocery & food	Cloudflare
14	lululemon	shop.lululemon.com	link	Fashion & luxury	Akamai Bot Manager
15	hyatt	www.hyatt.com	link	Travel	Kasada
16	macys	www.macys.com	link	Fashion & luxury	Akamai Bot Manager
17	etsy	www.etsy.com	link	Marketplaces & classifieds	DataDome
18	yelp	www.yelp.com	link	Reviews & local	DataDome
19	zara	www.zara.com	link	Fashion & luxury	Akamai Bot Manager
20	louisvuitton	us.louisvuitton.com	link	Fashion & luxury	Akamai Bot Manager
21	autozone	www.autozone.com	link	Retail & ecommerce	DataDome
22	homedepot	www.homedepot.com	link	Retail & ecommerce	Akamai Bot Manager
23	ashleyfurniture	www.ashleyfurniture.com	link	Retail & ecommerce	PerimeterX / HUMAN
24	autotrader	www.autotrader.com	link	Marketplaces & classifieds	Akamai Bot Manager
25	booking.com	www.booking.com	link	Travel	AWS WAF
26	carters	www.carters.com	link	Fashion & luxury	PerimeterX / HUMAN
27	databricks	www.databricks.com	link	Jobs & hiring	Not classified
28	emag	www.emag.ro	link	Retail & ecommerce	AWS WAF
29	expedia	www.expedia.com	link	Travel	Akamai Bot Manager
30	glassdoor	www.glassdoor.com	link	Jobs & hiring	Cloudflare
31	marketwatch	www.marketwatch.com	link	News & finance	DataDome
32	barrons	www.barrons.com	link	News & finance	DataDome
33	monster	www.monster.com	link	Jobs & hiring	DataDome
34	mouser	www.mouser.com	link	Retail & ecommerce	Akamai Bot Manager
35	nytimes	www.nytimes.com	link	News & finance	DataDome
36	ralphlauren	www.ralphlauren.com	link	Fashion & luxury	PerimeterX / HUMAN
37	realtor	www.realtor.com	link	Real estate	AWS WAF
38	tripadvisor	www.tripadvisor.com	link	Travel	DataDome
39	wsj	www.wsj.com	link	News & finance	DataDome
40	bloomberg	www.bloomberg.com	link	News & finance	PerimeterX / HUMAN
41	reuters	www.reuters.com	link	News & finance	DataDome
42	alibaba	www.alibaba.com	link	Retail & ecommerce	Not classified
43	coupang	www.coupang.com	link	Retail & ecommerce	Akamai Bot Manager
44	ebay	www.ebay.com	link	Marketplaces & classifieds	Akamai Bot Manager
45	target	www.target.com	link	Retail & ecommerce	PerimeterX / HUMAN
46	nike	www.nike.com	link	Fashion & luxury	Akamai Bot Manager
47	stockx	stockx.com	link	Marketplaces & classifieds	Cloudflare
48	cargurus	www.cargurus.com	link	Marketplaces & classifieds	DataDome
49	cars.com	www.cars.com	link	Marketplaces & classifieds	Cloudflare
50	crunchbase	www.crunchbase.com	link	Reviews & local	Cloudflare
51	verizon	www.verizon.com	link	Retail & ecommerce	Akamai Bot Manager
52	att	www.att.com	link	Retail & ecommerce	Akamai Bot Manager
53	goodrx	www.goodrx.com	link	Health & pharmacy	PerimeterX / HUMAN
54	LinkedIn	www.linkedin.com	link	Social media	In-house / custom
55	X (Twitter)	x.com	link	Social media	In-house / custom
56	Instagram	www.instagram.com	link	Social media	In-house / custom
57	Ticketmaster	www.ticketmaster.com	link	Tickets & events	Ticketmaster (in-house)
58	Capterra	www.capterra.com	link	Reviews & local	Cloudflare
59	Reddit	www.reddit.com	link	Social media	In-house / custom
60	Google Search	www.google.com	link	Search engines	In-house / custom
61	YouTube	www.youtube.com	link	Social media	In-house / custom
62	TikTok	www.tiktok.com	link	Social media	In-house / custom
63	Airbnb	www.airbnb.com	link	Travel	DataDome
64	Best Buy	www.bestbuy.com	link	Retail & ecommerce	Akamai Bot Manager
65	Redfin	www.redfin.com	link	Real estate	AWS WAF
66	Foot Locker	www.footlocker.com	link	Fashion & luxury	Kasada
67	SeatGeek	seatgeek.com	link	Tickets & events	DataDome
68	Facebook	www.facebook.com	link	Social media	In-house / custom
69	GOAT	www.goat.com	link	Marketplaces & classifieds	Cloudflare
70	Yahoo Finance	finance.yahoo.com	link	News & finance	Not classified
71	DigiKey	www.digikey.com	link	Retail & ecommerce	Cloudflare
72	Rightmove	www.rightmove.co.uk	link	Real estate	Not classified
73	Instacart	www.instacart.com	link	Grocery & food	Not classified
74	Pinterest	www.pinterest.com	link	Social media	In-house / custom
75	ZipRecruiter	www.ziprecruiter.com	link	Jobs & hiring	Cloudflare
76	Vinted	www.vinted.fr	link	Marketplaces & classifieds	DataDome
77	Idealista	www.idealista.com	link	Real estate	DataDome
78	Leboncoin	www.leboncoin.fr	link	Marketplaces & classifieds	DataDome
79	StubHub	www.stubhub.com	link	Tickets & events	AWS WAF
80	Skyscanner	www.skyscanner.net	link	Travel	PerimeterX / HUMAN
81	Congress.gov	www.congress.gov	link	Government	Cloudflare
82	TradingView	www.tradingview.com	link	News & finance	Not classified
83	CoinMarketCap	coinmarketcap.com	link	News & finance	Not classified
84	Apartments.com	www.apartments.com	link	Real estate	Akamai Bot Manager
85	Zoopla	www.zoopla.co.uk	link	Real estate	Cloudflare
86	Trustpilot	www.trustpilot.com	link	Reviews & local	AWS WAF
87	Yellow Pages	www.yellowpages.com	link	Reviews & local	Cloudflare
88	Kayak	www.kayak.com	link	Travel	Not classified
89	Allmenus	www.allmenus.com	link	Grocery & food	Not classified
90	temu	www.temu.com	link	Retail & ecommerce	Temu (in-house)
91	delta	www.delta.com	link	Travel	Akamai Bot Manager
92	kroger	www.kroger.com	link	Grocery & food	Akamai Bot Manager
93	axs	www.axs.com	link	Tickets & events	Cloudflare
94	roblox	www.roblox.com	link	Gaming & betting	Fastly Bot Management
95	draftkings	sportsbook.draftkings.com	link	Gaming & betting	Akamai Bot Manager
96	bing	www.bing.com	link	Search engines	Not classified
97	github	github.com	link	Developer & research	Not classified
98	stackoverflow	stackoverflow.com	link	Developer & research	Cloudflare
99	arxiv	arxiv.org	link	Developer & research	Not classified

License

MIT