That vagueness is expensive. Price intelligence for e-commerce and quick commerce, market research, travel fare tracking, brand monitoring and training data for language models all depend on fetching pages from the open web at scale. Most of that traffic goes through proxies, and most proxy networks bill for one thing: bandwidth. If you do not understand how many gigabytes your project actually moves, you cannot budget for it, and you certainly cannot negotiate for it.
This piece walks through how that cost really builds up, where the surprises come from, and what to check before you sign up for a plan.
Why the gigabyte is the unit that matters
Residential proxies route your requests through real household internet connections, which is why websites treat them like ordinary visitors. Providers pay for access to that traffic, so they pass the cost on per gigabyte. You are not paying per request, per IP or per hour. You are paying for every byte that crosses the connection, in both directions.
That changes how you should think about the job. Ten thousand requests can cost almost nothing or a small fortune depending on what each request pulls down. The budget question is never just “how many pages?” It is “how many pages, how heavy, how often, and how many times do we have to try?”
The page weight problem
Web pages have become heavy. The HTTP Archive’s Web Almanac 2025 reports that the median home page was 2.86 MB on desktop and 2.56 MB on mobile, and inner pages are not far behind.
Do the multiplication and the numbers get uncomfortable quickly. Load a million median mobile home pages in full and you have moved roughly 2.5 TB of data. Most of that is images, fonts, video, tracking scripts and ad code, none of which your analysts will ever look at.
This is where the biggest savings sit, and they cost nothing. A scraper that requests only the HTML document, or better still the JSON endpoint the page itself calls, often moves a small fraction of what a full browser load does. Teams that switch on a headless browser “just to be safe” for every target frequently multiply their bandwidth bill without noticing, because rendering pulls in every asset the page references.
The hidden multipliers
Even with lean requests, a few factors quietly inflate the total:
- Retries and blocks. Failed requests still consume bandwidth. A target that blocks a meaningful share of your traffic means you pay for the attempt and then again for the retry. A cheaper pool with a poor success rate can end up costing more per usable page.
- Location variants. In India, product availability and delivery options on quick commerce and grocery apps often depend on the pincode. If you need the view from Mumbai, Bengaluru, Delhi and Hyderabad, you are fetching the same page four times.
- Frequency. Moving from a daily price check to an hourly one is a 24x jump in traffic. It is worth asking whether the business decision really changes hour to hour.
- Pagination and discovery. Listing pages, search results and category crawls needed to find the product URLs are often forgotten in estimates, yet they can rival the product pages themselves.
A worked example
Take a mid-sized retail analytics team tracking 5,000 products across two marketplaces, once a day, from five cities. That is 50,000 requests a day, or about 1.5 million a month.
Assume the team has done the sensible thing and trimmed each request to around 400 KB by skipping images and media. That works out to about 600 GB a month. Add 15 percent for retries and you are near 690 GB. At an illustrative rate of $2 per GB, the monthly bill is roughly $1,380. At $4 per GB, it doubles.
Now imagine the same team renders every page in full at the median mobile weight of 2.56 MB. The same 1.5 million requests move about 3,840 GB, more than six times the trimmed figure, before a single retry. Nothing about the data they collect has changed. Only the bill has.
The exact numbers will differ for every project, but the shape of the calculation does not. Run it before you buy, not after the plan runs dry.
A simple budgeting formula
For a first estimate, this is enough:
Monthly GB = requests per run × average KB per request × runs per month × locations × (1 + retry rate) ÷ 1,000,000
Measure the average KB per request on a real sample of your targets rather than guessing. Fetch a hundred pages the way your production scraper will, check the bytes transferred, and use that number. Then add a buffer of 20 to 30 percent for new targets and the sites that redesign without warning.
What to check before you buy rotating residential proxies
Once you know roughly how many gigabytes you need, the plan itself becomes easier to choose. A few things matter more than the headline price:
- Per-GB rate at your real volume. Most providers lower the rate as plan size grows. Price the tier that matches your estimate, not the entry plan.
- Rollover. Web data traffic is lumpy. If unused gigabytes expire at the end of the month, a quiet month is money lost. Plans that carry bandwidth forward are much easier to budget.
- Targeting depth. If you need city-level or ISP-level views, confirm the provider offers them on the product you are buying, not only on a premium tier.
- Success rate on your targets. Test on the sites you actually care about. The cost per successful page is the number that matters.
- Currency. Many proxy vendors price in US dollars, so a budget set in rupees moves with the exchange rate. Prepaid plans make that exposure easier to plan for.
For teams that need broad coverage and want to keep the per-GB model transparent, it is worth comparing providers that publish their rates openly. If you are ready to buy rotating residential proxies, ProxyEmpire, for example, offers a pool of 30M+ residential IPs across 170+ countries with country, region, city and ISP targeting, per-GB pricing and rollover bandwidth on eligible plans, which suits the uneven traffic pattern most data projects have.
Not every target needs residential traffic, either. Sites that do not block datacenter IPs can usually be collected far more cheaply that way, so many teams run a split: datacenter proxies for the easy sources and residential proxies reserved for the ones that push back.
Stay on the right side of the rules
Budget is not the only constraint. India’s Digital Personal Data Protection Act, 2023 puts obligations on anyone processing personal data, and scraping personal information does not sit outside it. Keep collection focused on public, non-personal data such as prices, product listings and published content, respect site terms, and document what you collect and why. A clean, narrow scope also happens to be the cheapest one, because you are not paying to move data you should not be holding.
The bottom line
Bandwidth is the line item that decides whether a web data project stays affordable. The teams that control it are not the ones with the cheapest provider. They are the ones who know the weight of their requests, cut what they do not need, measure success rates honestly, and buy a plan sized to the traffic they actually generate.
Do that homework first and the proxy bill stops being a surprise. It becomes just another number in the budget, which is exactly where it belongs.