Pick the right IP type per target
Start every new target with static datacenter proxies, the cheapest per request. If the target answers with blocks or captchas, move it to rotating residential proxies, where each request leaves from a home connection. For very large jobs across many sites, unlimited residential plans remove the per GB meter.
Most teams end up with a mix: datacenter IPs for open APIs and simple sites, residential IPs for marketplaces, search pages and travel sites.
Rotation and sessions
For listing pages and search results, rotate the IP on every request: leave the session key out of the user name. For flows that need cookies across several pages, such as pagination behind a filter, add a session key and keep the IP for a few minutes.
Retry failed requests with a new session key instead of the same IP. A good crawler treats a block as a signal to switch IP and slow down that one target, not as a reason to stop.
Keep the cost per page low
Block images, fonts and video in your headless browser when you only need text. A product page that weighs 3 MB with images often needs 300 KB without them, which cuts your residential traffic bill by ten. At $0.35 per GB on the 500 GB pack, a million light pages cost far less than most teams expect.
Set it up step by step
- Step 1Buy a rotating residential pack (from $38.00 for 100 GB).
- Step 2Copy the gateway login from My proxies.
- Step 3Set the proxy in your crawler: http://USER:PASS@gate.proxysite.ink:9000.
- Step 4Add -country-xx to target a market, -session-xx to hold an IP.
- Step 5Retry blocked pages with a new session key.
Questions
Which proxy is best for web scraping?
Rotating residential proxies work on the widest range of sites. Datacenter proxies are cheaper for sites that do not filter by IP type. Many crawlers use both.
How many IPs do I need?
With rotating residential proxies you do not manage IPs: one login reaches the whole pool of 72M+ IPs.
Is scraping public data allowed?
Collecting public data is common practice. Respect each site's terms, avoid personal data you have no basis to process, and keep request rates reasonable.