Request Customization
Scrapingdog offers various functionalities and features like changing the origin of the request, passing custom headers, JS rendering, etc.
By passing these additional parameters you can customize the request to our web scraping api according to your demands.
dynamic
You can use this parameter for JS rendering. It is a boolean so you have to use it like this dynamic=true
. Our servers will render JavaScript for that page and will return you the resulting HTML code.
The cost of using this param is 5 credits with normal proxies and if you want to use it with premium residential proxies then 25 credits.
premium
When you want to use the premium residential proxy instead of the normal rotating proxy then you can use this param. This is also a boolean so you can access it by setting it to premium=true
. This feature will use a USA residential proxy by default.
The cost of using this feature is 10 credits and 25 credits if used with JS rendering dynamic=true
custom_headers
When you set custom_header=true
our server allows you to pass your own headers.
wait
wait parameter is a time in milliseconds that can be used with the combination of dynamic=true
in order to wait and load the website completely.
The value of wait can be any whole number between 0 and 35000.
Example - wait=5000
country
This parameter helps you access the geotargeting feature. You can pass any country's ISO code in order to send a request from that location. Like country=gb
will use United Kingdom proxies.
session_number
You can this parameter if you want to reuse the same proxy for multiple requests. You can set any string value to session_number while making the request. Cost does not change when you use this feature.
image
You can use this parameter if you want to scrape image URLs.
Type - Boolean
Last updated