# Documentation

Using Scrapingdog is very straightforward. Simply forward the URL you wish to scrape to the API, along with your API key, and the API will provide you with the HTML response from that designated URL.

Scrapingdog uses API keys for request authentication. To leverage the API, you must register for an account and incorporate your exclusive API key into each request.

API URL is: `https://api.scrapingdog.com/`

### Points to Remember

* Each request will be retried until it can be successfully completed (up to 60 seconds). Remember to set your timeout to 60 seconds to ensure this process goes smoothly. In cases where every request fails in 60 seconds, our API will return a `410` error, you may retry the request and you will not be charged for the unsuccessful request (you are only charged for successful requests, 200 and 404 status codes). Make sure to catch these errors! They will occur on roughly 1-2% of requests for hard-to-scrape websites.
* There is no overage allowed on the free plan, if you exceed 1000 requests per month on the free plan, you will receive a 403 error.
* Each request will return a string containing the raw HTML from the page requested, along with any headers and cookies.
* If you exceed your plan concurrent connection limit, the API will respond with a 429 status code, this can be solved by slowing down your request rate or upgrading to a bigger plan.

### Error Codes

| Error Codes | Description                                                                                                                                                           |
| ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 200         | Successfull Request                                                                                                                                                   |
| 410         | Request timeout                                                                                                                                                       |
| 404         | URL is wrong                                                                                                                                                          |
| 202         | <p>Your request is accepted and the scraping is still going on.<br><br><mark style="color:red;">This status code only appears in the LinkedIn Scraping API</mark></p> |
| 403         | Request Limit Reached.                                                                                                                                                |
| 429         | Concurrent connection limit reached.                                                                                                                                  |
| 401         | API Key is wrong.                                                                                                                                                     |
| 400         | Request failed.                                                                                                                                                       |

{% hint style="info" %}
You are only charged for successful requests with status codes <mark style="color:green;">`200`</mark> or <mark style="color:blue;">`404`</mark>.
{% endhint %}

### Credit Cost of APIs

{% hint style="info" %}
GWS is a General Web Scraping API
{% endhint %}

| API                                                  | Credit Cost                                                                                                                                                                              |
| ---------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| GWS with rotating proxy                              | 1                                                                                                                                                                                        |
| GWS with premium proxy                               | 10                                                                                                                                                                                       |
| GWS with JS rendering and rotating proxy             | 5                                                                                                                                                                                        |
| GWS with JS rendering and premium proxy              | 25                                                                                                                                                                                       |
| GWS with country                                     | 10                                                                                                                                                                                       |
| GWS with country and premium proxy                   | 10                                                                                                                                                                                       |
| GWS with JS rendering and country with premium proxy | 25                                                                                                                                                                                       |
| Amazon Search API                                    | 1                                                                                                                                                                                        |
| Amazon Product API                                   | 1                                                                                                                                                                                        |
| Amazon Reviews API                                   | 100                                                                                                                                                                                      |
| Linkedin Scraping API                                | <p>50 if the profile is not protected and 100 if it is.<br><br>For more info <a href="https://www.scrapingdog.com/blog/how-to-use-scrapingdog-linkedein-scraper-api/">read this</a>.</p> |
| Linkedin Jobs API                                    | 5                                                                                                                                                                                        |
| Twitter Scraper API                                  | 5                                                                                                                                                                                        |
| Bing Scraper API                                     | 5                                                                                                                                                                                        |
| Indeed Scraper API                                   | 1                                                                                                                                                                                        |
| Youtube Scraper API                                  | 5                                                                                                                                                                                        |
| Youtube Transcript API                               | 1                                                                                                                                                                                        |
| Youtube Channel API                                  | 5                                                                                                                                                                                        |
| Youtube Comment API                                  | 5                                                                                                                                                                                        |
| Walmart Scraper API                                  | 5                                                                                                                                                                                        |
| Flipkart Scraper API                                 | 5                                                                                                                                                                                        |
| eBay Scraper API                                     | 5                                                                                                                                                                                        |
| Screenshot API                                       | 5                                                                                                                                                                                        |
| Yelp Scraper API                                     | 1                                                                                                                                                                                        |
| Bing Scraper API                                     | 5                                                                                                                                                                                        |
| Google Lens API                                      | 5                                                                                                                                                                                        |
| Google Images API                                    | 10                                                                                                                                                                                       |
| Google Maps API                                      | 5                                                                                                                                                                                        |
| Google Shopping API                                  | 10                                                                                                                                                                                       |
| Google Scholar API                                   | 5                                                                                                                                                                                        |
| Google Trends API                                    | 5                                                                                                                                                                                        |
| Google Jobs API                                      | 5                                                                                                                                                                                        |
| Google Hotels API                                    | 5                                                                                                                                                                                        |
| Google AI Mode API                                   | 10                                                                                                                                                                                       |
| Google Immersive Product API                         | 5                                                                                                                                                                                        |
| Google Patents API                                   | 5                                                                                                                                                                                        |
| Google Finance API                                   | 5                                                                                                                                                                                        |
| Google Videos API                                    | 5                                                                                                                                                                                        |
| Google Shopping API                                  | 5                                                                                                                                                                                        |
| Google News API                                      | 5                                                                                                                                                                                        |
| Google Product API                                   | 5                                                                                                                                                                                        |
| Google SERP API                                      | 5 & 10                                                                                                                                                                                   |
| Universal Search API                                 | 20                                                                                                                                                                                       |

### Authentication

You can authenticate to our API by providing your API key, which you can obtain in the [member area](https://api.scrapingdog.com/login).
