ScrapingBee
Data AccessTired of getting blocked while scraping the web? You have to handle Javascript rendering, Headless browsers, Captcha solving, and proxy management. ScrapingBee does all of the above in real-time with a simple API call.
📚 Documentation & Examples
Everything you need to integrate with ScrapingBee
🚀 Quick Start Examples
// ScrapingBee API Example
const response = await fetch('https://www.scrapingbee.com/', {
method: 'GET',
headers: {
'Content-Type': 'application/json'
}
});
const data = await response.json();
console.log(data);ScrapingBee - API Documentation
ScrapingBee offers a simple and easy-to-use API that allows you to scrape data from any website using JavaScript. With our API, you can retrieve data in any format including JSON, CSV, and XML.
How to Get a ScrapingBee API Key
ScrapingBee requires an API key, available instantly on signup:
- Create an account at https://www.scrapingbee.com/ (dashboard at https://app.scrapingbee.com).
- Verify your email and open the dashboard.
- Copy your API key from the dashboard home.
Free trial: New accounts get 1,000 free API credits and no credit card is required to start. Credits are consumed per request, with more spent when you enable JavaScript rendering or premium proxies.
The key is passed as the api_key query parameter; the page you want to scrape goes in url:
const apiKey = 'YOUR_API_KEY';
const target = encodeURIComponent('https://example.com');
const url = `https://app.scrapingbee.com/api/v1/?api_key=${apiKey}&url=${target}`;
fetch(url)
.then(res => res.text())
.then(html => console.log(html));
Add &render_js=true for JavaScript-heavy pages or &premium_proxy=true to route through residential IPs. The base endpoint is https://app.scrapingbee.com/api/v1/.
Making Requests
To make a request to our API, you will need to include your API key as the api_key parameter. There are several endpoints available that you can use to scrape data from different websites. Here are some examples:
Basic Request
This is a basic request that scrapes the HTML content of a webpage.
const apiUrl = 'https://app.scrapingbee.com/api/v1/';
const apiKey = 'YOUR_API_KEY_HERE';
const websiteUrl = 'https://www.example.com/';
fetch(`${apiUrl}${websiteUrl}?api_key=${apiKey}`)
.then(response => response.text())
.then(data => console.log(data));
Advanced Request
This request uses advanced options to scrape a list of articles from a website.
const apiUrl = 'https://app.scrapingbee.com/api/v1/';
const apiKey = 'YOUR_API_KEY_HERE';
const websiteUrl = 'https://www.example.com/articles';
const options = {
javascript: true,
country_code: 'us',
premium_proxy: true
};
fetch(`${apiUrl}${websiteUrl}?api_key=${apiKey}&${new URLSearchParams(options)}`)
.then(response => response.json())
.then(data => console.log(data));
Conclusion
ScrapingBee's API provides a simple and efficient way to scrape data from any website. With our easy-to-use API, you can retrieve data in any format you want. Plus, our API is reliable and fast, making it the perfect solution for all your scraping needs.
