Exploring the WebScraping.AI API with JavaScript

Are you tired of the tedious task of web scraping? Want a more efficient and effective method to extract data from websites? Look no further than WebScraping.AI!

WebScraping.AI provides a simple RESTful API that allows you to easily scrape data from any website. In this blog post, we'll explore the various API endpoints and provide example code in JavaScript so that you can get started scraping right away.

Getting Started

To start using the API, sign up for a free account on WebScraping.AI. Once you have an account, you can obtain an API key by navigating to your settings page.

Your API key will be a string of characters that you'll use to authenticate your requests to the API.

API Endpoints

WebScraping.AI provides a number of API endpoints that give you different levels of control over how you scrape data from websites. Here are some of the most important endpoints:

Extract HTML content

This endpoint allows you to scrape HTML content from a given URL.

const fetch = require('node-fetch');

const url = 'https://example.com';
const apiKey = 'your-api-key';

fetch(`https://api.WebScraping.AI/scrape/html?url=${url}&api_key=${apiKey}`)
  .then(res => res.json())
  .then(data => console.log(data.html));

Extract text content

This endpoint allows you to scrape text content from a given URL.

const fetch = require('node-fetch');

const url = 'https://example.com';
const apiKey = 'your-api-key';

fetch(`https://api.WebScraping.AI/scrape/text?url=${url}&api_key=${apiKey}`)
  .then(res => res.json())
  .then(data => console.log(data.text));

Extract structured data

This endpoint allows you to scrape structured data from a given URL using CSS selectors.

const fetch = require('node-fetch');

const url = 'https://example.com';
const apiKey = 'your-api-key';
const cssSelector = '.product-name';

fetch(`https://api.WebScraping.AI/scrape/structured?url=${url}&css_selector=${cssSelector}&api_key=${apiKey}`)
  .then(res => res.json())
  .then(data => console.log(data.structured));

Scrape multiple pages

This endpoint allows you to scrape data from multiple pages at once.

const fetch = require('node-fetch');

const urls = [
  'https://example.com/page1',
  'https://example.com/page2',
  'https://example.com/page3'
];
const apiKey = 'your-api-key';

const requests = urls.map(url => {
  return fetch(`https://api.WebScraping.AI/scrape/html?url=${url}&api_key=${apiKey}`)
    .then(res => res.json())
});

Promise.all(requests)
  .then(data => console.log(data));

Conclusion

WebScraping.AI's API makes web scraping easy and efficient. With just a few lines of JavaScript code, you can scrape data from any website you choose. We hope this blog post has provided you with some useful examples to help you get started with the API. Happy scraping!