Apify
DevelopmentExploring Apify Public API with JavaScript
Apify is a platform that provides developers with a host of powerful tools to automate their web projects. The platform has a comprehensive API that enables developers to integrate Apify into their codebase and automate different aspects of their projects.
In this blog, we will explore the Apify public API with some example code snippets written in Javascript.
Getting Started with Apify Public API
To get started with the Apify public API, you'll need to create an account on the Apify platform and generate an API token. Once you have the token, you can use it to authenticate your requests to the API.
const ApifyClient = require('apify-client');
// Initialize the Apify client
const apifyClient = new ApifyClient({
token: "yourAPItoken"
});
Crawling Data with Apify
Apify provides a powerful web crawling tool that enables developers to extract data from any website. With the Apify public API, you can easily integrate web crawling into your Javascript codebase. Here's an example of how to use the Apify web crawler to extract data from a website:
// Set the crawler start URL
const startUrl = 'https://www.example.com/';
// Initialize the crawler
const crawler = new apifyClient.CheerioCrawler({
startUrls: [startUrl],
maxRequestsPerCrawl: 100,
// Extracts links to other pages
handlePageFunction: async ({ request, $ }) => {
const links = $('a');
const linkObjects = [];
links.each((index, element) => {
const href = $(element).attr('href');
linkObjects.push({
url: href,
});
});
// Returns an array of links to follow
return linkObjects;
},
// Extracts data from the page
handleSuccessFunction: async ({ request, $, body }) => {
const title = $('title').text();
const metaDescription = $('meta[name="description"]').attr('content');
console.log(`Title: ${title}`);
console.log(`Meta description: ${metaDescription}`);
},
});
// Run the crawler
crawler.run();
Running an Apify Actor
Apify actors are pre-built scripts that contain a set of instructions for performing a specific task. With the Apify public API, you can easily execute an actor and get the output in your Javascript code. Here's an example of how to use the Apify actor to download data from a website:
// Set the actor input parameters
const input = {
url: 'https://www.example.com/',
outputFileName: 'example.html',
};
// Run the actor
const run = await apifyClient.act('example-actor-id', input);
// Get the actor run output data
const { output } = await apifyClient.run.getActOutput(run.id);
// Output the downloaded content
console.log(output);
Conclusion
The Apify public API is a powerful tool that provides developers with endless opportunities to automate their web projects. In this blog, we explored the Apify public API with some example code snippets written in Javascript. With these examples, you can start integrating Apify into your Javascript codebase and automate different aspects of your web projects.
Related APIs
