Using the Dataflow Kit Public API

If you're looking to extract data from websites, the Dataflow Kit Public API may be just what you need. The API provides a simple, flexible way to extract data from any website and return it in a structured format. In this blog post, we'll take a deeper look at the Dataflow Kit Public API, and provide some example code in JavaScript.

Getting Started

To get started with the Dataflow Kit Public API, you'll need to create an account on the Dataflow Kit website. Once you've created an account, you'll be able to access the API documentation, which provides detailed information about using the API, as well as example code in several programming languages.

Example Code in JavaScript

To make it easy to get started with the Dataflow Kit Public API in JavaScript, we've provided some example code below. These examples will show you how to extract data from a website and return it in JSON format.

Example 1: Extracting Product Data from Amazon

This example shows how to extract product data from Amazon using the Dataflow Kit Public API. In this example, we'll extract the product name, price, and image URL.

const request = require('request')

const apiKey = 'YOUR_API_KEY'
const url = 'https://api.dataflowkit.com/v1/crawl/run'

const formData = {
    api_key: apiKey,
    url: 'https://www.amazon.com/Apple-iPhone-XR-Fully-Unlocked/dp/B07P76GK8J',
    extract: JSON.stringify([
        {
            name: 'Product Name',
            selector: '#productTitle',
            type: 'text'
        },
        {
            name: 'Product Price',
            selector: '#priceblock_ourprice',
            type: 'text'
        },
        {
            name: 'Product Image',
            selector: '#landingImage',
            type: 'attribute',
            attribute: 'src'
        }
    ])
}

request.post({ url, formData }, (err, _, body) => {
    if (err) {
        console.error(err)
    } else {
        console.log(JSON.parse(body))
    }
})

Example 2: Extracting News Headlines from CNN

This example shows how to extract news headlines from CNN using the Dataflow Kit Public API. In this example, we'll extract the headline and the link to the article.

const request = require('request')

const apiKey = 'YOUR_API_KEY'
const url = 'https://api.dataflowkit.com/v1/crawl/run'

const formData = {
    api_key: apiKey,
    url: 'https://www.cnn.com/',
    extract: JSON.stringify([
        {
            name: 'Headline',
            selector: '.cd__headline-text',
            type: 'text'
        },
        {
            name: 'Article Link',
            selector: '.cd__headline .cd__headline-text a',
            type: 'attribute',
            attribute: 'href'
        }
    ])
}

request.post({ url, formData }, (err, _, body) => {
    if (err) {
        console.error(err)
    } else {
        console.log(JSON.parse(body))
    }
})

Conclusion

The Dataflow Kit Public API provides an easy way to extract data from websites, and the examples we've provided in JavaScript make it easy to get started. By following our examples and reading through the API documentation, you'll be able to quickly and easily extract all the data you need from any website on the internet.