How to Get Data from the Twitter API in JavaScript

Introduction

Twitter is a popular social media platform where users can share their views, and ideas and read posts from other users on a variety of topics. Researchers, developers, bloggers and people from many fields use Twitter for data gathering. You can find a well-written blog on how to create a developer's account for the Twitter API. In this blog, all the parameters and filters are mentioned that one can use with the Standard Search API. It is highly suggested to go through that blog and get a basic idea about Twitter Search API. You can find the official documentation of API here.

We will look at how to use the Twitter API in conjunction with JavaScript in this blog post. Instead of using any package wrapped around Twitter API, we will use the core Fetch library to get desired data from Twitter. We are not using any module that is wrapped around Twitter API, such as twitter-v2 and twitter-lite. The purpose is to make you understand how to use core Twitter API using the Fetch library.

Prerequisites

Before we start using Fetch(), the following things need to be installed:

1. Node
   
   To get and install the node on your system, go to nodejs.dev and download the LTS(Long-term Support) version. After installation, open the terminal on your system and try the following commands and, you will see version numbers.
   
   Here, npm is a package manager for node applications. We need npm to install third-party libraries for our application.

2. Text Editor / IDE (Integrated development environment)
   
   You can use any text editor or IDE. However, I recommend using VSCode which is a free and open-source code editor from Microsoft.

3. Twitter API Keys
   
   To set up the developer’s account and get API keys, follow my previous blog.
   

node -v

npm -v

Extract Data using Fetch

In this section, we will see how to extract data from Twitter using the Fetch module. Fetch is a promise-based function used for sending and receiving data from the server. For this blog, we will focus only on getting data using Fetch.

JavaScript that we use in the browsers (Vanilla JavaScript) has a Fetch module in it. Here, we are using node which is a JavaScript Runtime Environment through which we can run JavaScript on our local machine. This runtime environment does not have Fetch in its core modules. Therefore, we need to install it externally for our projects.

Before installing node-fetch, go to any directory/folder you desire to work in and create a folder with the name of your choice. Now, to install Fetch, open the terminal in the folder you created on your system and write as follows:

npm install node-fetch

There are two ways to handle the fetch() method:

1. Using then() and catch()
2. Using async/await

Using then() and catch()

The fetch method returns a response object that contains a range of useful promise-based methods. Those methods are as follows:

• response.json() returns a promise resolved to a JSON object.
• response.text() returns a promise resolved to raw text.
• response.formData() returns a promise resolved to FormData.
• response.blob() returns a promise resolved to a Blob (a file-like object of raw data).
• response.arrayBuffer()() returns a promise resolved to an ArryBuffer (raw generic binary data).

As the Twitter API returns JSON data, we will only use the response.json() method. The following code is an example to fetch all tweets that contain the python keyword and does not contain any retweets:

const fetch = require('node-fetch')

const token = 'Bearer BEARER_TOKEN' // Replace BEARER_TOKEN with your token
const method = 'GET'
const options = {
  method: method,
  headers: {
    'Content-type': 'application/json',
    Authorization: token,
  },
}

const query = 'python -filter:retweets'

const a = fetch(`https://api.twitter.com/1.1/search/tweets.json?q=${query}`, options)
  .then((response) => response.json())
  .then((json) => console.log(json))
  .catch((error) => {
    console.log(error)
  })

Note: Replace the BEARER_TOKEN word with your Bearer Token that you got from the developers portal.

We can assign any search string and filter to the query. As shown in the code block, we used response.json() to get data in JSON format. Instead of a response, we can use any name that we pass as an argument in the first then() block. As mentioned earlier that response.json() again returns a promise. So, to resolve this promise, we used one more then() block and got the final response as data. In the process of resolving promises, if we get any error, then control will be passed to the catch() block. This block is used for handling any kind of error.

Using async/await

To use async/await, we need to create an async function and put our logic in that function. The following code shows how we can use the async/await with the fetch() method.

const fetch = require('node-fetch')

const token = 'Bearer BEARER_TOKEN' // Replace BEARER_TOKEN with your token
const method = 'GET'
const options = {
  method: method,
  headers: {
    'Content-type': 'application/json',
    Authorization: token,
  },
}

const query = 'python -filter:retweets'

const getData = async () => {
  try {
    const response = await fetch(
      `https://api.twitter.com/1.1/search/tweets.json?q=${query}`,
      options
    )
    const data = await response.json()
    console.log(data)
  } catch (error) {
    console.log('ERROR')
  }
}

getData()

As shown in the code, await will resolve promises and return the response. Here, we have used two awaits as fetch() method and response.json() both return promises.

Note: I have used the arrow function which is the syntax of JavaScript version ES6.

For the common data retrieval tasks, the above code is sufficient. You can use any query parameters and filters as per need and then assign them to the query. If you have used any different methods or filters, please share your expertise/methodology.

---

Conclusion

We can use any of the methods suitable to them. There can be other ways to get data. I have only mentioned one of them here. Many people prefer to use Axios which can be used in the same way as we used the fetch() method to get data using Twitter API. If you want to know about API, you can understand it easily through the "What is API?" blog.