Building multi-threaded applications in NodeJS.

NodeJS, a runtime that allows us to execute Javascript code on the server, NodeJS was created by Ryan Dahl and its first version was published in 2009. NodeJS is asynchronous and event-driven, it comes embedded with a built-in Event Loop that allows it to achieve concurrency while using only a single thread. Most of the Javascript code we write is executed using only a single CPU thread, most of the asynchronous behavior we observe when using Javascript comes from how the event loop works.

Today's article will not focus on the NodeJS concurrency model, rather today's article will focus on how we can utilize multiple threads in a NodeJS application, being a server-side runtime NodeJS gives us access to the CPU cores available on a computer but we don't need to configure anything to start writing multi-threaded applications with NodeJS. NodeJS comes baked in with a child_process module that allows us to write multi-threaded applications quite easily.

For this piece I'll be using the Node version 16.X for the examples in this article so if you are running a much older version of NodeJS then I'd suggest that you take your time to update your Node version either manually or using NVM. To use the child_process module we first need to import it into our Javascript file.

const childProcess = require('child_process');

The child process module provides the ability to spawn subprocesses. What the heck is a process and a sub-process anyway? When we use the node index.js command to execute a NodeJS application it creates a process, the word process is arbitrary and it refers to both the program and the execution context in which it is executed. All this is done using a single processor core on a single thread. The child process is another execution context or thread that our code can utilize to execute CPU-intensive tasks without blocking the main thread. The child process provides several ways to handle creating subprocesses. The child_process has the following methods defined on it that enable us to create child processes.

• exec
• execFile
• fork
• spawn

Let's dive into what we have for the day starting with the exec method.

Exec

The exec function spawns a shell and then executes a command within that shell, we can execute any shell commands we like inside the exec function, the syntax is as follows;

child_process.exec(command[, options][, callback])

Where command is the command we want to execute, options are an array of arguments we want to pass the shell with the command and the last is a callback function that will be fired when the process terminates. The callback function accepts three arguments, the first is any error NodeJS encounters while trying to execute the command, the second argument is the value returned from executing the command and the last is the stderr output of the child process, let's see an example of how the exec function works.

child_process.exec('node', ['-v'], (error, stdout, stderr) => {
  if(error) {
    console.log(error);
    return;
  }

  if (stderr) {
    console.log(stderr);
    return;
  }

  console.log(stdout) // v16.16.0
})

In the example we have above the exec function is used to check the node version the computer is currently running on. The first argument we pass to the exec function is the node command, the next argument is an array of the variables we want to pass to the command, and in this instance, we pass the -v which returns the node version. The last argument is the callback function which accepts three arguments. Inside the callback function, we check first to see if there is an error which we log out to the console if it exists. We also check the stderr and log it out the console if it stores a value otherwise if everything went fine then we just log out the result of the command which in this instance is the node version.

execFile

Another way to do this is to use the execFile method on the child process. This method is similar to the exec but it is generally considered to be safer against malicious input because it does not create a shell process like exec rather it executes our command in a child process.

child_process.execFile('nodeFile', ['-v'], (error, stdout, stderr) => {
  if(error) {
    console.log(error);
    return;
  }

  if (stderr) {
    console.log(stderr);
    return;
  }

  console.log(stdout) // 16.16.0
})

Fork

The fork method is used to create a new child process that can exchange messages back and forth with the main process that spawned it. The child process created is independent of the parent and contains its execution context and memory, it only maintains an IPC communication channel established between itself and the parent Process that created it. The syntax for this method is as follows;

child_process.fork(modulePath[, args][, options])

Where the modulePath is the path to the javascript file we want to execute, while the second is an optional string argument, and the third is an optional object with options to configure the child process, let's see an example of this; create a file name child.js and dump the following contents inside.

const array = [1, 2, 4, 5, 6];

function checkEven (array) {
  const filtered = array.filter(
    (value) => value % 2 == 0
  );

  const every = filtered.every(
    (value) => value % 2 == 0
  )

  return {
    filtered,
    every
  };
}

process.send(checkEven(array))
process.kill(process.pid);

We have a simple checkEven function that filters an array of numbers to get the even numbers, then we check if the new array created only contains even numbers, and we return the result of both expressions inside an object from the function. Then we call process.send to send a message up to the parent of this process, and in this case, the message is the result of calling the checkEven function, at the bottom we call process.kil() and pass in the id of this child process to kill it when we are done. Now create another file app.js and dump in the following;

const {fork} = require('child_process');
const child = fork(__dirname + "/child.js");


child.on('message', (message) => {
  console.log(message); // { every: true, filtered: [2, 4, 6] }
})

Inside this file, we destructure the fork method from the child_process module and create a new child process using the fork method we pass the path to the child.js file which we created earlier. We listen for the message event which is fired anytime the child sends a message to us then we logout the message to the console. We can send a message to the child by using the child.send(message) method.

Spawn

This method creates a new process which is similar to the process created by calling the fork method, however, the process created does not maintain any TCP communication channel with the parent, however, we can listen for events on the process and attach a listener to them. The syntax for this command is as follows;

child_process.spawn(command[, args][, options])

This spawn method is a bit similar to the exec and execFile in how it works, but to create a shell we need to specify the shell option in the options object. Instead of providing a callback function, we listen for the event to get the result of the command, or errors encountered while trying to run our command.

const { spawn } = require('child_process');
const child = spawn('node', ['-v'], { shell: true });

child.stdout.on('data', (data) => {
    console.log(`stdout: ${data}`); // v16.16.0
});

child.stderr.on('data', (data) => {
    console.error(`stderr: ${data}`);
});

child.on('close', (code) => {
    console.log(`child process exited with code ${code}`);
});

That's it for this one guys hope you found it useful drop your thoughts in the comment section below and I will see you in the next one.