In a previous article " Node JS Internals " we discussed Node JS internal architecture and also discussed why we should node increase Thread Pool size to handle multiple requests concurrently. I have told you that scalability and performance are not related to Thread Pool size.
For scalability and high performance, we can use clustering and worker threads.
Clustering
Let's say you are in a grand wedding and thousands of guests are in the wedding. There is one kitchen and one cook is preparing the food for all these guests. Sounds unpredictable, right? You are not utilizing the kitchen's full resources if you have only one cook.
This is exactly what happens in a Node JS application running on a multicore CPU when only one core is being used to handle all the requests. so, even though our machine has the power of multicores, without clustering, our application runs on just a one-core. One core is responsible for handling all the work.
When in your kitchen multiple cooks are working that's the clustering.
Clustering is a technique that is used to enable the single Node JS application to utilize multiple CPU cores effectively.
To implement clustering you have to use a cluster module from Node JS.
const cluster = require('cluster');
By using this cluster module, you can create multiple instances of our Node JS application. These instances are called workers. All workers share the same server port and handle incoming requests concurrently.
There are two types of processes in the cluster architecture.
1.Master Process:
The Master process is like the main cook in the kitchen who manages workers. It initializes the application, sets up the clustering environment, and also delegates tasks to worker processes. It does not directly handle application requests.
What does the Master process do?
Creates multiple worker processes using the cluster.fork() method. It also restarts workers if they crash or exit unexpectedly.
It makes sure that incoming requests are distributed across all worker processes. On Linux, this is handled by an operating system, and on Windows, Node JS itself acts as the load balancer.
It enables communication between workers via IPC(Inter-Process Communication).
2.Worker Processes:
Worker processes are the instance of the Node JS application created by the master process. Each process runs independently on a separate CPU core and handles incoming requests.
Worker processes cannot directly communicate with each other they communicate via master.
The worker process handles the incoming request and performs some tasks such as database query, computation, or any application logic.
const cluster = require('cluster');
const os = require('os');
if (cluster.isMaster) {
console.log(`Master ${process.pid} is running`);
// Fork workers
cluster.fork();
cluster.fork();
cluster.fork();
cluster.fork();
} else {
console.log(`Worker ${process.pid} is running`);
// Worker logic (e.g., server setup) goes here
}
Here, we are checking first this is the master process. if yes then it will create worker processes.
In our code, I am creating a worker process using cluster.fork().
But, this is not an ideal way of creating a worker process.
Suppose, you are creating 4 worker processes and your system has two cores.
To, solve this problem instead of creating worker processes hardcoded first find the CPU cores then consider that data create worker processes.
const cluster = require('cluster');
const os = require('os');
if (cluster.isMaster) {
console.log(`Master ${process.pid} is running`);
const numCPUs = os.cpus().length;
// Fork workers
for (let i = 0; i < numCPUs; i++) {
cluster.fork();
}
} else {
console.log(`Worker ${process.pid} is running`);
// Worker logic (e.g., server setup) goes here
}
I am using dual core system so output will look like this.
Master 12345 is running
Worker 12346 is running
Worker 12347 is running
Worker 12348 is running
Worker 12349 is running
Now, you have a question if you have a dual-core CPU then why 4 worker processes created?
This is because the number of logical cores is 4, My CPU supports hyperthreading or Simultaneous Multi-Threading (SMT).
Worker Threads
In the restaurant, the waiter takes the order and gives that order to a team of cooks because cooking takes some time. If table cleaning or any other waiter-related work comes then the waiter does this. When the order is ready the cook gives food back to the waiter and the waiter serves this food to the customer.
This is the same scenario related to worker threads. If any computationally expensive tasks like large-scale data processing, complex calculations, or heavy algorithms comes then the main thread delegates this task to the worker thread. The worker performs this task, not the main thread.
*Why, this is helpful? *
We know that the Node JS event loop is single-threaded and if this heavy computational work is done by the main thread then the event loop will be blocked. If you use these worker threads then these heavy tasks are given to worker threads and worker threads perform these tasks, not the main thread so the event loop does not get blocked.
Worker threads can communicate with the main thread via a message-passing system, and data can be sent between threads using structured cloning (deep copy).
Now, we are trying to mimic the worker threads working.
main.js (Main Thread)
const { Worker } = require('worker_threads');
function startWorker() {
const worker = new Worker('./worker.js'); // Create a worker using worker.js
// Listen for messages from the worker
worker.on('message', (message) => {
console.log('Message from worker:', message);
});
// Handle errors in the worker
worker.on('error', (error) => {
console.error('Worker error:', error);
});
// Handle worker exit
worker.on('exit', (code) => {
console.log(`Worker exited with code ${code}`);
});
// Send a message to the worker
worker.postMessage({ num: 100 });
}
startWorker();
worker.js (Worker Thread)
const { parentPort } = require('worker_threads'); // Access parentPort to communicate with the main thread
// Listen for messages from the main thread
parentPort.on('message', (data) => {
console.log('Received data from main thread:', data);
// Perform some processing
const result = { squared: data.num * data.num };
// Send the result back to the main thread
parentPort.postMessage(result);
});
If data contains large structures, it will be deeply cloned and passed over, which might have some performance overhead.
Working of code
The Worker class is used to spawn new threads.
You can send data to the worker using worker.postMessage and listen for messages with worker.on('message', callback).
In the worker thread, parentPort is the primary interface to communicate with the main thread.
You can listen for messages from the main thread (parentPort.on('message')) and send messages back using parentPort.postMessage.
The output will be:
Received data from main thread: { num: 100 }
Message from worker: { squared: 10000 }
Worker exited with code 0
Now, you also have one question: why don't we create hundreds of worker threads?
But, The reason is if you create more threads than the number of cores, threads will compete for CPU time, leading to context switching, which is expensive and reduces overall performance.
When should you use clustering, worker threads, or both in Node.js?
1. When to use Worker threads?
- CPU-Bound tasks:
Tasks involve heavy computations, such as Image/video processing, Data compression, or encryption, Machine learning inference, Scientific calculations
- Shared Memory is required:
You need to share data efficiently between threads without duplicating it.
- Single-core usage:
If your application needs to scale only within a single process but still requires parallelism for CPU-intensive tasks.
2.When to use clustering?
- I/O bound:
Tasks involve handling a high number of client requests, such as Web, servers, Chat applications, and APIs. Clustering helps scale horizontally by distributing requests across all CPU cores.
- Isolated memory:
Your application doesn’t need to share a lot of data between processes.
- Multi-Core Utilization:
You want to utilize all available cores by spawning multiple Node.js processes.
3.When to use both clustering and worker threads?
- I/O-Bound + CPU-Bound Tasks:
The application handles HTTP requests but offloads computationally intensive tasks. Example: A web server processes file uploads and performs image resizing or video transcoding.
- High Scalability:
You need both process-level and thread-level parallelism for high throughput. In an E-commerce site Clustering ensures multiple processes handle incoming requests. Worker threads process background tasks like generating personalized recommendations.
Thank You.
Feel free to ask the question or give any suggestions.
If you found this informative then like it.
Top comments (1)
Thanks for this.
Some questions:
1) do you have a way of knowing the actual number of workers being use at a time in nodejs (not when you spawn workers yourself but when you use core librairies like dns or fs that make use of the default worker pool)?
2) is pm2 a good alternative to Cluster usage?
3) how do you know if a library you use (for instance mongoose) make usage of default worker pool or cretates its own worker ?pool?
Thanks for your input