Threads in NodeJS

Node.js is a popular runtime environment for executing JavaScript code outside the browser, primarily known for its non-blocking, event-driven architecture. While Node.js is single-threaded by nature, it offers various mechanisms to leverage multithreading capabilities for performance optimization.

NodeJS itself is multithreaded and provides hidden threads, through libuv library. It handles I/O operations like reading files from a disk or network requests. Through these hidden threads NodeJS provides asynchronous methods that allow your code to make I/O requests without blocking the main thread.

Although Nodejs has some hidden threads, you can not use them to offload CPU-intensive tasks, such as complex calculations or image resizing. The only way to speed up a CPU-bound task is to increase the ‘processor speed’.

In recent years computers are shipping with extra cores (dual core, quadcore, octa core etc.) To leverage these cores NodeJS has introduced worker-threads module, which allows you to create threads and execute multiple JS tasks in parallel. Once a thread finishes a task it sends a message to main thread that contains the result of the operation so that it can be used with other parts of the code.

Advantage of using worker threads is that CPU-bound tasks do not block the main thread and you can divide and distribute a task to multiple workers to optimize it.

For deeper understanding Let us first know the difference between process and threads.

Process ⚙️

It is a running program in the operating system. It has its own memory and can not see nor access the memory of other running programs. It also has an instruction pointer, which indicates the instruction currently being executed in a program. Only one task can be executed at a time.

Example: Using child_process to create a new process

const { fork } = require('child_process');

// Fork a new Node.js process
const child = fork('child.js');

// Send a message to the child process
child.send({ hello: 'world' });

// Receive messages from the child process
child.on('message', (message) => {
  console.log('Received message from child:', message);
});
    

child.js

process.on('message', (message) => {
  console.log('Received message from parent:', message);
  process.send({ reply: 'hello parent' });
});
    

In this example:

○ The parent script forks a new process using fork('child.js').

○ The parent and child processes communicate through messages.

○ Each process has its own memory space.

Thread 🧵

Threads are like process: they have their own instruction pointer and execute one JS task at a time. Threads do not have their own memory . They reside within the process’s memory. Process contains multiple threads. Threads can communicate with one another through message passing or sharing the data in the process’s memory.

When it comes to the execution of threads, they have similar behavior to that of processes. If you have multiple threads running on a single core system, the operating system will switch between them in regular intervals, giving each thread a chance to execute directly on the single CPU. On a multi-core system, the OS schedules the threads across all cores and executes the JavaScript code at the same time. If you end up creating more threads than there are cores available, each core will execute multiple threads concurrently.

Example: Using worker_threads to create a new thread

const { Worker, isMainThread, parentPort, workerData } = require('worker_threads');
if (isMainThread) {
  // This is the main thread
  const worker = new Worker(__filename, {
    workerData: { start: 1, end: 1e6 }
  });
  // Listen for messages from the worker
  worker.on('message', (result) => {
    console.log(`Result from worker: ${result}`);
  });
  worker.on('error', (error) => {
    console.error('Worker error:', error);
  });
  worker.on('exit', (code) => {
    if (code !== 0) console.error(`Worker stopped with exit code ${code}`);
  });
} else {
  // This is the worker thread
  const { start, end } = workerData;
  let sum = 0;
  for (let i = start; i <= end; i++) {
    sum += i;
  }
  // Send the result back to the main thread
  parentPort.postMessage(sum);
}

In this example:

○ The main thread creates a worker thread using new Worker(__filename, { workerData: { start: 1, end: 1e6 } }).

○ The worker thread performs a computation and sends the result back to the main thread.

○ Both threads share the same memory space, allowing for efficient data sharing.

Now you have the learned about the threads, so we can proceed to the mater of concern.

Understanding Hidden Threads in NodeJS

NodeJS does provide extra threads, which is why it’s considered to be multithreaded. Nodejs implements the libuv library, which provides four extra threads to a node process. With these threads, the I/O opreations are handled separately and when they are finished, the event loop adds the callback associated with the I/O task in microtask queue. When call stack in the main thread is clear, the callback associated with the given I/o task does not execute in parallel; however the task itself of reading a file or a network request happens with the help of threads. Once the I/O task finishes the callback runs in the main thread. Now look the previous image for more understanding.

In addition to these four threads, the V8 engine, also provides two threads for handling things like automatic garbage collection. This brings the total number of threads in a process to seven : one main thread, four Node.js threads, and two V8 threads.

Offloading a CPU-Bound Task with the worker-threads Module

In this section, you will offload a CPU-intensive task to another thread using the worker-threads module to avoid blocking the main thread. To do this, you will create a worker.js file that will contain the CPU-intensive task. In the index.js file, you will use the worker-threads module to initialize the thread and start the task in the worker.js file to run in parallel to the main thread. Once the task completes, the worker thread will send a message containing the result back to the main thread.

If it shows two or more cores, you can proceed with this step.

Next, create and open the worker.js file in your text editor:

cd worker.js
    

In your worker.js file, add the following code to import the worker-threads module and do the CPU-intensive task:

multi-threading_demo/worker.js

const { parentPort } = require("worker_threads");

let counter = 0;
for (let i = 0; i < 20_000_000_000; i++) {
  counter++;
}
    

The first line loads the worker_threads module and extracts the parentPort class. The class provides methods you can use to send messages to the main thread. Next, you have the CPU-intensive task that is currenty in the calculateCount() function in the index.js file. Later in this step, you will delete this function from index.js.

Following this, add the highlighted code below:

multi-threading_demo/worker.js

const { parentPort } = require("worker_threads");

let counter = 0;
for (let i = 0; i < 20_000_000_000; i++) {
  counter++;
}
parentPort.postMessage(counter);
    

Here you invoke the postMessage() method of the parentPort class, which sends a message to the main thread containing the result of the CPU-bound task stored in the counter variable.

Save and exit your file. Open index.js in your text editor:

cd index.js
    

Since you already have the CPU-bound task in worker.js, remove the highlighted code from index.js:

multi-threading_demo/index.js

const express = require("express");

const app = express();
const port = process.env.PORT || 3000;
app.get("/non-blocking/", (req, res) => {
  res.status(200).send("This page is non-blocking");
});
function calculateCount() {
  return new Promise((resolve, reject) => {
    let counter = 0;
    for (let i = 0; i < 20_000_000_000; i++) {
      counter++;
    }
    resolve(counter);
  });
}
app.get("/blocking", async (req, res) => {
  const counter = await calculateCount();
  res.status(200).send(`result is ${counter}`);
});
app.listen(port, () => {
  console.log(`App listening on port ${port}`);
});

Next, in the app.get('/blocking') callback, add the following code to initialize the thread:

multi-threading_demo/index.js

const express = require("express");
const { Worker } = require("worker_threads");
...
app.get("/blocking", async (req, res) => {
  const worker = new Worker("./worker.js");
  worker.on("message", (data) => {
    res.status(200).send(`result is ${data}`);
  });
  worker.on("error", (msg) => {
    res.status(404).send(`An error occurred: ${msg}`);
  });
});
...

First, you import the worker_threads module and unpack the Worker class. Within the app.get('/blocking') callback, you create an instance of the Worker using the new keyword that is followed by a call to Worker with the worker.js file path as its argument. This creates a new thread and the code in the worker.js file starts running in the thread on another core.

Following this, you attach an event to the worker instance using the on('message') method to listen to the message event. When the message is received containing the result from the worker.js file, it is passed as a parameter to the method’s callback, which returns a response to the user containing the result of the CPU-bound task.

Next, you attach another event to the worker instance using the on('error') method to listen to the error event. If an error occurs, the callback returns a 404 response containing the error message back to the user.

Your complete file will now look like the following:

multi-threading_demo/index.js

const express = require("express");
const { Worker } = require("worker_threads");
const app = express();
const port = process.env.PORT || 3000;
app.get("/non-blocking/", (req, res) => {
  res.status(200).send("This page is non-blocking");
});
app.get("/blocking", async (req, res) => {
  const worker = new Worker("./worker.js");
  worker.on("message", (data) => {
    res.status(200).send(`result is ${data}`);
  });
  worker.on("error", (msg) => {
    res.status(404).send(`An error occurred: ${msg}`);
  });
});
app.listen(port, () => {
  console.log(`App listening on port ${port}`);
});

Save and exit your file, then run the server:

node index.js

Look the blocking requests took only 6.93 s to load without blocking the non-blocking endpoint which took only 7ms when both requests were made on same time.

Visit the http://localhost:3000/blocking tab again in your web browser. Before it finishes loading, refresh all http://localhost:3000/non-blocking tabs. You should now notice that they are loading instantly without waiting for the /blocking route to finish loading. This is because the CPU-bound task is offloaded to another thread, and the main thread handles all the incoming requests.

Now, stop your server using CTRL+C.

Now that you can make a CPU-intensive task non-blocking using a worker thread, you can use four worker threads to improve the performance of the CPU-intensive task.

Conclusion

Understanding and utilizing threads in Node.js can significantly enhance the performance and scalability of your applications. While Node.js is inherently single-threaded, it provides mechanisms like hidden threads through the libuv library for efficient I/O operations and the worker-threads module for CPU-bound tasks. By offloading intensive tasks to worker threads, you can keep your.

For more information on worker threads, refer: