Introduction to worker threads

In Node.js, worker threads are a feature enabling JavaScript code execution in separate threads. This feature was introduced to leverage multi-core processors and enhance the performance of Node.js applications, particularly for CPU-intensive tasks that can be parallelized. In this topic, you'll learn what worker threads are and how to use them correctly.

Why do you need worker threads?

The primary reason for needing worker threads is that Node.js is single-threaded. Worker threads are used to create threads that execute in parallel. You might wonder: What's the difference with asynchronous operations? Worker threads create separate operating system threads, which can consume more system resources and allow multiple tasks to be performed in parallel. Asynchronous tasks execute within a single Node.js thread, providing lightweight resource management but may be less suitable for intensive calculations. It's important to note that Node.js employs a non-blocking I/O model, making I/O operations asynchronous by default.

Creating a worker thread

To work with the worker thread module, you need to import it:

import { Worker } from 'node:worker_threads'

The Worker class allows you to create new worker threads and execute JavaScript code within them. Additionally, you need a couple more things to work with worker threads.

import { Worker, workerData, parentPort, isMainThread } from 'node:worker_threads'

if (isMainThread) {
  const worker = new Worker(__filename, {
    workerData: { message: "Hello from the main thread!" }
  });

  worker.on('message', (message) => {
    console.log(`Main thread received a message from the worker: ${message}`);
  });

  worker.postMessage("Hello from the main thread!");
} else {
  parentPort.on('message', (message) => {
    console.log(`Worker received a message from the main thread: ${message}`);

    parentPort.postMessage("Hello from the worker thread!");
  });

  console.log(`Worker data: ${workerData.message}`);
}

workerData is an object passed to a worker thread when it's created using the Worker method. parentPort is an object that enables the worker thread to send messages back to the main thread. isMainThread is a property that helps you determine whether the code in the current context is executed in the main thread or inside a worker thread.

In the code example above, you create a new worker in the main thread, and set it to run the same script file (__filename) with a different context. You pass some initial data to the worker using workerData. You then listen for messages from the worker and send a message to the worker. In the worker thread, you listen for messages from the main thread using parentPort.on('message', ...), access the workerData passed from the main thread, and send a message back to the main thread using parentPort.postMessage(...).

When can you use worker threads?

• Background calculations: Calculations that require significant time can be moved to worker threads so as not to block the main thread of execution.
• Intensive networking: If your application performs a lot of network requests, you can move this work to worker threads so that the main thread remains available to handle other tasks.
• Real-time data processing: Worker threads can be used to process real-time data such as streaming audio, video, or sensor data.

Below, you'll find an example of code to demonstrate how worker threads can be used for real-time data processing:

const { Worker, isMainThread, parentPort, workerData } = require('worker_threads');
const fs = require('fs');
const path = require('path');
const { createServer } = require('http');
const { Readable } = require('stream');

const videoFilePath = path.join(__dirname, 'sample.mp4');

if (isMainThread) {
  // Create a readable stream for the video file
  const videoStream = fs.createReadStream(videoFilePath);

  const server = createServer((req, res) => {
    res.setHeader('Content-Type', 'video/mp4');
    videoStream.pipe(res);
  });

  server.listen(3000, () => {
    console.log('Server is running on http://localhost:3000');
  });
} else {
  const videoStream = fs.createReadStream(workerData.videoPath);

  videoStream.on('data', (frameData) => {
    parentPort.postMessage({ frameData });
  });
}

1. The main thread sets up an HTTP server to serve the video file.
2. A worker thread is created to process video frames.
3. The video frames are read from the video file using a readable stream, and each frame is sent to the worker thread for processing.
4. The worker thread sends the processed frame data back to the main thread, which is then streamed to the client.

Remember that this is a simplified example, you don't have to understand the full code, just try to understand the concept of worker threads.

Disadvantages of worker threads

Worker threads, while powerful tools in Node.js, do have some drawbacks. Each thread requires its own system resources, and creating a large number of threads can consume a significant amount of memory.

Worker threads are not suitable for all tasks; they are most effective for computationally intensive work. For tasks that involve frequent interactions with the DOM, such as web pages, or require event synchronization, using worker threads may be less effective.

Conclusion

The worker threads module allows you to execute code in parallel using threads. It's useful for background calculations, intensive networking, and real-time data processing. However, it's important to use this tool carefully, as it can consume a significant amount of memory. It's also essential to distinguish when to use worker threads versus asynchronous operations.