Parallel Processing with worker_threads in Node.js

Photo by Bernd 📷 Dittrich on Unsplash

Node.js is famous for its single-threaded, event-driven architecture. This design works beautifully for I/O-bound operations such as reading files, making network requests, or querying databases. But what happens when you need to perform CPU-intensive tasks — like image processing, data encryption, or complex mathematical computations? The main thread blocks, your server stops responding to requests, and your users are left waiting.

For a long time, the answer was to offload heavy work to child processes using child_process.fork(). While that approach works, it comes with significant overhead: each child process gets its own V8 instance, its own memory heap, and communication happens via serialized IPC messages.

Enter the worker_threads module — available as a stable API in Node.js since v12 LTS. Worker threads run JavaScript in parallel, share memory when needed, and operate inside the same process with far less overhead than child processes. In this article, you’ll learn how to create workers, communicate between threads, share memory efficiently, and apply this knowledge to real-world scenarios.

What Are Worker Threads?

Worker threads allow you to run JavaScript code in parallel OS-level threads within the same Node.js process. Unlike child processes, worker threads share the same process memory space, which means they can transfer and even share data far more efficiently.

Each worker thread gets its own V8 isolate (its own JavaScript execution context) and its own event loop, but they all live inside a single operating system process.

Here’s a simple mental model: think of the main thread as a restaurant manager and worker threads as kitchen staff. The manager takes orders (handles incoming I/O) and delegates heavy cooking (CPU work) to the kitchen staff, who work in parallel without blocking the manager from greeting new guests.

Your First Worker Thread

Let’s start with the simplest possible example. You need two files: one for the main thread and one for the worker.

worker.js

const { parentPort } = require('worker_threads');

// Perform a heavy computation
let sum = 0;
for (let i = 0; i < 1_000_000_000; i++) {
sum += i;
}
// Send the result back to the main thread
parentPort.postMessage(sum);

main.js

const { Worker } = require('worker_threads');

console.log('Main thread: starting heavy computation in a worker...');

const worker = new Worker('./worker.js');

worker.on('message', (result) => {
console.log(`Main thread: worker returned ${result}`);
});

worker.on('error', (err) => {
console.error('Worker error:', err);
});

worker.on('exit', (code) => {
console.log(`Main thread: worker exited with code ${code}`);
});
console.log('Main thread: I am NOT blocked!');
Main thread: starting heavy computation in a worker...
Main thread: I am NOT blocked!
Main thread: worker returned 499999999067109000
Main thread: worker exited with code 0

The key takeaway here is that the message “I am NOT blocked!” prints immediately — before the worker finishes its billion-iteration loop. The main thread continues to process events, handle requests, and remain responsive while the worker crunches numbers in a parallel OS thread. When the worker finishes, it sends the result via postMessage, which triggers the ‘message’ event back on the main thread.

Passing Data to Workers with workerData

In most real scenarios, you need to send initial data to a worker when it starts — for example, which file to process or what range of numbers to compute. The workerData option lets you do exactly this.

range-worker.js

const { parentPort, workerData } = require('worker_threads');

const { start, end } = workerData;
let sum = 0;
for (let i = start; i < end; i++) {
sum += i;
}
parentPort.postMessage(sum);

main-range.js

const { Worker } = require('worker_threads');

function runWorker(start, end) {
return new Promise((resolve, reject) => {
const worker = new Worker('./range-worker.js', {
workerData: { start, end }
});
worker.on('message', resolve);
worker.on('error', reject);
});
}
async function main() {
const TOTAL = 1_000_000_000;
const THREADS = 4;
const chunkSize = TOTAL / THREADS;
console.time('parallel');
const promises = [];
for (let i = 0; i < THREADS; i++) {
const start = i * chunkSize;
const end = start + chunkSize;
promises.push(runWorker(start, end));
}
const results = await Promise.all(promises);
const totalSum = results.reduce((acc, val) => acc + val, 0);
console.timeEnd('parallel');
console.log(`Total sum: ${totalSum}`);
}
main();
parallel: 652ms
Total sum: 499999999067109000

If you ran the same computation on a single thread, it would take roughly 2,000–2,500ms. By splitting the work across 4 workers, each worker handles only 250 million iterations, and they all execute simultaneously. The workerData object is cloned (using the structured clone algorithm) and passed into each worker at creation time, where it becomes available via require(‘worker_threads’).workerData. This approach scales linearly with available CPU cores for embarrassingly parallel tasks.

Two-Way Communication with MessagePort

So far, communication has been one-directional: the worker sends a result back to the main thread. But what if you need a continuous back-and-forth conversation — like dispatching multiple tasks to the same worker?

The parentPort in the worker and the Worker instance in the main thread form a built-in message channel. You can use postMessage on either side.

echo-worker.js

const { parentPort } = require('worker_threads');

parentPort.on('message', (msg) => {
if (msg === 'exit') {
process.exit(0);
}
// Echo back in uppercase
parentPort.postMessage(msg.toUpperCase());
});

main-echo.js

const { Worker } = require('worker_threads');

const worker = new Worker('./echo-worker.js');
worker.on('message', (msg) => {
console.log(`Worker says: ${msg}`);
});
worker.postMessage('hello');
worker.postMessage('node.js worker threads');
worker.postMessage('exit');
Worker says: HELLO
Worker says: NODE.JS WORKER THREADS

This pattern is useful when you want a long-running worker that processes multiple messages over its lifetime, rather than creating and destroying a new worker for each task. The main thread sends strings, the worker transforms them and sends them back, and when the main thread sends ‘exit’, the worker shuts itself down. Notice that the ‘exit’ message doesn’t produce an echo because process.exit(0) terminates the worker before it gets a chance to post back.

Sharing Memory with SharedArrayBuffer

One of the most powerful features that sets worker_threads apart from child_process is the ability to share memory between threads. With SharedArrayBuffer, multiple threads can read and write to the same block of memory without copying data back and forth.

shared-memory.js

const { Worker, isMainThread, parentPort, workerData } = require('worker_threads');

if (isMainThread) {
// Main thread: create shared memory
const sharedBuffer = new SharedArrayBuffer(4); // 4 bytes = 1 Int32
const sharedArray = new Int32Array(sharedBuffer);
sharedArray[0] = 0; // initial value
console.log(`Before workers: sharedArray[0] = ${sharedArray[0]}`);
const workers = [];
for (let i = 0; i < 4; i++) {
workers.push(
new Promise((resolve, reject) => {
const w = new Worker(__filename, { workerData: { sharedBuffer } });
w.on('exit', resolve);
w.on('error', reject);
})
);
}
Promise.all(workers).then(() => {
console.log(`After workers: sharedArray[0] = ${sharedArray[0]}`);
});
} else {
// Worker thread: increment the shared value 1,000,000 times
const sharedArray = new Int32Array(workerData.sharedBuffer);
for (let i = 0; i < 1_000_000; i++) {
Atomics.add(sharedArray, 0, 1);
}
}
Before workers: sharedArray[0] = 0
After workers: sharedArray[0] = 4000000

This example uses a single file with the isMainThread check to run both the main and worker code. The main thread creates a SharedArrayBuffer of 4 bytes (enough for one 32-bit integer), wraps it with Int32Array, and passes it to four workers via workerData. Each worker increments that shared integer 1,000,000 times.

The critical detail here is Atomics.add(). Without it, you’d get a race condition — multiple threads reading the same value, adding 1, and writing back, overwriting each other’s work. The result without Atomics would be some unpredictable number less than 4,000,000. Atomics.add performs an atomic read-modify-write operation, guaranteeing that each increment is thread-safe. The final result is exactly 4,000,000 (4 workers Ă— 1,000,000 increments).

Transferring Data with transferList

When you postMessage an object, Node.js clones it by default. For large ArrayBuffer objects, cloning is expensive. Instead, you can transfer ownership of the buffer to the other thread — the original thread loses access, but the transfer is nearly instantaneous.

transfer-example.js

const { Worker, isMainThread, parentPort } = require('worker_threads');

if (isMainThread) {
const worker = new Worker(__filename);
const largeBuffer = new ArrayBuffer(1024 * 1024 * 64); // 64 MB
console.log(`Before transfer: buffer byteLength = ${largeBuffer.byteLength}`);
// Transfer (not clone) the buffer to the worker
worker.postMessage(largeBuffer, [largeBuffer]);
console.log(`After transfer: buffer byteLength = ${largeBuffer.byteLength}`);
worker.on('message', (msg) => {
console.log(`Worker received buffer of ${msg} bytes`);
});
} else {
parentPort.on('message', (buf) => {
parentPort.postMessage(buf.byteLength);
});
}
Before transfer: buffer byteLength = 67108864
After transfer: buffer byteLength = 0
Worker received buffer of 67108864 bytes

After the transfer, the main thread’s largeBuffer.byteLength becomes 0 — the buffer has been moved, not copied. The worker receives the full 64 MB buffer instantly. This is a zero-copy operation, making it ideal for scenarios where you need to pass large datasets (images, audio frames, video chunks) to a worker for processing. The second argument to postMessage is the transferList array, which specifies which ArrayBuffer objects should be transferred rather than cloned.

A Practical Example: Parallel File Hashing

Let’s bring everything together with a real-world use case — hashing multiple files in parallel. This combines worker_threads with the crypto and fs modules.

hash-worker.js

const { parentPort, workerData } = require('worker_threads');
const fs = require('fs');
const crypto = require('crypto');

const { filePath } = workerData;
const content = fs.readFileSync(filePath);
const hash = crypto.createHash('sha256').update(content).digest('hex');
parentPort.postMessage({ filePath, hash });

hash-main.js

const { Worker } = require('worker_threads');
const path = require('path');

function hashFile(filePath) {
return new Promise((resolve, reject) => {
const worker = new Worker('./hash-worker.js', {
workerData: { filePath }
});
worker.on('message', resolve);
worker.on('error', reject);
});
}
async function main() {
const files = [
path.join(__dirname, 'file1.txt'),
path.join(__dirname, 'file2.txt'),
path.join(__dirname, 'file3.txt'),
path.join(__dirname, 'file4.txt'),
];
console.time('parallel-hashing');
const results = await Promise.all(files.map(hashFile));
console.timeEnd('parallel-hashing');
results.forEach(({ filePath, hash }) => {
console.log(`${path.basename(filePath)}: ${hash}`);
});
}
main();
parallel-hashing: 45ms
file1.txt: a3f2b8c91d...
file2.txt: 7e4c0a12bb...
file3.txt: 1f9d3b57ee...
file4.txt: c8a1e43f02...

Each worker independently reads a file and computes its SHA-256 hash. Because the hashing runs in separate threads, all four files are processed simultaneously. For large files or many files, this can dramatically reduce total processing time compared to sequential hashing. This pattern is directly applicable to build tools, file integrity checkers, backup systems, and deployment pipelines.

When to Use worker_threads vs. Other Approaches

Understanding when worker threads are the right tool is just as important as knowing how to use them.

Use worker_threads when you have CPU-intensive work (heavy computations, data transformation, image/video processing, encryption) that would block the event loop. They live in the same process, share memory, and have low creation overhead compared to child processes.

Use child_process when you need to run a separate program (like a Python script or shell command), need full process isolation for security, or are dealing with code that might crash and shouldn’t take down your main process.

Use the cluster module when you want multiple instances of your HTTP server listening on the same port to handle more concurrent requests — this is about scaling I/O capacity, not about parallelizing a single computation.

Don’t use workers for I/O-bound tasks (database queries, API calls, file reading). Node.js’s built-in async I/O already handles these efficiently on a single thread. Adding workers for I/O adds complexity without performance gains.

The worker_threads module gives Node.js developers a powerful, low-overhead mechanism for true parallel execution. Throughout this article, you’ve seen how to create basic workers, pass initial data with workerData, build two-way communication channels, share memory safely with SharedArrayBuffer and Atomics, transfer large buffers with zero-copy efficiency, and apply all of these patterns to a practical file-hashing scenario.

The key takeaways are: worker threads solve the CPU-bound bottleneck that has historically been Node.js’s Achilles’ heel; SharedArrayBuffer combined with Atomics enables safe, high-performance shared memory; and the transferList mechanism lets you move large data between threads without the cost of serialization.

As you integrate worker threads into your projects, start with the simplest pattern (a worker that takes input, does work, and returns a result), and only reach for shared memory when profiling shows that data copying is a bottleneck. Like any concurrency tool, worker threads introduce complexity — but when applied to the right problems, they can transform a sluggish Node.js application into a high-performance powerhouse.


Parallel Processing with worker_threads in Node.js was originally published in Server-Side JavaScript on Medium, where people are continuing the conversation by highlighting and responding to this story.

Scroll to Top