Did you just discover …
Did you just discover that NodeJS is single threaded? Maybe you are new to NodeJS development. Maybe you have been using Node within an environment where the threading model was never a concern, for instance within AWS Lambda. For whatever reason, discovering and understanding Node’s threading model can usually elicit a couple of reactions:
“Oh that’s cool!”
or
“Really? So what do I do now?”
An Arrow in the Quiver
This article will focus on one of the options available within the NodeJS environment. This is not the only option, but could be considered as one of the options that might fit into a larger architecture. This article outlines usage of Node Worker threads. For demonstration purpose, the code presented will utilize the Piscina worker pool library. (https://github.com/piscinajs/piscina)
Demonstrating the Problem
Let’s say you have an awesome NodeJS setup that suits your needs perfectly. One last bit of functionality is required that demands you process an image file in a minimal way. This processing however is CPU intensive, which winds up tying up the primary event loop of Node. Whenever this CPU intensive operation is called, it blocks all other calls. Within Chrome dev tools, this may look something like below.
This demonstration API takes a parameter of how long to “sleep” or tie up the CPU. In this case, a 10 second timeout request is initially sent, with a one second sleep request sent immediately after. Since NodeJS can only handle a single thread of work, the second request has to wait until the first is complete.
Using an alternate version utilizing a worker, the Chrome network request tab looks much different as shown below.
The second request no longer needed to wait for the first request to complete. This may be the difference between a responsive server, and a server that leaves a client waiting for a completely different request to complete.
Worker Pool Example
Below are the complete NodeJS files used to produce the above results.
server.js
const express = require('express');
const cors = require('cors');
const sleeper = require('./sleep');
const path = require('path');
const Piscina = require('piscina');// https://nodejs.org/api/worker_threads.html#worker-threadsconst app = express();
const port = 3030;
app.use(cors());
const piscina = new Piscina({
filename: path.resolve(__dirname, 'worker.js')
});app.get('/message', (req, res) => {
var theTimeout = req.query.timeout;
theTimeout = ! theTimeout ? 2000 : theTimeout;
sleeper.sleep(theTimeout);
res.send({result:`client requested timeout was ${theTimeout}`});
})app.get('/message-worker', async (req, res) => {
var theTimeout = req.query.timeout;
theTimeout = ! theTimeout ? 2000 : theTimeout;
const result = await piscina.run({ millis: theTimeout });
res.send ( {result:result });
});app.listen(port, () => {
console.log(`Demo app listening on port ${port}`)
});
sleep.js
function sleep(milliseconds) {
const date = Date.now();
let currentDate = null;
do {
currentDate = Date.now();
} while (currentDate - date < milliseconds);
}
exports.sleep = sleep;worker.js
const sleeper = require('./sleep');
module.exports = ({ millis }) => {
sleeper.sleep(millis);
return `waited in worker.... ${millis} millis.`;
};There are two main entry points, message, and message-worker. The message path simply calls the sleep function to simulate running a CPU intensive process. The message-worker method delegates to the worker.js which runs in a new thread within Node.
Gotchyas
Using this technique to free up Node’s main thread can be applied in some situations, but it’s not a cure-all for anytime you run into troubles. Resources are not free, and eventually you will wind up in trouble if this is used for every request in your backend. Some questions you should ask yourself are:
- What should my resource cap be if this functionality is called multiple times?
- Are there options for deployment that mitigate the need to adding complexity to my Node application?
- Is this a bandaid for a larger usage issue?
Conclusion
Hopefully this article shed some light on a simple mechanism to alleviate a CPU intensive path within a NodeJS application. Happy coding!
