Node.js has become an extremely popular runtime environment for JavaScript developers, allowing them to build fast and scalable server-side applications. However, with great power comes great responsibility, and it's important to follow best practices when developing Node applications to ensure they are reliable, performant, and secure. In this guide, we'll cover some of the most important best practices to keep in mind when developing Node applications.
As Node.js applications grow, they become harder to maintain and extend. Developers struggle to incorporate new features into a monolithic codebase with hundreds of dependencies. To overcome this pitfall, i recommend structuring your solution by components as component-based architecture helps you organize your code into manageable and testable units, making it easier to maintain and extend your application. Here are a few tips to structure your Node.js application by components:
Each component should have dedicated layers for web, logic, and data access code. This separation of concerns significantly eases mocking and testing your system. You should avoid mixing web objects like Express req and res with other layers, as this makes your application dependent on specific web frameworks.
Cross-cutting-concern in house utilities, like a logger or encryption, should be wrapped by your code and exposed as private NPM packages. This allows sharing them among multiple codebases and projects as well as keeping them maintainable.
Define your Express app in at least two files: the API declaration (app.js) and the networking concerns (WWW). It is even better to locate your API declaration within components. This separation makes it easier to test your API and maintain hundreds of lines of code in a single file.
Ensure that your configuration setup can read keys from files and environment variables, keep secrets outside committed code, and provide a hierarchical structure for easier findability. Alternatively use packages like rc, nconf, config, and convict to help you achieve these goals.
These is an example of how you could structure your code:
├── api │ ├── routes.js │ ├── controllers.js │ └── middlewares.js ├── services │ ├── users.js │ ├── orders.js │ └── products.js ├── models │ ├── user.js │ ├── order.js │ └── product.js ├── config │ ├── development.json │ ├── production.json │ └── test.json ├── utils │ ├── logger.js │ └── encrypt.js ├── app.js ├── server.js
In this example, we have two components: users and orders. Each component has its own folder with files for the data model, business logic, and API endpoints. The each file exports the public API of the component, which can be used by other components or by the main entry point of the application. The main entry point of the application is app.js, which loads the configuration woth every packages and database connections. The server.js file is responsible for starting the HTTP server and mounting the components. This approach makes it easy to reason about each component and reduces the chances of introducing bugs due to dependencies between components.
Error handling is a crucial part of any Node.js application. Proper error handling helps in detecting and resolving issues, and thus ensures that the application remains stable and reliable. Node.js provides multiple ways of handling errors, let's see some of the best practices for handling errors in Node.js:
Async error handling with callback functions in Node.js is prone to excessive nesting and can lead to unmanageable code. Instead, it is recommended to use Async-Await or Promises, which provide a cleaner and more familiar syntax for handling errors.
For example, when using promises, the catch method can be used to handle any errors thrown by the promise:
somePromiseFunction()
.then(result => {
// handle result
})
.catch(error => {
// handle error
});
Similarly, when using Async-Await, the try-catch block can be used to handle any errors:
async function someAsyncFunction() {
try {
const result = await somePromiseFunction();
// handle result
} catch (error) {
// handle error
}
}
Using custom error types or strings as errors complicates the error handling logic and can lead to loss of information. It is recommended to use only the built-in Error object (or an object that extends the Error object) to ensure uniformity and prevent loss of information.
For example, throwing an error with a string message:
throw 'Some error occurred';
should be replaced with throwing an error using the built-in Error object:
throw new Error('Some error occurred');
It is important to differentiate between operational errors and programmer errors. Operational errors refer to known cases where the error impact is fully understood and can be handled thoughtfully. For example if a record if nowhere to be found.
const record = await database.get() // DB call
if (!record) throw new Error('Record not found');
On the other hand, programmer errors refer to unknown code failures that dictate gracefully restarting the application. For example, trying to read an undefined variable is a programmer error that should lead to gracefully restarting the application.
const record = await database.get() // DB call
record.id // Uncaught TypeError: Cannot read properties of null (reading 'id')
Error handling logic, such as logging or sending emails, should be encapsulated in a centralized service that all endpoints call when an error occurs. This approach prevents code duplication and ensures that all errors are handled correctly.
const ErrorHandlerService = {
logError(error) {
loggingLibrary.log(error);
},
sendEmail(error) {
emailLibrary.sendEmail('admin@example.com', 'Error occurred', error.message);
},
};
app.get('/example', (req, res) => {
try {
throw new Error('Example error');
} catch (error) {
ErrorHandlerService.logError(error);
ErrorHandlerService.sendEmail(error);
res.status(500).send('An error occurred');
}
});
In this example, ErrorHandlerService is a centralized error handling service that contains two methods: logError and sendEmail. These methods handle logging and sending emails when an error occurs.
The app.get function is an example endpoint that might throw an error. In the catch block, the endpoint calls the ErrorHandlerService to handle the error. The service logs the error using a third-party logging library and sends an email using a third-party email library.
By encapsulating error handling logic in a centralized service, we prevent code duplication and ensure that all errors are handled correctly.
When an unknown error occurs, such as a programmer error, it is recommended to simply gracefully shutdown the application before restarting carefully using a process management tool like PM2 or Forever.
process.on('uncaughtException', (error) => {
console.error(`Uncaught Exception: ${error.stack}`);
// log the error or notify the dev team before exiting
process.exit(1);
});
In the example code above, the uncaughtException event is used to handle any unknown errors that may occur in the application. When this event is triggered, the error is logged to the console and the process is exited with an error code of 1. This will ensure that the application restarts carefully using a process management tool like PM2 or Forever.
Note: If you do not handle promise rejections properly, your application can crash or behave unpredictably. In Node.js, any exception thrown within a promise will be swallowed and discarded unless you explicitly handle it. But dont worry process.unhandledRejection event ensures that your errors are not swallowed, and you can take appropriate action to handle them.
APM (Application Performance Monitoring) products are tools that can help you proactively monitor your Node.js application and detect errors, crashes, and slow parts that you may have missed. APM products can be integrated with your codebase or API to collect data and generate insights about the performance and health of your application. By using APM products, you can discover errors and downtime quickly and take corrective action before they impact your users. My go to APM is logtail, but there others out there like datadog and sematext.
Failing fast in case of an error is an important practice for any software developer. It means that you want your code to detect errors as soon as possible and stop the execution of the program to prevent further damage.
One way to achieve this in Node.js is by validating the input of your functions using a dedicated library like ajv or Joi. Input validation is crucial in preventing bugs and errors in your code, and it should be considered a standard practice.
When you validate your input, you ensure that the data you receive meets the expected requirements. For example, if you have a function that expects a number, but the input you receive is a string, it can cause a lot of problems down the line. By validating the input, you can catch this error early and fail fast, preventing further issues.
Dedicated validation libraries like ajv or Joi provide easy-to-use APIs that allow you to define validation rules for your input data. You can specify the type, format, and other constraints that your data should meet. These libraries also provide helpful error messages that can help you quickly identify the issues with your input.
Asynchronous programming is one of the key features of Node.js, allowing developers to write non-blocking code that can handle multiple requests concurrently. Promises are a widely used abstraction for handling asynchronous operations in Node.js. However, there are some important considerations to keep in mind when using promises, especially when returning them from functions.
One of the most important things to remember is to always await promises before returning them. This is because when you return a promise without awaiting it, the function that returns the promise won't appear in the stacktrace. This can make it difficult to understand the flow that leads to an error, especially if the cause of the abnormal behavior is inside the missing function.
To illustrate this point, let's consider an example. Imagine we have a function called fetchUserData() that retrieves user data from a database and returns a Promise that resolves to an object containing the user's name, email, and other details. Now, let's say we have another function called getUserEmail() that calls fetchUserData() and returns the user's email address. Here's what the code might look like:
function fetchUserData(userId) {
return db.query(`SELECT * FROM users WHERE id = ${userId}`);
}
function getUserEmail(userId) {
return fetchUserData(userId).then(user => user.email);
}
This code looks correct at first glance, but there's a subtle issue here. If fetchUserData() fails for some reason, the error stacktrace won't include getUserEmail(), making it harder to understand the flow of the code. To fix this, we need to await the promise returned by fetchUserData() before returning it in getUserEmail():
async function getUserEmail(userId) {
const user = await fetchUserData(userId);
return user.email;
}
By awaiting the promise before returning it, we ensure that the full error stacktrace is available in case of an error.
It's worth noting that this issue only arises when returning promises from functions. If you're simply calling a function that returns a promise, you don't need to worry about this as long as you're properly handling errors using catch blocks or try/catch statements.
In addition to ensuring the full error stacktrace is available, there are other benefits to always awaiting promises. For example, it makes it easier to read and reason about the code, as the flow is more explicit and predictable. It also makes it easier to compose promises and handle errors, as you don't need to worry about the order in which promises are resolved.
SInce the dawn of javascript a lot of developer have been putting some effort into gathering the best practices for writing javascript code and it turned into a package : ESLint which is the de-facto standard for checking possible code errors and fixing code style. Not only does it identify nitty-gritty spacing issues, but it also detects serious code anti-patterns, such as developers throwing errors without classification. ESLint can automatically fix code styles, but other tools like prettier and beautify are more powerful in formatting the fix and work in conjunction with ESLint. Using ESLint will save developers time that would have been wasted overthinking the project's code style.
On top of ESLint's standard rules that cover vanilla JavaScript, add Node.js specific plugins like eslint-plugin-node, eslint-plugin-mocha, and eslint-plugin-node-security. Many faulty Node.js code patterns might escape under the radar. For example, developers might require(variableAsPath) files with a variable given as a path, which allows attackers to execute any JS script. Node.js linters can detect such patterns and complain early.
Note: As Node is a JavaScript runtime, every JavaScript best practice also applies. See my JavaScript clean code guide.
As a developer, fortunately or unfortunately your job doesn't stop at building apps you also have to deploy them so you must be aware of the importance of taking into account the best practices for a successful deployment. A smooth deployment guarantees the best performance of your application, which means a better experience for your users. Here are some essential practices that you better follow:
Monitoring is a crucial aspect of building any software system, and Node.js applications are no exception. It involves collecting and analyzing data about the system's behavior, such as its performance, resource utilization, and error rates. The primary goal of monitoring is to detect issues before they affect end-users and to provide insights that help developers improve the application's reliability, scalability, and maintainability.
To begin monitoring a Node.js application, you must first define the key metrics that you want to track. These metrics may vary depending on the application's use case and complexity, but some common ones include response time, throughput, error rate, CPU usage, memory usage, and network latency. You can use tools like Prometheus, Grafana, or Datadog to collect and visualize these metrics in real-time.
Once you have defined the metrics to track, you can set up alerts that notify you when certain thresholds are crossed. For example, if the response time of an API endpoint exceeds a certain value, you can receive an email or a Slack notification to investigate the issue promptly. You can use tools like PagerDuty or OpsGenie to configure these alerts and integrate them into your workflow.
In addition to monitoring the application's runtime behavior, it's also essential to monitor its logs. Logs are messages generated by the application that describe its actions and events. They can help you identify errors, track user activity, and debug issues. You can use tools like ELK stack, Graylog, or Splunk (which all offer different advantages which i'm not gonna get into) to collect, store, and analyze your application's logs.
Another critical aspect of monitoring is tracing. Tracing involves following the execution path of a request or transaction across the system's components, including databases, caches, and external APIs. This helps you identify bottlenecks, understand the dependencies between components, and optimize the application's performance. You can use tools like Jaeger, Zipkin, or OpenTelemetry to trace requests and visualize their paths.
As we mentionned logging is a critical part of understanding what's happening within your application but it is a double-edged sword. If you're not careful, logging can quickly become overwhelming, making it difficult to identify the important information that you need. That's where smart logging comes in.
Smart logging is a process of logging that is designed to increase transparency by capturing only the most relevant information. Rather than simply logging every single event that occurs within an application, smart logging focuses on capturing key data that can be used to understand the flow of the application and diagnose issues as they occur. This approach ensures that the logs are not just a warehouse of debug statements, but a useful tool that can be used to improve the application and troubleshoot issues.
One of the most important aspects of smart logging is planning your logging platform from day one. This means thinking about how your logs will be collected, stored, and analyzed. You'll want to ensure that you're capturing the right data and that the data is being stored in a way that is easy to query and analyze. This will allow you to extract the information that you need, such as error rates, response times, and other key metrics.
Another key aspect of smart logging is ensuring that your logs are designed to be easy to reason about. This means adding context to your log statements so that you can easily understand what's happening within your application. For example, rather than simply logging an error message, you might include additional information such as the user who triggered the error or the specific component that caused the error. This context can be incredibly valuable when trying to diagnose issues within an application.
Finally, it's important to make sure that your logs are easy to read and analyze. This means using a consistent logging format and ensuring that your logs are stored in a way that can be easily queried and analyzed. Many developers choose to use a dedicated logging platform or service, such as Logtail or Papertrail, to help manage their logs and make them easier to work with.
As we already mentionned Logging is an essential part of any application as it helps developers understand how the application is performing and identify potential issues that may arise. However, routing logs within the application code can create problems that can have a significant impact on the application's performance.
When logs are routed within the application, it can create an unnecessary burden on the application's resources. This can cause the application to become slow and unresponsive, making it difficult for developers to identify and resolve issues. Additionally, routing logs within the application can lead to the loss of logs, which can make it difficult to diagnose issues when they arise.
To avoid these issues, it is important to separate the logging functionality from the application code. Developers should write logs to stdout using a logger utility, and then let the execution environment handle the routing of logs to their appropriate destination. By doing this, the application's resources are freed up, and the logging process becomes more efficient and reliable.
Here is an example of how logging can be done in Node.js thanks to winston:
const { createLogger, format, transports } = require('winston');
const logger = createLogger({
format: format.combine(
format.timestamp(),
format.json()
),
transports: [new transports.Console()],
});
function someFunction() {
logger.info('Some message');
}
module.exports = someFunction;
In this example, we are using the winston library to create a logger instance. We then use this logger to log messages within the someFunction() function. The logger is configured to log to the console, but this can be easily changed to log to other destinations such as a file or a remote logging service.
CPU-intensive tasks like SSL termination and gzip compression can potentially block the single-threaded Node.js event loop, leading to poor application performance and user experience.
One solution to this problem is to delegate these tasks to a reverse proxy. A reverse proxy is a server that sits between the client and the application server, forwarding client requests to the appropriate backend server and returning the server's response to the client. Reverse proxies are commonly used for load balancing, SSL termination, and caching.
By using a reverse proxy, you can offload the CPU-intensive tasks to a separate server, freeing up your Node.js application to handle only the application logic. This approach can lead to significant performance improvements, as the reverse proxy is designed to handle these tasks efficiently and effectively.
One popular reverse proxy solution is Traefik, which is an open-source reverse proxy and load balancer that supports multiple backends, including Docker, Kubernetes, and Swarm. Traefik is highly configurable and provides features like SSL termination, automatic service discovery, and health checking.
To use Traefik with your Node.js application, you can configure your application to listen on a specific port and then configure Traefik to route incoming requests to that port so your Node.js application can focus on the application logic.
It is important to ensure that your code runs smoothly and identically across all environments. One of the issues that can arise is that when you install packages from npm, it may fetch the latest patch version of the package, which can cause issues if that version has introduced breaking changes. To overcome this issue, you can lock your dependencies to a specific version.
One way to lock your dependencies is to use npm config files, specifically the .npmrc file. This file can be used to specify various configurations for npm, including setting the default package version to save. By setting the default package version to save as the exact version, rather than the latest, you can ensure that all environments use the same version of the package.
It is worth noting that as of npm version 5, dependencies are locked by default. This means that you don't need to take any additional steps to lock your dependencies, as it is done automatically.
Additionally, there are other package managers that provide dependency locking by default, such as Yarn. Yarn is a popular package manager that was created by Facebook, and it locks dependencies by default.
Node.js is a powerful platform for building scalable and high-performance applications. However, by default, Node.js only utilizes a single CPU core, even if your server has multiple cores available. This can result in a significant bottleneck for applications that need to handle a large number of requests simultaneously.
To overcome this limitation, it's important to utilize all the CPU cores available on your server for this we can use Node Cluster which is a built-in module in Node.js that allows developers to create child processes that share server ports. By using this module, you can take advantage of all the CPU cores available on your server. Here's an example of how to use it:
const cluster = require('cluster');
const numCPUs = require('os').cpus().length;
if (cluster.isPrimary) {
console.log(`Primary ${process.pid} is running`);
for (let i = 0; i < numCPUs; i++) {
cluster.fork();
}
cluster.on('exit', (worker) => {
console.log(`Worker ${worker.process.pid} died`);
});
} else {
const http = require('http');
http.createServer((req, res) => {
res.writeHead(200);
res.end('hello world\n');
}).listen(8000);
console.log(`Worker ${process.pid} started`);
}
In this example, we create child processes using the cluster.fork() method. Each child process runs the same code and shares the server port. This allows us to handle more requests simultaneously.
As an alternative you can use PM2, which is a popular process manager for Node.js applications. It can be used to run multiple instances of your application, each running on a separate CPU core. PM2 also provides features like automatic restarts and log management.
Here's an example of how to use PM2 to start multiple instances of your application:
pm2 start app.js -i max
In this example, the -i max flag tells PM2 to start as many instances of the application as there are CPU cores available on the server.
The V8 engine that powers Node.js has a soft limit on memory usage, which is around 1.4GB. Once the limit is reached, the application may experience significant performance issues, such as slow response times, crashes, and other stability issues. To avoid these problems, it's crucial to monitor memory usage in your Node.js applications.
There are several ways to measure and guard memory usage in Node.js. Here are some of the most popular methods:
const heapdump = require('heapdump');
heapdump.writeSnapshot('/path/to/heapdump.snap');
const used = process.memoryUsage().heapUsed / 1024 / 1024;
console.log(`The script uses approximately ${used} MB`);
I'm gonna say it again because it's important Node.js is a single-threaded application, meaning it can only execute one task at a time. If you use Node.js to serve static files, the server will have to wait for the files to be read from disk and sent to the client, which can be quite long thus cause performance issues, especially when serving a large number of static files.
To overcome this issue, it is recommended to use dedicated middleware, such as Nginx, S3, or a CDN to serve frontend assets. These tools are specifically designed to handle static files efficiently, which means that Node.js can do what it's made for allocate all its resources for serving dynamic content.
Stateless architecture refers to an approach where the server does not store any session or application data on its local disk or memory, but rather stores it externally in a database, cache, or other external storage. This means that each request is treated as a unique request, and the server does not maintain any information from previous requests.
The reason for this approach is that it enables horizontal scaling, where multiple servers can be added or removed as needed to handle an increase or decrease in traffic. If the servers are stateful, meaning they store information from previous requests, then adding or removing servers becomes more complicated, as the state must be synchronized between them.
By storing data externally, you also avoid the risk of losing data if a server crashes or goes down for maintenance. The data is still available and can be retrieved by another server in the cluster.
One way to achieve statelessness is by using serverless platforms such as AWS Lambda, Google Cloud Functions, or Azure Functions. These platforms handle the scaling for you and enforce stateless behavior by design. You only pay for the amount of time your code is running, which can result in significant cost savings.
Another way to achieve statelessness is to use external data stores for session management, caching, and file storage. For example, instead of using cookies to store session data on the server, you can store session data in a database or a cache such as Redis. Instead of storing uploaded files on the server, you can store them in a cloud-based storage service such as AWS S3.
Security is an important aspect of any web application, and it is essential to take measures to ensure that your code and dependencies are secure.
One way to address this issue is to use tools that can automatically detect vulnerabilities in your code and dependencies. These tools scan your codebase and its dependencies to identify potential security issues, such as outdated versions of libraries that may contain known vulnerabilities. Using these tools can help you stay up-to-date with the latest security patches and best practices, and can help prevent potential security breaches.
Npm audit is the perfect tool for the job, it's a built-in tool in Node.js that can help detect vulnerabilities in your project's dependencies. When you run npm install, npm audit automatically checks for vulnerabilities in the installed packages and provides a report on any issues found but you can also execute it as a standalone with npm audit and incorporate it into your CI.
Setting the NODE_ENV environment variable to 'production' or 'development' is an important aspect of Node.js development that should not be overlooked. In a typical Node.js application, there are often two environments: production and development. The production environment is where your application runs when it is deployed to a live server, while the development environment is where you develop and test your application.
Setting the NODE_ENV environment variable to 'production' tells your application that it is running in the production environment. This enables your application to take advantage of production optimizations that are built into many npm packages. These optimizations are designed to improve the performance and efficiency of your application in production environments.
One example of an optimization that is commonly used in production environments is minification. Minification is a process of removing unnecessary characters and whitespace from your code, reducing its size and improving load times. When NODE_ENV is set to 'production', many packages will automatically minify your code, resulting in faster load times and improved performance.
Another optimization that is commonly used in production environments is caching. Caching is a technique that stores frequently accessed data in memory, reducing the number of times your application needs to access disk or network resources. When NODE_ENV is set to 'production', many packages will automatically enable caching, resulting in faster response times and improved performance.
In Node.js, we can achieve automated, atomic, and zero-downtime deployments using containerization tools such as Docker. Here's an example of how we can create a Docker container for a Node.js application:
FROM node:lts-alpine
WORKDIR /app
COPY package.json /app
RUN npm ci --omit=dev
COPY . /app
CMD ["npm", "start"]
Here, we start by creating a base image of the latest version of Node.js. We then set the working directory to /app, copy the package.json file, and install the dependencies. We then copy the rest of the application files, and finally, start the application using the npm start command.
We can then use a CI tool such as Jenkins to build, test, and deploy the Docker container to a production environment. Here's an example of how we can create a Jenkins pipeline for our Node.js application:
pipeline {
agent any
stages {
stage('Build') {
steps {
sh 'docker build -t myapp .'
}
}
stage('Test') {
steps {
sh 'docker run --rm myapp npm test'
}
}
stage('Deploy') {
steps {
sh 'docker stop myapp || true'
sh 'docker rm myapp || true'
sh 'docker run -d --name myapp -p 80:3000 myapp'
}
}
}
}
Here, we define a Jenkins pipeline with three stages: Build, Test, and Deploy. In the Build stage, we build the Docker container using the docker build command. In the Test stage, we run the tests inside the Docker container using the docker run command. Finally, in the Deploy stage, we stop and remove any existing containers with the same name, and then run a new container with the name myapp, exposing port 3000 to port 80 on the host machine.
As with any technology, it's essential to stay up-to-date with best practices and recommended usage. One such best practice is to always use a Long-term support (LTS) release of Node.js. LTS releases are stable, well-tested, and receive critical bug fixes, security updates, and performance improvements for an extended period. On the other hand, non-LTS releases have a shorter lifespan and may be more prone to bugs, security issues, and breaking changes.
To illustrate this, let's take a look at the Node.js release schedule:
Even-numbered releases are LTS versions that receive support for 30 months from the date of their release. Odd-numbered releases are the latest and greatest features, but only receive support for six months.
For example, as of this writing, the latest LTS release is version 18.x, while the latest non-LTS release is version 19.x. If you're building a new application, it's recommended to use the latest LTS version, as it will be supported for a more extended period, and you can benefit from critical bug fixes and security updates.
it is crucial to ensure that our production code always uses the exact version of the packages we have tested it with. This is where the npm ci command comes in handy.
The npm ci command is a strict, clean install of your dependencies that ensures that they exactly match the versions specified in your package.json and package-lock.json files. Unlike npm install, which can install newer versions of the packages, npm ci installs only the exact versions specified, making it easier to maintain a consistent and reliable production environment.
Using npm ci is particularly important in automated environments such as continuous integration (CI) pipelines. In these environments, you want to ensure that every server in the production cluster is running the same code. Without using npm ci, there is a risk of different servers running different code versions, which can lead to issues in production.
This command will remove any existing node_modules folder and install all dependencies from scratch based on the versions specified in your package-lock.json file.
It is important to note that npm ci is not a replacement for npm install in all cases. While it is perfect for automated environments and production deployments, npm install is still the go-to command during development. npm install allows for the installation of newer versions of packages, which is essential during development when we may need to upgrade packages to fix bugs or add new features.
By following these best practices, you can ensure that your Node applications are reliable, performant, and secure. While there are many other best practices to keep in mind, these are some of the most important ones to focus on. Keep learning, experimenting, and improving your skills as a Node developer, and you'll be able to build amazing applications that delight your users.
Honestly i am no one, i've just been coding for 3 years now and i like to document every solutions to every problem i encounter. Extracting as much code snippets and tutorials i can so that if i ever need it again i just have to pop back here and get a quick refresher.
Feel free to me through this learning journey by providing any feedback and if you wanna support me: