Categories
Node.js Best Practices

Node.js Best Practices — Maintaining Production Code

Node.js is a popular runtime to write apps for. These apps are often production quality apps that are used by many people. To make maintaining them easier, we’ve to set some guidelines for people to follow.

In this article, we’ll look at the best practices for maintaining production code.

Discover Errors and Downtime Using APM Products

Application monitor and performance (APM) products check our codebase and API so that it can go beyond traditional monitoring and measure the overall user experience across services and tiers.

Some of these like Scout can check for repeated and slow database queries for example so that we can look at them and have to fix those issues that are listed.

Otherwise, we may spend lots of time measuring API performance and downtime and we may still miss the slowest code parts in production.

Make Our Code Production-Ready

We should plan for production on day 1 so that we don’t have to worry about issues that only arise after deploying to production.

Measure and Guard Memory Usage

Our apps shouldn’t be using too much memory. Also, we should be aware of memory leaks in our apps. In small apps, we may gauge memory periodically using shell commands, but in medium and large apps we can use more robust monitor tools to watch for memory usage.

An APM tool may help with this as well. For Express apps, we can add the express-status-monitor package to watch the status of our app, including CPU and memory usage, response time, request per second, and more.

We just have to install the package by running:

npm i `express-status-monitor`

Then we can use it by writing the following code:

const express = require('express');
const bodyParser = require('body-parser');

const app = express();
app.use(require('express-status-monitor')());
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));

app.get('/', (req, res, next) => {
  res.send('hello')
});

app.listen(3000, () => console.log('server started'));

We just put the package straight into our app with app.use(require(‘express-status-monitor’)()); .

Then when we go to the /status page, we’ll see all the performance and health metrics listed.

Get Frontend Assets Out of Node

Frontend assets shouldn’t be in our Node app. Instead, we can host them in their own location with services like S3. This way, our front end code isn’t tightly coupled with our Node app, and it’s also faster since our Node app doesn’t have to hosts these assets in addition to running its own code.

If we move them to another server, then we won’t tie up our Node app by serving hundreds of HTML, CSS, JavaScript, and media files.

Be Stateless and Kill Our Servers Almost Every Day

We should store any kind of data with external data stores. This includes sessions, cookies, cache and uploaded files. We can enforce this by killing and rebuild our servers daily.

The reason we want to do this is to make our app independent of its data. Otherwise, failure in the app’s server will result in downtime instead of just killing the faulty server. Scaling out will also be more challenging because of the reliance on a specific server.

Use Tools that Automatically Detect Vulnerabilities

Tools that detect vulnerabilities helps us check for them without doing any work ourselves. This is good because we don’t want to check all our code and external packages manually since there’s so much code.

These tools warn us if it finds any vulnerabilities so we can fix them soon.

Assign a Transaction ID to Each Log Statement

Assigning transaction IDs to log statements helps us identify each log entry easily as it has a unique ID associated with it. For example, we can log each entry with a unique ID with morgan by using it in conjunction with the uuid module and our own middleware.

For instance, we can write the following:

const express = require('express');
const bodyParser = require('body-parser');
const morgan = require('morgan')
const uuid = require('uuid')
const fs = require('fs');
const path = require('path');

morgan.token('id', (req) => {
  return req.id
})

const app = express();
app.use((req, res, next) => {
  req.id = uuid.v4()
  next()
})
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));

const accessLogStream = fs.createWriteStream(path.join(__dirname, 'access.log'), { flags: 'a' })
app.use(morgan(':id :method :url :response-time', { stream: accessLogStream }))

app.get('/', (req, res, next) => {
  res.send('hello');
});

app.listen(3000, () => console.log('server started'));

In the code above, we added our own middleware to set req.id to a UUID. Then we write that to the file along with the response time.

Conclusion

We can add monitoring to our app with APM products. To free our app from unnecessary work, we should move our front end assets out of our Node app. Also, Node apps should be stateless. We can enforce that by killing and rebuilding the server every day. It prevents issues with losing data that we need on a server.

ID should be assigned to log entries so that we can look them up later.

Categories
Node.js Best Practices

Node.js Best Practices — Production and Security

Node.js is a popular runtime to write apps for. These apps are often production quality apps that are used by many people. To make maintaining them easier, we’ve to set some guidelines for people to follow.

In this article, we’ll look at deployment and security-related best practices for Node apps.

Set NODE_ENV=production

We should set NODE_ENV to production for production environments and development for development. Setting NODE_ENV to production makes production-related optimizations activate.

Omitting this single property may degrade performance significantly. For instance, using Express without NODE_ENV set to production makes it slower by a factor of 3.

In Express, if NODE_ENV is production , it’ll cache view templates and CSS generated from CSS extensions, and it’ll also generate less verbose error messages to keep them from being exposed to the public.

Design Automated, Atomic and Zero-Downtime Deployments

Deployments need to be automated so that we don’t have to worry about it once the automated deployment pipeline is set up. This way, we don’t have to worry about running into issues caused by human error. Manual deployments are also much slower than automatic.

Even better, we can create automated sandbox environments with Docker so that each app runs in its own environment. This solves lots of issues with conflicting runtimes, packages, and environment variables.

Otherwise, we have to deploy manually every time, which may cause issues. Also, we have to watch it to make sure that it’s done, and so people are less likely to deploy apps since it’s such a painful and error-prone process.

Use an LTS Release of Node.js

Long Term Service (LTS) releases of Node are supported for much longer than non-LTS Node versions. They constantly receive critical bug fixes, security updates, and performance improvements. Non-LTS releases are only supported for a few months after its releases with updates.

Therefore, to avoid having to upgrade Node all the time, we should use the LTS versions.

Embrace Linter Security Rules

ESLint plugins like eslint-plugin-security checks for security issues with our code so that we can fix them as early as possible. This helps catch security weaknesses like eval , invoking a child process, or importing a module with user inputted string.

With it, we can check before our code is committed or pushed. With this, we also follow security best practices all the time by everyone in the team.

Limit Concurrent Requests Using a Middleware

Denial of Service (DOS) attacks are very popular and easy to do. Therefore, we should implement rate-limiting using a service like load balances, cloud firewalls, reverse proxy, or a package in our app.

To implement rate-limiting in our app endpoints, we can use packages like rate-limiter-flexible or express-rate-limit .

For instance, we can use express-rate-limit as follows to add a rate limit to all the endpoints in our Express app:

const express = require('express');
const bodyParser = require('body-parser');
const rateLimit = require("express-rate-limit");

const app = express();
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));
const limiter = rateLimit({
  windowMs: 15 * 60 * 1000,
  max: 100
});

app.use(limiter);

app.get('/', (req, res, next) => {
  res.send('hello');
});

app.listen(3000, () => console.log('server started'));
module.exports = app;

As we can see, we only have to add a simple middleware to prevent denial of service attacks on our app. Therefore, we should do it from the beginning for apps of any size.

Extract Secrets from Configuration Files or Use Packages to Encrypt Them

Configuration files often have secret data that shouldn’t be seen by most people. Therefore, we should never store them in our source code. Instead, we have to make sure of secret-management systems like Vault, Docker secrets, or use environment variables.

We should have pre-commit or push hooks to prevent accidentally committing secrets to code. Source control can be mistakenly made public, which exposes the secrets to other people that shouldn’t see them.

For example, we can use the dotenv package to store environment variables for Node apps. To use it, we write:

const express = require('express');
const bodyParser = require('body-parser');
require('dotenv').config();

const app = express();

app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));

app.get('/', (req, res) => {
  res.send(process.env.DB_HOST);
});

app.listen(3000, () => console.log('server started'));

Given that our .env file has:

DB_HOST=foo

Once we ran:

require('dotenv').config();

Then process.env.DB_HOST returns 'foo' , so we’ll see foo displayed on the screen if we go to / .

Conclusion

We should make sure that NODE_ENV is set to production so that optimizations are applied for packages like Express.

Any deployment should be automated. Also, Node.js should be the LTS version. Also, we should check for security vulnerabilities in our code and also rate limit our API endpoints to prevent denial of service attacks.

Finally, we should keep secrets out of our code.

Categories
Node.js Best Practices

Node.js Best Practices — Going to Production

Node.js is a popular runtime to write apps for. These apps are often production quality apps that are used by many people. To make maintaining them easier, we’ve to set some guidelines for people to follow.

In this article, we’ll look at what we should with our Node app before and after going to production.

Use Production-Like Environment for E2E Testing

Running end to end tests in a production-like environment ensures that we won’t run into different issues in productions that we catch beforehand.

Also, we should run our tests in a database with clean data so that we can repeat our tests.

Refactor Regularly Using Static Analysis Tools

Before putting our code to production, we should refactor our code so that it runs quickly in production. The automated tests will help us to make sure that refactoring won’t break any existing functionality.

Poor code quality will create more bugs and performance issues that are hard to fix.

Carefully Choose Our CI Platform

Jenkins and CircleCI are popular platforms for continuous integration. Having a CI (continuous integration) pipeline lets us run tests and deploy automatically in the background rather than running everything manually. It frees us to do other kinds of work and frees us from manually managing the infrastructure.

We’ve to choose carefully since migration from one to the other will be a pain.

Monitoring Our App

We should monitor our app so that our app runs properly and not taking up too many resources. To do this, we can use monitoring tools and add health check endpoints to check if our app is running.

This way, we don’t have to let our customers tell us that our app has failed.

Increase Transparency Using Smart Logging

Logging lets us troubleshoot problems easily by spotting the activities in the lo that may be causing problems. Most logging platforms can control how logs are collected, stored, and analyzed to ensure that it’s only storing the data that we want.

Delegate Anything Possible to a Reverse Proxy

If something can be done with a reverse proxy, then they don’t belong in our app. CPU intensive tasks like SSL, Gzipping, termination should all be done on a reverse proxy to take the load off our app.

This is especially important for Node apps since it only runs on one thread, so we don’t want to tie it up by making it do infrastructure-related tasks that belongs to the reverse proxy.

Lock Dependencies

We should lock our app’s dependencies so that they won’t change versions across environments. Nowadays, this should be done automatically since npm install generates a package-lock.json if it doesn’t exist. If it does exist, then npm install will use the versions in the file to install the dependencies.

If it doesn’t exist in our repo or if we use fine-grained control of how the versions are locked, we can run npm shrinkwrap . This command repurposes package-lock.json into a publishable npm-shrinkwrap.json or creates a new one.

It takes precedence over package-lock.json .

Guard Process Uptime Using the Right Tool

Our app must be restarted when it fails. We can use Forever or PM2 to watch our app and restart it when it crashes. If we have a cluster, then we also have to manage that.

Utilize All CPU Cores

We should use all the CPU cores to run our app with the fastest performance possible. A CPU core is useless if it’s left idling. If that’s the case, we should replicate the Node processes and utilize all CPUs. For small apps, we may use Node Cluster or PM2. Otherwise, we may use a Docker cluster like ECS.

Create a ‘Maintenance Endpoint’

We can use this to securely expose diagnostic information from our app without logging into the server. Some information are just easier to get using code.

In Node, we can use the os module to expose information about our server. For example, we can get the platform for our app as follows and return it via an endpoint:

const express = require('express');
const bodyParser = require('body-parser');
const os = require('os');

const app = express();
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));

app.get('/', (req, res, next) => {
  res.send(os.platform());
});

app.listen(3000, () => console.log('server started'));
module.exports = app;

Of course, in a production app, this should be secured with authentication.

Conclusion

Before going to production, we should have a set of end to end tests that run in a clean production-like environment. The data should be reset each test run so that they actually run properly. This also helps with testing after refactoring.

We should make sure that we have automated deploy to free our time for other tasks.

Also, make sure that our server’s CPU cores are all utilized.

Finally, we may want to create a secure maintenance endpoint to expose some information to us without logging into the server.

Categories
Node.js Best Practices

Node.js Best Practices — Security Attacks

Node.js is a popular runtime to write apps for. These apps are often production quality apps that are used by many people. To make maintaining them easier, we’ve to set some guidelines for people to follow.

In this article, we’ll look at some basic security practices to be aware of writing Node apps.

Prevent Query Injection Vulnerabilities with ORM/ODM Libraries

We should never pass in user-inputted strings straight into our app to prevent SQL or NoSQL injection attacks. Inputs should be validated and sanitized before being passed into database queries.

All reputable data access libraries like Sequelize, Knex, and Mongoose have built-in protection against script injection attacks.

Unsanitized strings can easily destroy data and expose them to unauthorized parties if they’re left unsanitized.

Collection of Generic Security Best Practices

We should keep up-to-date with general security best practices so we can implement them when we’re developing and running apps.

Adjust the HTTP Response Headers for Enhanced Security

We can use modules like helmet to secure headers to prevent attacks from using common attacks like cross-site scripting with our apps.

To add helmet and use it, we run:

npm i helmet

and then use it as follows:

const express = require('express');
const bodyParser = require('body-parser');
const helmet = require('helmet');
const app = express();
app.use(helmet());

app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));

app.get('/', (req, res) => {
  res.send('hello');
});

app.listen(3000, () => console.log('server started'));

Helmet automatically protects us from cross-site scripting, enables strict transport security, and keep clients from sniffing the MIME types from responses.

The X-Powered-By header is also removed from the response so that attackers won’t know that our app is an Express app.

Constantly and Automatically Inspect for Vulnerable Dependencies

We can use npm audit or snyk to check for packages with vulnerable dependencies before going to production. Otherwise, attacks may take advantage of the vulnerabilities to commit attacks.

Avoid Using the Node.js crypto Library for Handling Passwords, use Bcrypt

bcrypt provides hash and salt functionality. Therefore it’s better for handling secrets than the built-in crypto library. It’s also faster.

We don’t want attackers to be able to brute-force passwords and tokens with dictionary attacks.

Escape HTML, JS and CSS Output

We should escape these kinds of code so that attacks can’t run malicious client-side code with our app. Dedicated libraries can explicitly mark the data as pure content and should never be executed.

Validate Incoming JSON Schemas

JSON schemas should be validated to make sure that the income request payload has valid data. For instance, we can use the jsonschema library to validate the structure and values of the JSON that’s sent.

We can use the jsonschema library as follows with an Express app:

const express = require('express');
const bodyParser = require('body-parser');
const Validator = require('jsonschema').Validator;
const v = new Validator();
const app = express();

const addressSchema = {
  "id": "/SimpleAddress",
  "type": "object",
  "properties": {
    "address": { "type": "string" },
  },
  "required": ["address"]
};

const schema = {
  "id": "/SimplePerson",
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "address": { "$ref": "/SimpleAddress" },
  },
  "required": ["name", "address"]
};

v.addSchema(addressSchema, '/SimpleAddress');
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));

app.post('/person', (req, res) => {
  if (v.validate(req.body, schema).errors.length) {
    return res.send(400)
  }
  res.send('success');
});

app.listen(3000, () => console.log('server started'));

In the code above, we required the jsonschema library and use its validator. Then we defined the /SimpleAddress schema, which is referenced by the /SimplePerson schema.

We add the /SimpleAddress schema with:

v.addSchema(addressSchema, '/SimpleAddress');

to reference it in /SimplePerson .

Then we can check our request body against our schema with:

v.validate(req.body, schema).errors.length

Then we stop the request from proceeding if the request body fails validation.

Support blacklisting JWTs

JSON Web Tokens (JWTs) that were used for malicious user activity should be revoked. Therefore, our app needs a way to revoke these tokens.

Conclusion

We should secure our app by checking for vulnerabilities and revoking tokens that were used for malicious purposes. Also, we need to take steps to prevent malicious from running on client and server-side by sanitizing data everywhere.

Finally, we should validate request bodies to make sure that valid data is submitted to our app.

Categories
Node.js Best Practices

Node.js Best Practices — Error Handling

Node.js is a popular runtime to write apps for. These apps are often production quality apps that are used by many people. To make maintaining them easier, we have to set some guidelines for people to follow.

In this article, we’ll look at the best practices for error handling in Node.js apps.

Use only the built-in Error object

In JavaScript, we can create a subclass of the error object, so there’s no need to create a custom type or return some error value to throw errors.

We can either instantiate the Error class or a subclass of the Error class. This way, it’ll increase uniformity and decrease confusion. It’s much less headache for everyone reading the code.

For example, we can throw an Error instance by writing:

throw new Error('error');

We can catch errors and handle them gracefully by using the catch block as follows:

try {
  foo();
} catch (e) {
  if (e instanceof RangeError) {
    console.error(`${e.name} - ${e.message}`)
  }
}

The code above only handles RangeError instances.

We can get the error name and stack trace from the Error instance. Also, we can create our own error class by subclassing the Error class:

class CustomError extends Error {
  constructor(foo = 'bar', ...params) {
    super(...params)
    if (Error.captureStackTrace) {
      Error.captureStackTrace(this, CustomError)
    }

    this.name = 'CustomError'
    this.foo = foo
    this.date = new Date()
  }
}

Then we can throw a CustomError instance and catch it by writing:

try {
  throw new CustomError('baz');
} catch (e) {
  if (e instanceof CustomError) {
    console.error(`${e.name} - ${e.foo}`)
  }
}

Distinguish Operational vs Programmer Errors

Operational errors by the user should be handled by the user. For instance, if invalid input is entered, we should handle it by sending them an error message.

On the other hand, if the error is a programmer error, then we should fix them. Keeping the programmer error unfixed isn’t a good idea since it leads to bad user experience and data corruption.

For user errors, we should send the client 400 series response codes for web apps. 400 is for bad input, 401 for unauthorized, and 403 for forbidden.

Handle Errors Centrally, Not within an Express Middleware

In Express apps, error handling should be done centrally in its own middleware rather than spread out in multiple middlewares.

For instance, we should write something like the following code to handle errors:

const express = require('express');
const bodyParser = require('body-parser');

const app = express();

app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));

app.get('/', (req, res, next) => {
  try {
    throw new Error('error')
    res.send('hello')
  } catch (err) {
    next(err)
  }
});

app.use((err, req, res, next) => {
  res.send('error occurred')
})

app.listen(3000, () => console.log('server started'));

In the code above, we added an error handler to the end of the app so that it’ll run when all routes call next .

This way, we don’t have to write multiple pieces of code to handle errors. In the code above:

app.use((err, req, res, next) => {
  res.send('error occurred')
})

is the error handler middleware. The next function in the GET route calls the error handler since it threw an error.

Document API errors using Swagger

Documentation is very important since it lets people know how to use the API and make sure that changes to the API won’t accidentally create undesirable behavior.

For instance, adding Swagger to an Express app is easy since we have the Swagger UI Express package to do it for us. We can install the Swagger UI Express package by writing:

npm install swagger-ui-express

Then we can create a swagger.json file as follows:

{
  "openapi": "3.0.0",
  "info": {
    "title": "Sample API",
    "description": "Optional multiline or single-line description in [CommonMark](http://commonmark.org/help/) or HTML.",
    "version": "0.1.9"
  },
  "servers": [
    {
      "url": "https://UnwittingRudeObservation--five-nine.repl.co",
      "description": "Optional server description, e.g. Main (production) server"
    },
    {
      "url": "https://UnwittingRudeObservation--five-nine.repl.co",
      "description": "Optional server description, e.g. Internal staging server for testing"
    }
  ],
  "paths": {
    "/": {
      "get": {
        "summary": "Returns a hello message.",
        "description": "Optional extended description in CommonMark or HTML.",
        "responses": {
          "200": {
            "description": "A JSON string message",
            "content": {
              "application/json": {
                "schema": {
                  "type": "string",
                  "items": {
                    "type": "string"
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

Then in our app, we can write:

const express = require('express');
const bodyParser = require('body-parser');
const swaggerUi = require('swagger-ui-express');
const swaggerDocument = require('./swagger.json');
const app = express();

app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));

app.get('/', (req, res, next) => {
  res.send('hello')
});

app.use('/api-docs', swaggerUi.serve, swaggerUi.setup(swaggerDocument));
app.listen(3000, () => console.log('server started'));

We can edit our Swagger API documentation by going to https://editor.swagger.io/.

Conclusion

If we want to throw errors in our Node app, we should throw an instance of Error or subclasses of Error . Catching errors can be done in the catch block for Node apps. It should also be handled by a central error handler for Express apps.

To make using and changing the API easy, we should use Swagger to document the routes that our API has. For Express apps, we can use Swagger UI Express to serve our swagger.json file in an easy to read format as a separate page.

For user errors, we should send errors back to them. Programmer errors should be fixed by us.