Author: John Au-Yeung

Like any kind of apps, JavaScript apps also have to be written well.

Otherwise, we run into all kinds of issues later on.

In this article, we’ll look at some best practices we should follow when writing Node apps.

Use JWT-Based, Stateless Authentication

We can authenticate by using JSON web tokens.

A JSON web token consists of 3 parts.

They include:

• header with the type of the token and hashing algorithm
• payload has the claims
• a signature that signs the payload.

We can add it easily with some add-ons.

For instance, we can use the koa-jwt package to add the token.

We can write:

const koa = require('koa')
const jwt = require('koa-jwt')

const app = koa()

app.use(jwt({
  secret: 'secret'
}))

// Protected middleware
app.use(function *(){
  this.body = {
    foo: 'bar'
  }
})

We just call app.use to use the jwt middleware.

The object has the secret to sign the token.

Then we added a protected middleware after that.

The token content will be available with this.state.user .

The JWT module doesn’t depend on any database layer.

They’re all verified on their own.

They can also contain the time to live values.

To ensure that our communication is secure, we still have to ensure that API endpoints are available through an HTTPS connection.

Use Conditional Requests

Conditional requests are HTTP requests that have different results depending on the request header values.

The headers check whether a version of a resource stored on the server match a given version of the same resource.

The headers can be the timestamp since the last modification.

It can also be an entity tag, which differs for each version.

The headers are:

• Last-Modified which indicates when the resource was last modified
• Etag which is the entity tag
• If-Modified-Since which is used with the Last-Modifed header
• If-None-Match used with the Etag header.

Use Rate Limiting

We can limit the number of requests that can be made to our API.

To tell our API clients how many requests are left, we can set the following response headers:

• X-Rate-Limit-Limit, tells us the number of requests allows in a given time interval
• X-Rate-Limit-Remaining, the number of requests remaining in the same interval
• X-Rate-Limit-Reset , tells us the time when the rate limit will be reset

We can add libraries to add rate limit capabilities to our app.

With Koa, we can use the koa-ratelimit package.

Create a Proper API Documentation

It’s hard to know how to use our API without any documentation.

To make our lives easier, we can use API Blueprint or Swagger to create our documentation.

Future of APIs

There’re alternatives to REST.

We can use GraphQL to listen to HTTP requests, but it has type checking and we can selectively select resources.

With type checking and the ability to selectively query resources, we can be more efficient and reduce the chance of errors.

Conclusion

We can use JWT for authentication.

Conditional requests let us make requests differently according to headers.

Alternatives to REST APIs should also be considered.

Like any kind of apps, JavaScript apps also have to be written well.

Otherwise, we run into all kinds of issues later on.

In this article, we’ll look at some best practices we should follow when writing Node apps.

Structure Express Applications

We should have the following folder structure for our Express app:

src/
  config/
  controllers/
  providers/
  services/
  models/
  routes.js
  db.js
  app.js
test/
  unit/
  integration/
server.js
(cluster.js)
test.js

We have the src folder with the code that runs in production.

config has config.

controllers have the controllers.

providers have the logic for the controller routes.

services have the business logic.

models have the database models.

routes.js loads all routes.

db.js has the database routes.

app.js loads the Express app.

test has the tests.

unit has the unit tests.

integration has the integration tests.

server.js in the entry point of the Express app.

cluster.js is an optional file for creating clusters.

test.js is the main test file to run all the tests in the test directory.

Improve Express.js Performance and Reliability

There are a few ways to improve performance and reliability.

Set NODE_ENV to production

We should set the NODE_ENV environment variable to production so we get the benefits of the production config.

It’s 3 times than in dev.

This is because there’s compression and caching to make our app faster.

We can either run:

export NODE_ENV=production

to set the environment variable

or:

NODE_ENV=production node server.js

to set the environment variable and run the app at the same time.

Enable Gzip Compression

We can enable gzip compression for assets to do compression on our assets.

We can install the compression middleware by running:

npm i compression

Then we can use it by writing:

const compression = require('compression')
const express = require('express')
const app = express()
app.use(compression())

This isn’t the best way to do gzip compression since it uses resources on the Express app.

Instead, we can enable gzip in the Nginx instead to offload the work to the reverse proxy.

Always Use Asynchronous Functions

If we have anything other than some simple operations, we should probably use async code.

We should promises most of the time or async/await for short.

For example, we can write:

(async () => {
  const foo = () => {
    //...
    return val
  }
​
  const val = await asyncFunction;
})()

We have an asyncFunction that returns a promise, so we use await to get the result.

We can’t run synchronous functions on different threads since Node is single-threaded.

Therefore, we can only use async code to run long-running operations.

We can also spawn child processes or create multiple instances of our app to run different tasks on different processes.

Logging Correctly

We should collect our logs in a central location.

This way, we can find them when we need them.

Winston and Morgan and useful logging packages that can integrate with other services for centralized logging.

We can also use some service like Sematext to do logging.

For example, we can write:

const { stLogger, stHttpLoggerMiddleware } = require('sematext-agent-express')
​
const express = require('express')
const app = express()
app.use(stHttpLoggerMiddleware)
​

app.get('/api', (req, res, next) => {
  stLogger.info('An info.')
  stLogger.debug('A debug.')
  stLogger.warn('A warning.')
  stLogger.error('An error.')

  res.send('Hello World.')
})

We have the the sematext-agent-express package which has a logger than logs to the Sematext service.

Then we get the logs in a nice dashboard with the service.

Conclusion

We can better structure our Express app and run our production Express app in production mode.

Also, logging can be easy with a centralized logging service.

Like any kind of apps, JavaScript apps also have to be written well.

Otherwise, we run into all kinds of issues later on.

In this article, we’ll look at some best practices we should follow when writing Node apps.

Use Promises for Async Error Handling

We should use promises for async error handling.

Handling async errors in the callback style would be a mess if we have many nested callbacks.

Node style callbacks don’t allow us to chain the async calls without nesting.

To avoid this, we should make sure that we use promises.

For example, instead of writing:

getData(someParameter, function(err, result){
    if(err !== null && err !== undefined)
    getMoreData(a, function(err, result){
          if(err !== null && err !== undefined)
        getMoreData(b, function(c){
                getMoreData(d, function(e){
                    //...
                });
            });
        });
    });
});

We have 4 nested callbacks, which is definitely hard to read even if if we omit the logic code.

Instead, we write:

doWork()
  .then(doWork)
  .then(doMoreWork)
  .then(doWork)
  .catch(errorHandler)
  .then(verify);

We can make this a long cleaner since promises has a then method which we can pass callbacks into.

If we want to catch errors, we use the catch method.

We can also use the async and await syntax to chang promises and use try-catch to catch errors.

For instance, we can write:

const work = async () =>
  try {
    const r1 = await doWork();
    const r2 = await doMoreWork();
    const r3 = await doWork();
    const r4 = await verify();
  }
  catch (e) {
    `errorHandler(e)
  }
}`

Each then callback returns a promise so we can use await on them.

Use Only the Built-in Error Object

The built-in error constructor should be used to create errors.

The Error constructor has the error message and stack trace.

They’re useful for troubleshooting issues which will be lost if we throw anything else

For example, instead of writing:

throw 'error';

We write:

throw new Error("error");

It’ll have the stack trace up to where the error is thrown.

Distinguish Operational vs Programmer Errors

We should distinguish operational vs programmer errors.

Operational errors are errors where the impact of the error is fully understood and can be handled.

Programmer errors refer to unknown code failures where we need to gracefully restart the application.

For the errors we can handle, we should be able to handle them and avoid restating our app.

We can throw errors, by writing:

if(!product) {
  throw new Error("no product selected");
}

const myEmitter = new MyEmitter();
myEmitter.emit('error', new Error('error'));

We can throw errors in synchronous code and event emitters.

We can throw an error in a promise with:

new Promise((resolve, reject) => {
  DAL.getProduct(productToAdd)
    .then((product) =>{
       if(!product)
         return reject(new Error("no product added"));
       }
    })
});

We have a promise that calls reject with an Error instance to throw the error.

These are the right ways to throw errors.

Other ways that are around include listening to the uncaughtException event:

process.on('uncaughtException', (error) => {
  if (!error.isOperational)
    process.exit(1);
});

Listening to the uncaughtException event changes the behavior of the event so it shouldn’t be listened to.

process.exit is also a bad way to end a program because it abruptly ends the program.

Conclusion

We should throw and catch errors in the right way.

This way our app will handle them gracefully.

Like any kind of apps, JavaScript apps also have to be written well.

Otherwise, we run into all kinds of issues later on.

In this article, we’ll look at some best practices we should follow when writing Node apps.

Handle Errors and Exceptions Properly

We should handle errors and exceptions properly on our Express app.

In synchronous code or an async function, we can use try-catch:

async function foo() {
  try {
    const baz = await bar()
    return baz
  } catch (err) {
    console.error(err);
  }
}

If we want to catch errors in middleware, we can create our own middleware to catch errors.

For example, we can write:

function errorHandler(err, req, res, next) {
  console.error(err)
  res.status(err.status || 500).send(err.message)
}

router.use(errorHandler)

We have the errorHandler middleware to catch errors from other middleware that are added before it.

Then we can call router.use or app.use with the errorHandler to handle the error.

Watch Out For Memory Leaks

We should watch for memory leaks so that our app doesn’t run out of memory.

Increasing memory usage is bad if it happens continuously.

This means that the app keeps using memory.

There’re apps like Sematext Agent Express module which lets us watch the CPU and memory usage of our app.

With this package, we can integrate it with:

const { stMonitor, stLogger, stHttpLoggerMiddleware } =
require('sematext-agent-express')
stMonitor.start()

const express = require('express')
const app = express()
app.use(stHttpLoggerMiddleware)

We just call stMonitor.start() to start monitoring with Sematext when our app starts.

Other tools like Scout also shows us the memory usage of each line of code.

We can see which ones are using more memory and the times that uses more memory.

These can also monitor each request for performance issues.

JavaScript

We can improve our Express app’s JavaScript code to make it easier to test and maintain.

Pure Functions

Pure functions are functions that let returns something and don’t change any outer state.

If we pass in the same parameters to them, then they’ll always return the same value.

This makes their behavior predictable and simplifies everyone’s lives.

We can create new objects instead of mutating existing objects with pie functions.

Some examples of pure functions in JavaScript include the array instance’s map and filter methods and many more.

Object Parameters

To make working with parameters, we should reduce the number of parameters in our function.

One easy way to do it is to add an object parameter to our function.

Then we can use destructuring to destructure the properties into variables.

This way, we won’t have to worry about the order of the parameters.

For example, we can write:

const foo = ({ a, b, c }) => {
  const sum = a + b + c;
  return sum;
}

We just take the a , b , and c parameters and use them as variables.

Write Tests

Tests are great for catching regressions.

This way, if we change our code, we can have peace of mind that we didn’t kill any existing code if the existing tests pass.

With JavaScript, there are many test frameworks, including Mocha, Chai, Jasmine, and Jest.

We can use any of them to run test.

With Chai, we can write:

const chai = require('chai')
const expect = chai.expect

const foo = require('./src/foo')

describe('foo', function () {
  it('should be a function', function () {
    expect(foo).to.be.a('function')
  })
})

to import the foo file and run tests on it with Chai.

Conclusion

We should handle exceptions properly.

Also, we can use tests to prevent errors.

Pure functions and object parameters also help us write cleaner code.

Like any kind of apps, JavaScript apps also have to be written well.

Otherwise, we run into all kinds of issues later on.

In this article, we’ll look at some best practices we should follow when writing Node apps.

DevOps Tools

DevOps tools will make configuring our environment and server setup easier.

Also, we need a process manager to restart our Express app automatically in case it crashes.

And we need a reverse proxy and load balancer to expose our app to the Internet, cache requests, and balance the load across multiple worker processes.

This lets us maintain high performance in our app.

Managing Environment Variables in Node.js with dotenv

The dotenv library is very useful for letting us read variables from an .env file

For example, we can write:

NODE_ENV=production
DEBUG=false

in our .env file.

And then we can read it with:

require('dotenv').config()
​
const express = require('express')
const app = express()

It’ll read the environment variables from a file named .env by default.

However, it can also read from multiple environment variable files.

Make Sure the Application Restarts Automatically with a Process Manager

If we have an Express app, it’ll crash when an unhandled error is encountered.

Therefore, it’s important that we restart our app automatically so that it won’t be down.

We can do that with Systemd by creating a new file in /lib/systemd/system called app.service :

[Unit]
Description=Node.js as a system service.
Documentation=https://example.com
After=network.target
[Service]
Type=simple
User=ubuntu
ExecStart=/usr/bin/node /my-app/server.js
Restart=on-failure
[Install]
WantedBy=multi-user.target

The Service section has the ExecStart line which runs our app on startup.

It also has the Restart line to restart on failure.

We can then reload the daemon start the script with:

systemctl daemon-reload
systemctl start fooapp
systemctl enable fooapp
systemctl status fooapp

PM2

PM2 is a process manager that lets us manage the Express app process.

We can install it by running:

npm i -g pm2

Then we can run our app by running:

pm2 start server.js -i max

-i max is the max number of threads to run our app with.

This way, it’ll spawn enough workers to use all CPU cores.

Load Balancing and Reverse Proxies

The Node cluster module lets us spawn worker processes that serve our app.

We can create a cluster by writing:

const cluster = require('cluster')
const numCPUs = require('os').cpus().length
const app = require('./src/app')
const port = process.env.PORT || 8888
​
const masterProcess = () => Array.from(Array(numCPUs)).map(cluster.fork)
const childProcess = () => app.listen(port)
​
if (cluster.isMaster) {
  masterProcess()
} else {
  childProcess()
}
​
cluster.on('exit', () => cluster.fork())

The master process creates the cluster and the child presses listen for requests at the given port.

The masterProcess counts how many CPU cores there are and calls cluser.fork to create child processes equal to the number of cores available.

The exit event listener will restart the process if it fails with cluster.fork .

Then in our Systemd file, we replace:

ExecStart=/usr/bin/node /my-app/server.js

with:

ExecStart=/usr/bin/node /my-app/cluster.js

And then we can restart Systemd with:

systemctl daemon-reload
systemctl restart fooapp

Conclusion

We can read environment variables with dotenv and add clusters with the cluster module.