Category: Nodejs

Like any kind of apps, JavaScript apps also have to be written well.

Otherwise, we run into all kinds of issues later on.

In this article, we’ll look at some best practices we should follow when writing Node apps.

Handle Errors Centrally

We should handle errors centrally in our Node app.

For example, we can call next in one middleware to call the error handling middleware that’s added after it.

We can write:

try {
  User.addNew(req.body).then((result) => {
    res.status(200).json(result);
  }).catch((error) => {
    next(error)
  });
} catch (error) {
  next(error);
}

to catch the error in the promise chain with the catch method.

And with the try-catch block, we catch errors in the synchronous code.

Then we can add an error handler middleware after that code by writing:

app.use((err, req, res, next) => {
  errorHandler.handleError(err).then((isOperationalError) => {
    if (!isOperationalError)
      next(err);
  });
});

We invoke another promise chain to handle any operational errors.

These are errors that we can anticipate and handle gracefully.

Document API Errors Using Swagger

API errors should be documented in some way so they can be handled.

We don’t want our app to crash.

Swagger lets us make document our APIs in an automated way.

If we don’t handle errors, then the app will crash.

Shut the Process Gracefully When Unknown Errors are Encountered

If our app encounters some errors that aren’t handled, then we should shut our app down gracefully and restart it.

A common way to do this is to use a process manager like PM2 to do this.

We don’t want any error to crash our app and bring it down.

If we don’t have a process manager, then we’ll have to restart it ourselves every time.

This is definitely not acceptable since it can happen many times a day at any time.

We can create our own error handler by writing:

function errorHandler() {
  this.handleError = function(error) {
    return logger.logError(err));
  }

  this.isTrustedError = function(error) {
    return error.isOperational;
  }
}

We have an error handler constructor with some methods to handle errors.

Use a Mature Logger

If a Node app it’s in production, then the errors won’t be visible unless we log them somewhere.

There’re many logging solutions available for Node apps, including Winston, Bunyan, and Log4js.

They’ll help us find errors faster.

We shouldn’t use crude solutions like console.log for logging errors in production.

The errors should be available in some cloud services or files that we can search.

For instance, we can use Winston by writing:

const logger = new winston.Logger({
  level: 'info',
  transports: [
    new (winston.transports.Console)(),
    new (winston.transports.File)({ filename: 'somefile.log' })
  ]
});

logger.log('info', 'log: %s', 'something', { anything: 'This is metadata' });

We use the winston.Logger constructor to create the logger.

Then we call log to log what we want in the format we want.

Discover Errors and Downtime Using APM Products

Errors and downtime should be recorded with APM products.

This way, we can find them easily all in one place.

They provide features like website or API monitoring.

Each request is recorded with the performance metrics of them.

If there are slow code, it’ll record where the slowness occurred.

Also, they’ll aggregate the data and then show them together in one dashboared.

Conclusion

Errors should be handled centrally.

Also, logging and monitoring should be done with various libraries and services.

Like any kind of apps, JavaScript apps also have to be written well.

Otherwise, we run into all kinds of issues later on.

In this article, we’ll look at some best practices we should follow when writing Node apps.

Use JWT-Based, Stateless Authentication

We can authenticate by using JSON web tokens.

A JSON web token consists of 3 parts.

They include:

• header with the type of the token and hashing algorithm
• payload has the claims
• a signature that signs the payload.

We can add it easily with some add-ons.

For instance, we can use the koa-jwt package to add the token.

We can write:

const koa = require('koa')
const jwt = require('koa-jwt')

const app = koa()

app.use(jwt({
  secret: 'secret'
}))

// Protected middleware
app.use(function *(){
  this.body = {
    foo: 'bar'
  }
})

We just call app.use to use the jwt middleware.

The object has the secret to sign the token.

Then we added a protected middleware after that.

The token content will be available with this.state.user .

The JWT module doesn’t depend on any database layer.

They’re all verified on their own.

They can also contain the time to live values.

To ensure that our communication is secure, we still have to ensure that API endpoints are available through an HTTPS connection.

Use Conditional Requests

Conditional requests are HTTP requests that have different results depending on the request header values.

The headers check whether a version of a resource stored on the server match a given version of the same resource.

The headers can be the timestamp since the last modification.

It can also be an entity tag, which differs for each version.

The headers are:

• Last-Modified which indicates when the resource was last modified
• Etag which is the entity tag
• If-Modified-Since which is used with the Last-Modifed header
• If-None-Match used with the Etag header.

Use Rate Limiting

We can limit the number of requests that can be made to our API.

To tell our API clients how many requests are left, we can set the following response headers:

• X-Rate-Limit-Limit, tells us the number of requests allows in a given time interval
• X-Rate-Limit-Remaining, the number of requests remaining in the same interval
• X-Rate-Limit-Reset , tells us the time when the rate limit will be reset

We can add libraries to add rate limit capabilities to our app.

With Koa, we can use the koa-ratelimit package.

Create a Proper API Documentation

It’s hard to know how to use our API without any documentation.

To make our lives easier, we can use API Blueprint or Swagger to create our documentation.

Future of APIs

There’re alternatives to REST.

We can use GraphQL to listen to HTTP requests, but it has type checking and we can selectively select resources.

With type checking and the ability to selectively query resources, we can be more efficient and reduce the chance of errors.

Conclusion

We can use JWT for authentication.

Conditional requests let us make requests differently according to headers.

Alternatives to REST APIs should also be considered.

Like any kind of apps, JavaScript apps also have to be written well.

Otherwise, we run into all kinds of issues later on.

In this article, we’ll look at some best practices we should follow when writing Node apps.

Structure Express Applications

We should have the following folder structure for our Express app:

src/
  config/
  controllers/
  providers/
  services/
  models/
  routes.js
  db.js
  app.js
test/
  unit/
  integration/
server.js
(cluster.js)
test.js

We have the src folder with the code that runs in production.

config has config.

controllers have the controllers.

providers have the logic for the controller routes.

services have the business logic.

models have the database models.

routes.js loads all routes.

db.js has the database routes.

app.js loads the Express app.

test has the tests.

unit has the unit tests.

integration has the integration tests.

server.js in the entry point of the Express app.

cluster.js is an optional file for creating clusters.

test.js is the main test file to run all the tests in the test directory.

Improve Express.js Performance and Reliability

There are a few ways to improve performance and reliability.

Set NODE_ENV to production

We should set the NODE_ENV environment variable to production so we get the benefits of the production config.

It’s 3 times than in dev.

This is because there’s compression and caching to make our app faster.

We can either run:

export NODE_ENV=production

to set the environment variable

or:

NODE_ENV=production node server.js

to set the environment variable and run the app at the same time.

Enable Gzip Compression

We can enable gzip compression for assets to do compression on our assets.

We can install the compression middleware by running:

npm i compression

Then we can use it by writing:

const compression = require('compression')
const express = require('express')
const app = express()
app.use(compression())

This isn’t the best way to do gzip compression since it uses resources on the Express app.

Instead, we can enable gzip in the Nginx instead to offload the work to the reverse proxy.

Always Use Asynchronous Functions

If we have anything other than some simple operations, we should probably use async code.

We should promises most of the time or async/await for short.

For example, we can write:

(async () => {
  const foo = () => {
    //...
    return val
  }
​
  const val = await asyncFunction;
})()

We have an asyncFunction that returns a promise, so we use await to get the result.

We can’t run synchronous functions on different threads since Node is single-threaded.

Therefore, we can only use async code to run long-running operations.

We can also spawn child processes or create multiple instances of our app to run different tasks on different processes.

Logging Correctly

We should collect our logs in a central location.

This way, we can find them when we need them.

Winston and Morgan and useful logging packages that can integrate with other services for centralized logging.

We can also use some service like Sematext to do logging.

For example, we can write:

const { stLogger, stHttpLoggerMiddleware } = require('sematext-agent-express')
​
const express = require('express')
const app = express()
app.use(stHttpLoggerMiddleware)
​

app.get('/api', (req, res, next) => {
  stLogger.info('An info.')
  stLogger.debug('A debug.')
  stLogger.warn('A warning.')
  stLogger.error('An error.')

  res.send('Hello World.')
})

We have the the sematext-agent-express package which has a logger than logs to the Sematext service.

Then we get the logs in a nice dashboard with the service.

Conclusion

We can better structure our Express app and run our production Express app in production mode.

Also, logging can be easy with a centralized logging service.

Like any kind of apps, JavaScript apps also have to be written well.

Otherwise, we run into all kinds of issues later on.

In this article, we’ll look at some best practices we should follow when writing Node apps.

Use Promises for Async Error Handling

We should use promises for async error handling.

Handling async errors in the callback style would be a mess if we have many nested callbacks.

Node style callbacks don’t allow us to chain the async calls without nesting.

To avoid this, we should make sure that we use promises.

For example, instead of writing:

getData(someParameter, function(err, result){
    if(err !== null && err !== undefined)
    getMoreData(a, function(err, result){
          if(err !== null && err !== undefined)
        getMoreData(b, function(c){
                getMoreData(d, function(e){
                    //...
                });
            });
        });
    });
});

We have 4 nested callbacks, which is definitely hard to read even if if we omit the logic code.

Instead, we write:

doWork()
  .then(doWork)
  .then(doMoreWork)
  .then(doWork)
  .catch(errorHandler)
  .then(verify);

We can make this a long cleaner since promises has a then method which we can pass callbacks into.

If we want to catch errors, we use the catch method.

We can also use the async and await syntax to chang promises and use try-catch to catch errors.

For instance, we can write:

const work = async () =>
  try {
    const r1 = await doWork();
    const r2 = await doMoreWork();
    const r3 = await doWork();
    const r4 = await verify();
  }
  catch (e) {
    `errorHandler(e)
  }
}`

Each then callback returns a promise so we can use await on them.

Use Only the Built-in Error Object

The built-in error constructor should be used to create errors.

The Error constructor has the error message and stack trace.

They’re useful for troubleshooting issues which will be lost if we throw anything else

For example, instead of writing:

throw 'error';

We write:

throw new Error("error");

It’ll have the stack trace up to where the error is thrown.

Distinguish Operational vs Programmer Errors

We should distinguish operational vs programmer errors.

Operational errors are errors where the impact of the error is fully understood and can be handled.

Programmer errors refer to unknown code failures where we need to gracefully restart the application.

For the errors we can handle, we should be able to handle them and avoid restating our app.

We can throw errors, by writing:

if(!product) {
  throw new Error("no product selected");
}

const myEmitter = new MyEmitter();
myEmitter.emit('error', new Error('error'));

We can throw errors in synchronous code and event emitters.

We can throw an error in a promise with:

new Promise((resolve, reject) => {
  DAL.getProduct(productToAdd)
    .then((product) =>{
       if(!product)
         return reject(new Error("no product added"));
       }
    })
});

We have a promise that calls reject with an Error instance to throw the error.

These are the right ways to throw errors.

Other ways that are around include listening to the uncaughtException event:

process.on('uncaughtException', (error) => {
  if (!error.isOperational)
    process.exit(1);
});

Listening to the uncaughtException event changes the behavior of the event so it shouldn’t be listened to.

process.exit is also a bad way to end a program because it abruptly ends the program.

Conclusion

We should throw and catch errors in the right way.

This way our app will handle them gracefully.

Like any kind of apps, JavaScript apps also have to be written well.

Otherwise, we run into all kinds of issues later on.

In this article, we’ll look at some best practices we should follow when writing Node apps.

Handle Errors and Exceptions Properly

We should handle errors and exceptions properly on our Express app.

In synchronous code or an async function, we can use try-catch:

async function foo() {
  try {
    const baz = await bar()
    return baz
  } catch (err) {
    console.error(err);
  }
}

If we want to catch errors in middleware, we can create our own middleware to catch errors.

For example, we can write:

function errorHandler(err, req, res, next) {
  console.error(err)
  res.status(err.status || 500).send(err.message)
}

router.use(errorHandler)

We have the errorHandler middleware to catch errors from other middleware that are added before it.

Then we can call router.use or app.use with the errorHandler to handle the error.

Watch Out For Memory Leaks

We should watch for memory leaks so that our app doesn’t run out of memory.

Increasing memory usage is bad if it happens continuously.

This means that the app keeps using memory.

There’re apps like Sematext Agent Express module which lets us watch the CPU and memory usage of our app.

With this package, we can integrate it with:

const { stMonitor, stLogger, stHttpLoggerMiddleware } =
require('sematext-agent-express')
stMonitor.start()

const express = require('express')
const app = express()
app.use(stHttpLoggerMiddleware)

We just call stMonitor.start() to start monitoring with Sematext when our app starts.

Other tools like Scout also shows us the memory usage of each line of code.

We can see which ones are using more memory and the times that uses more memory.

These can also monitor each request for performance issues.

JavaScript

We can improve our Express app’s JavaScript code to make it easier to test and maintain.

Pure Functions

Pure functions are functions that let returns something and don’t change any outer state.

If we pass in the same parameters to them, then they’ll always return the same value.

This makes their behavior predictable and simplifies everyone’s lives.

We can create new objects instead of mutating existing objects with pie functions.

Some examples of pure functions in JavaScript include the array instance’s map and filter methods and many more.

Object Parameters

To make working with parameters, we should reduce the number of parameters in our function.

One easy way to do it is to add an object parameter to our function.

Then we can use destructuring to destructure the properties into variables.

This way, we won’t have to worry about the order of the parameters.

For example, we can write:

const foo = ({ a, b, c }) => {
  const sum = a + b + c;
  return sum;
}

We just take the a , b , and c parameters and use them as variables.

Write Tests

Tests are great for catching regressions.

This way, if we change our code, we can have peace of mind that we didn’t kill any existing code if the existing tests pass.

With JavaScript, there are many test frameworks, including Mocha, Chai, Jasmine, and Jest.

We can use any of them to run test.

With Chai, we can write:

const chai = require('chai')
const expect = chai.expect

const foo = require('./src/foo')

describe('foo', function () {
  it('should be a function', function () {
    expect(foo).to.be.a('function')
  })
})

to import the foo file and run tests on it with Chai.

Conclusion

We should handle exceptions properly.

Also, we can use tests to prevent errors.

Pure functions and object parameters also help us write cleaner code.