Category: Express

Categories
Express JavaScript Nodejs

List of Express body-parser Errors

By default, Express 4.x or later doesn’t come with anything to parse request bodies. Therefore, we need to add a library to handle this.

body-parser is a middleware that we can use to parse various kinds of request bodies. It throws several kinds of errors that we need to manage. In this article, we’ll look at what errors it throws.

Errors

When body-parser throws an error, it sends a response code and an error message. The message property of the error has the error message.

The following errors are common errors that are thrown.

Content-Encoding Unsupported

This error is thrown when the request has a Content-Encoding header that contains an encoding but the inflation option was set to false.

The status of the response will be 415m and type property is set to encoding.unsupported. The charset property will be set to the encoding that’s unsupported.

For example, we can reproduce it if we have:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();  
const options = {  
  inflate: false,  
};
app.use(bodyParser.urlencoded(options));
app.post('/', (req, res) => {  
  res.send(req.body);  
});
app.listen(3000);

Then when we send a POST request to / with Content-Encoding set to abc and Content-Type set to application/x-www-form-urlencoded, we get:

content encoding unsupported

and a 415 status code.

Request Aborted

This error occurs when the request is aborted by the client before the reading of the body is finished.

The received property will be set to the number of bytes received before the request was aborted.

The status of the response will be 400 and type property set to request.aborted.

Request Entity Too Large

The error will occur when the request body is larger than what’s specified in the limit option.

A 413 error response will be sent with the type property set to entity.too.large.

For example, if we set limit to 0 as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();  
const options = {  
  inflate: false,  
  limit: 0  
};
app.use(bodyParser.json(options));
app.post('/', (req, res) => {  
  res.send(req.body);  
});
app.use((err, req, res, next) => {  
  res.status(400).send(err);  
});
app.listen(3000);

Then when we make a POST request to / with a JSON body that has content, then we get:

{"message":"request entity too large","expected":20,"length":20,"limit":0,"type":"entity.too.large"}

Request Size Didn’t Match Content-Length

The request’s length didn’t match the length from the Content-Length header.

This occurs when the request is malformed. Content-Length usually is calculated based on characters instead of bytes.

The status returned will be 400 and the type property is set to request.size.invalid.

For example, if we have:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();  
const options = {  
  inflate: false,    
};
app.use(bodyParser.urlencoded(options));
app.post('/', (req, res) => {  
  res.send(req.body);  
});
app.listen(3000);

and we send a POST request to / with the Content-Length header set to -1 and Content-Type set to application/x-www-form-urlencoded, we get a 400 response.

Stream Encoding Shouldn’t Be Set

req.setEncoding method is called prior to body-parser methods will cause this error.

We can’t call req.setEncoding directly when using body-parser.

The status will be 500 and the type property set to stream.encoding.set.

Too Many Parameters

The error occurs when the number of parameters in the request body exceeds what’s set in the parameterLimit.

The response status will be 413 and the type property set to parameters.too.many

Unsupported Charset “BOGUS”

This will occur when the request has a Content-Encoding header that contains an unsupported encoding. The encoding is contained in the message as well as in the encoding property.

The status will be set to 415. The type will be set to encoding.unsupported, and the encoding property is set to the encoding that’s unsupported.

Conclusion

There’re multiple kinds of errors raised by body-parser.

They involve sending bad headers or data that are not accepted by it, or canceling requests before all the data is read.

Various 400 series error status codes will be sent as the response along with the corresponding error messages and stack trace.

Categories
Express JavaScript

Guide to the Express Application Object — Middleware

The core part of an Express app is the Application object. It’s the application itself.

In this article, we’ll look at the methods of the app object and what we can do with it, including using app.use to set middleware.

app.use([path,] callback [, callback…])

We can use the app.use method to mount middleware that’s run app-wide.

They can be run when requests for specific paths are made or for all paths.

It takes the following arguments:

  • path — it can be a string or regex representing paths or patterns of paths. The default is / .
  • callback — a function to handle requests. It can be a middleware function, a series of them, array of them, or a combination of all of the above

We can provide multiple callbacks that behave just like middleware, but they call next('route') to skip the remaining middleware functions.

We can use these to impose preconditions on some routes and pass control to the route if we don’t need to call the remaining middleware.

Since router and app implement the middleware interface, we can use them like any other middleware function.

For example, we can use it as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));  
app.use((req, res, next) => {  
  console.log(`Request made at ${Date.now()}`)  
  next();  
})

app.get('/', (req, res) => {  
  res.send('hi');  
})

app.listen(3000);

Then we get the request time logged with every request.

We call next to call the route handler.

Middlewares are run sequentially, so the order they’re included in the code is important.

For example, if we have:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));  
app.use((req, res, next) => {  
  res.send('End with middleware');  
})

app.get('/', (req, res) => {  
  res.send('hi');  
})

app.listen(3000);

Then we get End with middleware instead of hi since we sent our response with the middleware, which we included first.

Error-Handling Middleware

We can create our own middleware for handling errors instead of using Express’s default error handler.

The error handler takes 4 arguments, which are the error object, request object, response object, and next function.

For instance, we can define one as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
app.get('/', (req, res, next) => {  
  try {  
    throw new Error('foo');  
  }  
  catch (ex) {  
    next(ex);  
  }  
})

app.use((err, req, res, next) => {  
  res.status(500).send('error');  
})

app.listen(3000);

The error should be displayed since we included the error handler after our GET request handler.

Paths

We can path in a string or regex path or path pattern as the first argument of the use method. This way, the middleware will only be called when the route matches the one specified in string or regex.

To match a constant path, we can write the following:

app.use('/abc', (err, req, res, next) => {  
  res.status(500).send('error');  
})

We can match patterns as follows. A ? will make the character preceding it optional:

app.use('/abc?d', (err, req, res, next) => {  
  res.status(500).send('error');  
})

The when requests made to /abcd or /abd has an error, the middleware above will be called.

The + sign means that one or more characters preceding it will be matched. For example, if we have:

app.use('/abc+d', (err, req, res, next) => {  
  res.status(500).send('error');  
})

The when requests made to /abccd,/abcccd, etc. has an error, the middleware above will be called.

* is a wildcard character. We can use it as follows:

app.use('/ab*cd', (err, req, res, next) => {  
  res.status(500).send('error');  
})

Then we get error when we make a request to /abcccd , /abFoocd , and so on if that has an error.

To match a group of optional characters, we can use parentheses and a question mark after it.

For instance, if we have:

app.use('/a(bcd)?e', (err, req, res, next) => {  
  res.status(500).send('error');  
})

Then we get error when we make a request to /abcde or/ae that has an error.

Passing in Middleware

We can pass in various combinations of middleware in addition to just one middleware.

We can pass in a router object as a middleware as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));const router = express.Router();  
router.get('/', (req, res, next) => {  
  res.send();  
});

app.use(router);app.listen(3000);

An Express app is also valid middleware:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
const subApp = express();
subApp.get('/', (req, res, next) => {  
  res.send();  
});

app.use(subApp);app.listen(3000);

We can also pass in a series of middleware as arguments:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));  
const r1 = express.Router();  
r1.get('/', (req, res, next) => {  
  console.log('r1 called');  
  next();  
});

const r2 = express.Router();  
r2.get('/', (req, res, next) => {  
  console.log('r2 called');  
  next();  
});  
app.use(r1, r2);
app.listen(3000);

Then they’ll be called in order, so we get:

r1 called  
r2 called

in the console.

Likewise, we can pass in an array of middlewares as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));  
const r1 = express.Router();  
r1.get('/', (req, res, next) => {  
  console.log('r1 called');  
  next();  
});

const r2 = express.Router();  
r2.get('/', (req, res, next) => {  
  console.log('r2 called');  
  next();  
});  
app.use([r1, r2]);
app.listen(3000);

Then we get the same output since they’re called in the same order that they’re listed.

A combination of them also works:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));  
const foo = (req, res, next) => {  
  console.log('foo called');  
  next();  
}

const r1 = express.Router();  
r1.get('/', (req, res, next) => {  
  console.log('r1 called');  
  next();  
});

const r2 = express.Router();  
r2.get('/', (req, res, next) => {  
  console.log('r2 called');  
  next();  
});

const subApp = express();  
subApp.get('/', (req, res, next) => {  
  console.log('subApp called');  
  next();  
})

app.use(foo, [r1, r2], subApp);app.listen(3000);

Then we get:

foo called  
r1 called  
r2 called  
subApp called

from the console. They’re still called in the order they’re listed.

Static Files

We can expose static folders with the express.static middleware. For example, if we want to expose the public folder in the project folder, we can write:

app.use(express.static(path.join(__dirname, 'public')));

Conclusion

app.use can be used to one more middleware in various combinations, including a single middleware, an array of middlewares, a comma-separated list of middlewares or a combination of them.

They’re called in the order that they’re included, and the next one can be called by calling next .

Error handlers are also middleware functions. The only difference is that they have an error parameter before the request and response parameters.

Finally, we can expose static folders with the express.static middleware.

Categories
Express JavaScript Nodejs

Guide to the Express Router Object — Request and Parameter Handling

The Expressrouter object is a collection of middlewares and routes. It a mini-app within the main app.

It can only perform middleware and routing functions and can’t stand on its own.

It also behaves like middleware itself, so we can use it with app.use or as an argument to another route’s use method.

In this article, we’ll look at the router object’s methods, including all , param , and methods for listening to specific kinds of requests.

Methods

router.all(path, [callback, …] callback)

The router.all method takes a callback for handling all kinds of requests.

We can pass in a constant path, or a string with the path pattern or a regex.

For example, we can pass in middleware that’s run for all routes attached to the router as follows:

const express = require('express');  
const bodyParser = require('body-parser');
const app = express();  
const fooRouter = express.Router();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
const mw1 = (req, res, next) => {  
  console.log('middleware 1 called');  
  next();  
}
const mw2 = (req, res, next) => {  
  console.log('middleware 2 called');  
  next();  
}
fooRouter.all('*', mw1, mw2);
fooRouter.get('/', (req, res) => {  
  res.send('foo');  
})
app.use('/foo', fooRouter);app.listen(3000, () => console.log('server started'));

Then we get:

middleware 1 called  
middleware 2 called

if we make a request to /foo since anything that starts with /foo routes through the fooRouter , and we have the fooRouter.all method call with the middlewares passed in.

Equivalently, we can write:

const express = require('express');  
const bodyParser = require('body-parser');
const app = express();  
const fooRouter = express.Router();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
const mw1 = (req, res, next) => {  
  console.log('middleware 1 called');  
  next();  
}
const mw2 = (req, res, next) => {  
  console.log('middleware 2 called');  
  next();  
}
fooRouter.all('*', mw1);  
fooRouter.all('*', mw2);
fooRouter.get('/', (req, res) => {  
  res.send('foo');  
})
app.use('/foo', fooRouter);
app.listen(3000, () => console.log('server started'));

They’re the same as long as the order of fooRouter.all is called in the same as the order the callbacks are passed in.

router.METHOD(path, [callback, …] callback)

router.METHOD is for handling requests with the given method. For example, router.get for handling GET requests, router.post for handling POST requests, etc.

router.get also automatically calls for the HTTP HEAD in addition to the GET method if router.head wasn’t called.

We can provide multiple callbacks and they’re all treated equally. These callbacks may invoke the next('route') call to bypass the remaining route callbacks.

For example, we can use it as follows:

const express = require('express');  
const bodyParser = require('body-parser');
const app = express();  
const fooRouter = express.Router();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
fooRouter.get('/', (req, res) => {  
  res.send('foo');  
})
app.use('/foo', fooRouter);
app.listen(3000, () => console.log('server started'));

Then when we make a request to /foo , we get back foo .

We can also pass in a regex for the route path . For example, we can write:

const express = require('express');  
const bodyParser = require('body-parser');
const app = express();  
const fooRouter = express.Router();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
fooRouter.get('/ab+c/', (req, res) => {  
  res.send('foo');  
})
app.use('/foo', fooRouter);
app.listen(3000, () => console.log('server started'));

to listen to requests for paths /foo/abc , /foo/abbc , /foo/abbbc , etc., since we specified in the regex that we look for any number of the character b in the path.

router.param(name, callback)

router.param lets us trigger a callback function call when a specific parameter is passed in when the request is made from the client.

name is the parameter placeholder name that we look for.

The parameters of the callback function are:

  • req, the request object.
  • res, the response object.
  • next, indicating the next middleware function.
  • The value of the name parameter.
  • The name of the parameter.

For example, we can use it as follows:

const express = require('express');  
const bodyParser = require('body-parser');
const app = express();  
const fooRouter = express.Router();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
fooRouter.param('name', (req, res, next, name) => {  
  req.name = name;  
  next();  
})
fooRouter.get('/:name', (req, res) => {  
  res.send(req.name);  
})
app.use('/foo', fooRouter);
app.listen(3000, () => console.log('server started'));

Then we make a request to /foo/abc then we get abc since the fooRouter.param found the name parameter was passed in with the URL.

The name parameter has the value 'abc' because it grabbed the part after /foo/ and then we assigned name to req.name and called next .

After that, the route handler we passed into foorRouter.get is called, then we passed req.name into res.send and sent it as the response.

Conclusion

The Express router lets us create sub-apps of an Express app so we don’t have to add all the route handlers and middlewares to the main app.

With the all method, we can listen to all kinds requests. We can also listen to specific kinds of requests like GET or POST requests with the respective methods. They all take a string or regex path and a route handler callback.

Finally, we have the param method to get route parameters and do what we want with it.

Categories
Express JavaScript Nodejs

Use body-parser Express Middleware to Parse Text and URL-Encoded Requests

By default, Express 4.x or later doesn’t come with anything to parse request bodies. Therefore, we need to add something to do this.

In this article, we’ll look at how to use the body-parser middleware to do this with text and URL-encoded request bodies.

Parsing Text Bodies

We can parse text request bodies with the text method. It supports automatic inflation of gzip and deflate encodings.

The parsed string will be set as the value of req.body.

It takes an optional option object with the following properties:

  • defaultCharset — specifies the default character set for the text content if it’s not specified in the Content-Type header. Defaults to utf-8.
  • inflate — compressed request bodies will be inflated when this is set to true . Otherwise, they’ll be rejected.
  • limit — controls the maximum request body size. If it’s number, then it’s measured in bytes. If it’s a string then it can be parsed into a number of bytes.
  • type — this is used to determine what media type it’ll parse. It can be a string, array of strings or a function. If it’s not a function, then it’s directly passed into the type-is library. Otherwise, the request is parsed if the data type the function is called with returns a truthy value
  • verify — this is a function with signature (req, res, buf, encoding), where buf is a Buffer object of the raw request body. The parsing can be aborted by throwing an error in the function.

For example, we can use it as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();  
const options = {  
  inflate: true,  
  limit: 1000,  
  defaultCharset: 'utf-8'  
};  
app.use(bodyParser.text(options));
app.post('/', (req, res) => {  
  res.send(req.body);  
});
app.listen(3000);

Then when we make a POST request to / with the body foo, we get foo back.

Parsing URL Encoded Request Bodies

We can use the urlencoded method to pares URL encoded bodies. It supports automatic inflation of gzip and deflate encodings.

The parsed request body will be set as the value of req.body. The object will contain key-value pairs, where the value can be a string or an array when extended is set to false or anything otherwise.

It takes an optional option object with the following properties:

  • extended — the extended option allows us to choose between parsing the URL-encoded data with the querystring library when false or the qs library when this is set to true . The extended syntax lets us encode rich objects and arrays, allowing for a JSON-like experience with URL-encoded. The default value is true, but using the default value has been deprecated.
  • inflate — compressed request bodies will be inflated when this is set to true. Otherwise, they’ll be rejected.
  • limit — controls the maximum request body size. If it’s number, then it’s measured in bytes. If it’s a string then it can be parsed into a number of bytes.
  • parameterLimit — lets us control the maximum number that are allowed in the URL encoded data. If it’s more than the given value then a 413 response code will be returned. The default is 1000.
  • type — this is used to determine what media type it’ll parse. It can be a string, array of strings or a function. If it’s not a function, then it’s directly passed into the type-is library. Otherwise, the request is parsed if the data type the function is called with returns a truthy value
  • verify — this is a function with signature (req, res, buf, encoding), where buf is a Buffer object of the raw request body. The parsing can be aborted by throwing an error in the function.

For example, we can use it as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();  
const options = {  
  inflate: true,  
  limit: 1000,  
  extended: true  
};  
app.use(bodyParser.urlencoded(options));
app.post('/', (req, res) => {  
  res.send(req.body);  
});
app.listen(3000);

Then when we send a URL encoded POST body to the / route with the body name=Mary&age=10, then we get:

{"name":"Mary","age":"10"}

We can send an array by sending:

name=Mary&age=10&favoriteFood=apple&favoriteFood=orange

Then we get back:

{"name":"Mary","age":"10","favoriteFood":["apple","orange"]}

as the response. This assumes that extends is true.

If we set the parameterLimit to 1 as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();  
const options = {  
  inflate: true,  
  limit: 1000,  
  extended: true,  
  parameterLimit: 1,  
};  
app.use(bodyParser.urlencoded(options));
app.post('/', (req, res) => {  
  res.send(req.body);  
});
app.listen(3000);

then we get a 413 error back.

Conclusion

With the text method, body-parser can parse text request bodies. We’ll get a string with the parsed body with req.body.

To parse URL-encoded request bodies, we can use the urlencoded method. It can parse arrays and objects. URL-encoded bodies are sent as query strings, and we can send a query with the same key multiple times to make it parsed as an array.

Categories
Express JavaScript Nodejs

Security Best Practices for Using Express Apps in Production

When we use an Express app for production, i.e. it’s used by external users, we have to be careful about security since it’s available to the outside world.

In this article, we’ll look at some security best practices when running Express apps in a production environment.

Don’t Use Outdated Versions of Express

Outdated versions of Express are no longer maintained and may have unpatched security vulnerabilities, leaving our apps at risk for all sorts of attacks.

Use TLS

If we’re transporting data that needs to be secured, we should use TLS so that users’ data is protected. This means that we avoid man-in-the-middle attacks and packet sniffing.

Even our apps don’t transport secure data, it still gives users more confidence that our website is secure.

We can get free TLS certificates with Let’s Encrypt. It’s an automated service that generates new certificates that are trusted by browsers,

Use Helmet

Helmet is a series of Express middlewares to protect our apps with some well-known security vulnerabilities on the web by setting HTTP headers.

The middlewares included with Helment includes:

  • csp — sets the Content-Security-Policy header to help prevent cross-site scripting attacks and other cross-site injections.
  • hidePoweredBy — removes the X-Powered-By header
  • hpkp — adds public key pinning header to prevent man-in-the-middle attacks with forged certificates
  • hsts — sets Strict-Transport-Security header that enforces HTTPS connects to the server.
  • ieNoOpen 0 sets X-Download-Options for IE 8 or later
  • noCache — sets Cache-Control and Pragma header to disable client-side caching
  • noSniff — sets X-Content-Type-Options header to prevent browsers from MIME-sniffing a response away from the declared content type
  • frameguard — sets the X-Frame-Options header to provide clickjacking protection
  • xssFilter — sets X-XSS-Protection to enable cross-site scripting filter in most recent web browsers

It’s a good idea to at least disable the x-powered-by response header so that attackers won’t know that our app is an Express app and launch specific attacks for it.

Use Cookies Securely

We can use the express-session or cookie-session middleware to use cookies securely.

express-session stores session data on the server. It only saves the session ID on the cookie itself.

We can set up a session store for production use since it uses an in-memory store by default.

cookie-session stores the whole cookie on the client-side. We shouldn’t exceed 4093 bytes when we set our cookies.

The cookie data will be visible to the client, so secret data shouldn’t be sent to the client with cookie-session .

Don’t Use Default Session Cookie Name

The default session cookie name set by Express shouldn’t be used since it can identify that an app is running on Express.

We can do this by passing in an object to session as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const session = require('express-session');  
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
app.set('trust proxy', 1);  
app.use(session({  
  secret: 'secret',  
  name: 'sessionId'  
}))
app.listen(3000);

Set Cookie Security Options

To enhance security, we can set various cookie options. The following options are available:

  • secure — ensures browsers only send cookie over HTTPS
  • httpOnly — ensures cookie is sent only over HTTP(S) and not client-side JavaScript. This helps us prevent XSS attacks
  • domain — indicates the domain of the cookie, which can be used to compare against the domain of the server in the URL that’s requesting the cookie.
  • path — indicates the path to compare against the requested path, then send the cookie in the request
  • expires — set the expiration date for persistent cookies.

For example, we can use it as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const session = require('express-session');  
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));  
const expiryDate = new Date(Date.now() + 60 * 60 * 1000)  
app.use(session({  
  name: 'session',  
  secret: 'secret',  
  keys: ['foo', 'bar'],  
  cookie: {  
    secure: true,  
    httpOnly: true,  
    domain: 'EnormousBeneficialScript--five-nine.repl.co',  
    path: '/',  
    expires: expiryDate,  
  }  
}));
app.get('/', (req, res) => {  
  req.session.foo = 'foo';  
  res.send(req.session.foo);  
})
app.listen(3000);

Prevent Brute-Force Attacks

We can rate limit our authorization routes so that attackers can’t use brute-force attacks to guess the credentials stored in our app’s database.

For example, we can limit by IP address by limiting the number of failed attempts that people can make if they have the same user name and IP address.

Then we lock out the user for some time if the number of failed attempts what we can tolerate.

If more than 100 failed attempts in a day are made then we block the IP address.

We can use the rate-limiter-flexible package to do rate-limiting of our routes.

Conclusion

To keep our Express app secure in a production environment, we should take a few precautions.

First, we should use TLS for transporting data to keep them from being sniffed and prevent man-in-the-middle attacks.

Next, we should set headers in ways that prevent users from finding information about our app and to enforce secure communication.

Also, we should use cookies securely by signing it with a secret and set the security options for them.

Finally, to prevent brute-force attacks, we should set a rate limit for our API calls so that attackers can’t guess our users’ credentials by repeated login attempts.