Categories
Express Nodejs Testing Testing

Add Tests to Express Apps With Jest and SuperTest

Automated tests are essential to the apps we write since modern apps have so many moving parts.

In this piece, we’ll look at how to write apps to test an Express app that interacts with a database with Jest and SuperTest.


Creating the App We’ll Test

We create a project folder by creating an empty folder and running the following to create a package.json file with the default answers:

npm init -y

Then we run the following to install the packages for our apps:

npm i express sqlite3 body-parser

Then, we create the app.js file for our app and write:

const express = require('express');
const sqlite3 = require('sqlite3').verbose();
const bodyParser = require('body-parser');
const app = express();
const port = process.env.NODE_ENV === 'test' ? 3001 : 3000;
let db;
if (process.env.NODE_ENV === 'test') {
    db = new sqlite3.Database(':memory:');
}
else {
    db = new sqlite3.Database('db.sqlite');
}

db.serialize(() => {
    db.run('CREATE TABLE IF NOT EXISTS persons (id INTEGER PRIMARY KEY, name TEXT, age INTEGER)');
});

app.use(bodyParser.json());
app.get('/', (req, res) => {
    db.serialize(() => {
        db.all('SELECT * FROM persons', [], (err, rows) => {
            res.json(rows);
        });
    })
})

app.post('/', (req, res) => {
    const { name, age } = req.body;
    db.serialize(() => {
        const stmt = db.prepare('INSERT INTO persons (name, age) VALUES (?, ?)');
        stmt.run(name, age);
        stmt.finalize();
        res.json(req.body);
    })
})

app.put('/:id', (req, res) => {
    const { name, age } = req.body;
    const { id } = req.params;
    db.serialize(() => {
        const stmt = db.prepare('UPDATE persons SET name = ?, age = ? WHERE id = ?');
        stmt.run(name, age, id);
        stmt.finalize();
        res.json(req.body);
    })
})

app.delete('/:id', (req, res) => {
    const { id } = req.params;
    db.serialize(() => {
        const stmt = db.prepare('DELETE FROM persons WHERE id = ?');
        stmt.run(id);
        stmt.finalize();
        res.json(req.body);
    })
})

const server = app.listen(port);
module.exports = { app, server };

The code above has the app we’ll test.

To make our app easier to test, we have:

const port = process.env.NODE_ENV === 'test' ? 3001 : 3000;
let db;
if (process.env.NODE_ENV === 'test') {
    db = new sqlite3.Database(':memory:');
}
else {
    db = new sqlite3.Database('db.sqlite');
}

So we can set the process.env.NODE_ENV to 'test' to make our app listen to a different port than it does when the app is running in a nontest environment.

We’ll use the 'test' environment to run our tests.

Likewise, we want our app to use a different database when running unit tests than when we aren’t running them.

This is why we have:

let db;
if (process.env.NODE_ENV === 'test') {
    db = new sqlite3.Database(':memory:');
}
else {
    db = new sqlite3.Database('db.sqlite');
}

We specified that when the app is running in a 'test' environment we want to use SQLite’s in-memory database rather than a database file.


Writing the Tests

Initialization the code

With the app made to be testable, we can add tests to it.

We’ll use the Jest test runner and SuperTest to make requests to our routes in our tests. To add Jest and SuperTest, we run:

npm i jest supertest

Then, we add app.test.js to the same folder as the app.js file we had above.

In app.test.js, we start by writing the following:

const { app } = require('./app');
const sqlite3 = require('sqlite3').verbose();
const request = require('supertest');
const db = new sqlite3.Database(':memory:');

beforeAll(() => {
    process.env.NODE_ENV = 'test';
})

In the code above, we included our Express app from our app.js. Then we also included the SQLite3 and SuperTest packages.,

Then, we connected to our in-memory database with:

const db = new sqlite3.Database(':memory:');

Next, we set all the tests to run in the 'test' environment by running:

beforeAll(() => {
    process.env.NODE_ENV = 'test';
})

This will make sure we use port 3001 and the in-memory database we specified in app.js for each test.

To make our tests run independently and with consistent results, we have to clean our database and insert fresh data every time.

To do this, we create a function we call on each test:

const seedDb = db => {
    db.run('CREATE TABLE IF NOT EXISTS persons (id INTEGER PRIMARY KEY, name TEXT, age INTEGER)');
    db.run('DELETE FROM persons');
    const stmt = db.prepare('INSERT INTO persons (name, age) VALUES (?, ?)');
    stmt.run('Jane', 1);
    stmt.finalize();
}

The code above creates the persons table if it doesn’t exist and deletes everything from there afterward.

Then we insert a new value in there to have some starting data.


Adding Tests

With the initialization code complete, we can write the tests.

GET request test

First, we write a test to get the existing seed data from the database with a GET request.

We do this by writing:

test('get persons', () => {
    db.serialize(async () => {
        seedDb(db);
        const res = await request(app).get('/');
        const response = [
            { name: 'Jane', id: 1, age: 1 }
        ]
        expect(res.status).toBe(200);
        expect(res.body).toEqual(response);
    })
});

We put everything inside the callback of db.serialize so the queries will be run sequentially.

First, we call seedDb, which we created above to create the table if it doesn’t exist, to clear out the database, and to add new data.

Then, we call the GET request by writing:

await request(app).get('/');

This gets us the res object with the response resolved from the promise.

request(app) will start the Express app so we can make the request.

Next, we have the response for us to check against for correctness:

const response = [
  { name: 'Jane', id: 1, age: 1 }
]

Then, we check the responses to see if we get what we expect:

expect(res.status).toBe(200);
expect(res.body).toEqual(response);

The toBe method checks for shallow equality, and toEqual checks for deep equality. So we use toEqual to check if the whole object structure is the same.

res.status checks the status code returned from the server, and res.body has the response body.

POST request test

Next, we add a test for the POST request. It’s similar to the GET request test.

We write the following code:

test('add person', () => {
    db.serialize(async () => {
        seedDb(db);
        await request(app)
            .post('/')
            .send({ name: 'Joe', age: 2 });

        const res = await request(app).get('/');
        const response = [
            { name: 'Jane', id: 1, age: 1 },
            { name: 'Joe', id: 2, age: 2 }
        ]
        expect(res.status).toBe(200);
        expect(res.body).toEqual(response);
    })
});

First, we reset the database with:

seedDb(db);

We made our POST request with:

await request(app)
  .post('/')
  .send({ name: 'Joe', age: 2 });

This will insert a new entry into the in-memory database.

Finally, to check for correctness, we make the GET request — like in our first test — and check if both entries are returned:

const res = await request(app).get('/');
const response = [
  { name: 'Jane', id: 1, age: 1 },
  { name: 'Joe', id: 2, age: 2 }
]
expect(res.status).toBe(200);
expect(res.body).toEqual(response);

PUT and DELETE tests

The test for the PUT request is similar to the POST request. We reset the database, make the PUT request with our payload, and then make the GET request to get the returned data, as follows:

test('update person', () => {
    db.serialize(async () => {
        seedDb(db);
        await request(app)
            .put('/1')
            .send({ name: 'Joe', age: 2 });

        const res = await request(app).get('/');
        const response = [
            { name: 'Jane', id: 1, age: 1 }
        ]
        expect(res.status).toBe(200);
        expect(res.body).toEqual(response);
    })
});

Then we can replace the PUT request with the DELETE request and test the DELETE request:

test('delete person', () => {
    db.serialize(async () => {
        seedDb(db);
        const res = await request(app).delete('/1');
        const response = [];
        expect(res.status).toBe(200);
        expect(res.body).toEqual(response);
    })
});

Running the Tests

To run the tests, we add the following to the scripts section:

"test": "jest --forceExit"

We have to add the --forceExit option so Jest will exist after the tests are run. There’s no fix for the issue where Jest tests using SuperTest don’t exit properly yet.

Then we run the following to run the tests:

npm test

And we should get:

PASS  ./app.test.js
  √ get persons (11ms)
  √ add person (2ms)
  √ update person (2ms)
  √ delete person (6ms)

Test Suites: 1 passed, 1 total
Tests:       4 passed, 4 total
Snapshots:   0 total
Time:        2.559s
Ran all test suites.
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?

We should get the same thing no matter how many times we run the tests since we reset the database and made all database queries run sequentially.

Also, we used a different database and port for our tests than other environments, so the data should be clean.


Conclusion

We can add tests run with the Jest test runner. To do this, we have to have a different port and database for running the tests. Then we create the tables if they don’t already exist, clear all the data, and add seed data so we have the same database structure and content for every test.

With SuperTest, we can run the Express app automatically and make the request we want. Then, we can check the output.

Categories
Express JavaScript Mega-Guides Nodejs

Express Basics Mega-Guide

We can create back end apps running on the Node.js platform with Express.

In this article, we’ll look at how to create basic routes with Express.

Basic Routing

Routing is where an Express application response to a client request from a URL or path and a specific HTTP request method, like GET or POST.

Each route in Express can have one more handler function, which are executed when the route is matched.

The general definition of a route takes the following format:

app.METHOD(PATH, HANDLER);

app is the Express app instance.

The METHOD is an HTTP request method in lowercase. Possible methods include GET, POST, PUT, and DELETE.

PATH is the path for the route. HANDLER is the handler function that’s run when the route is matched.

For example, we can write:

app.get('/', (req, res) => {  
  res.send('Hello World!')  
})

To display 'Hello World!' on the screen.

If we want our app to accept a POST request, we can use app.post as follows:

app.post('/', (req, res) => {  
  res.send('Received POST request');  
})

We can test this with an HTTP client like Postman by send a POST request to the URL that our app is running on. Then we should get:

Received POST request

In the response body.

Likewise, we can do the same for PUT and DELETE requests as follows:

app.put('/', (req, res) => {  
  res.send('Got a PUT request at /user')  
})

app.delete('/', (req, res) => {  
  res.send('Got a DELETE request')  
})

Notice that in each route handler, we have a req and res parameter. The req has the request object which has the URL, headers and other fields.

The res object lets us render a response back to the client-side.

Request Object

The req parameter we have in the route handlers above is the req object.

It has some properties that we can use to get data about the request that’s made from the client-side. The more important ones are listed below.

req.baseUrl

The req.baseUrl property holds the base URL of the router instance that’s mounted.

For example, if we have:

const express = require('express');
const app = express();  
const greet = express.Router();  
greet.get('/', (req, res) => {  
  console.log(req.baseUrl);  
  res.send('Hello World');  
})

app.use('/greet', greet);

app.listen(3000, () => console.log('server started'));

Then we get /greet from the console.log .

req.body

req.body has the request body. We can parse JSON bodies with express.json() and URL encoded requests with express.urlencoded() .

For example, if we have:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))
app.post('/', (req, res) => {  
  res.json(req.body)  
})

app.listen(3000, () => console.log('server started'));

Then when we make a POST request with a JSON body, then we get back the same that we sent in the request.

req.cookies

We can get cookies that are sent by the request with the req.cookies property.

req.hostname

We can get the hostname from the HTTP header with req.hostname .

When the trust proxy setting doesn’t evaluate to false , then Express will get the value from the X-Forwarded-Host header field. The header can be set by the client or by the proxy.

If there’s more than one X-Forwarded-Host header, then the first one will be used.

For example, if we have:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))

app.get('/', (req, res) => {  
  res.json(req.hostname)  
})

app.listen(3000, () => console.log('server started'));

Then we get the domain name that the app is hosted in if there’re no X-Forwarded-Host headers and trust proxy doesn’t evaluate to false .

req.ip

We can get the IP address that the request is made from with this property.

req.method

The method property has the request method of the request, like GET, POST, PUT or DELETE.

req.params

params property has the request parameters from the URL.

For example, if we have:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))

app.get('/:name/:age', (req, res) => {  
  res.json(req.params)  
})

app.listen(3000, () => console.log('server started'));

Then when we pass in /john/1 as the parameter part of the URL, then we get:

{  
    "name": "john",  
    "age": "1"  
}

as the response from the route above.

req.query

The query property gets us the query string from the request URL parsed into an object.

For example, if we have:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))

app.get('/', (req, res) => {  
  res.json(req.query)  
})

app.listen(3000, () => console.log('server started'));

Then when we append ?name=john&age=1 to the end of the hostname, then we get back:

{  
    "name": "john",  
    "age": "1"  
}

from the response.

Response Object

The response object has some useful methods to let us return various kinds of responses.

res.append

The append method lets us attach response headers to our responses.

For example, if we have the following code:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))

app.get('/', (req, res) => {  
  res.append('Link', 'http://localhost/', 'http://localhost:3000'])  
  res.append('Set-Cookie', 'foo=bar; Path=/; HttpOnly')  
  res.append('Warning', 'Alert')  
  res.send();  
})

app.listen(3000, () => console.log('server started'));

Then when we go to Postman, we should see the same data returned in the Headers tab of the response when we look at the data.

Note that we have to run res.send() to actually send the response.

res.attachment

res.attachment let us add a file to the response. It doesn’t send the response.

For example, we can use it as follows:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))
app.get('/', (req, res) => {  
  res.attachment('../public/foo.txt');  
  res.send();  
})

app.listen(3000, () => console.log('server started'));

Then if we have a foo.txt in the public folder, then the file will be downloaded if we make the request to the route.

Note that again we have res.send() to actually send the response.

res.cookie

res.cookie lets us add a cookie to the response.

For example, if we have:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))

app.get('/', (req, res) => {  
  res.cookie('name', 'foo', { domain: 'repl.it', path: '/', secure: true })  
  res.send();  
})

app.listen(3000, () => console.log('server started'));

Then we send a cookie with name foo to the client. We can check in Postman under the Cookies link in the top right corner.

res.download

res.download sends a file response to the server.

For example, if we have:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))
app.get('/', (req, res) => {  
  res.download('./public/foo.txt');  
})
app.listen(3000, () => console.log('server started'));

Then when a request is made to this route, then we’ll get a file downloaded.

res.json

res.json lets us send a JSON response to the client. The parameter can be any JSON type, including object, array, string, Boolean, number, or null.

For example, if we have:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))

app.get('/', (req, res) => {  
  res.json({ message: 'hi' });  
})

app.listen(3000, () => console.log('server started'));

Then we get:

{"message":"hi"}

as the response.

res.redirect

We can use this to redirect to another URL with the string passed in. For example, if we have:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))
app.get('/', (req, res) => {  
  res.redirect('http://medium.com'));  
})

app.listen(3000, () => console.log('server started'));

Then we’ll see the content of http://medium.com when we make the request to the route above.

res.status

res.status lets us send a status code response. We can use it with the end , send , or sendFile methods by calling them after calling status .

For example, if we have:

const express = require('express')  
const app = express()app.use(express.json())  
app.use(express.urlencoded({ extended: true }))

app.get('/', (req, res) => {  
  res.status(403).end();  
})
app.listen(3000, () => console.log('server started'));

Then we get a 403 response.

Handling All Requests

We can use the app.all method to handle all request methods. To use it, we can write something like the following:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))
app.all('*', (req, res, next) => {  
  console.log('route called');  
  next();  
})

app.get('/', (req, res) => {  
  res.send('hi');  
})

app.listen(3000, () => console.log('server started'));

The app.all call above will be invoked when any method or URL is sent. It’ll log 'route called' and then call next to call the specific route handler for that route.

Non-Letter Symbols

Express can match routes with non-letter symbols. For example, if we have:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))
app.get('/foo.bar', (req, res) => {  
  res.send('hi');  
})

app.listen(3000, () => console.log('server started'));

Then when we make a GET request to foo.bar then we get hi .

Special Characters

Optional Character

If the ? is in a URL path, then it’s considered an optional character. For example, ab?c matches ac or abc .

For example, if we have:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))
app.get('/ab?c', (req, res) => {  
  res.send('hi');  
})

app.listen(3000, () => console.log('server started'));

Then when we make a request to /ac or /abc , we get hi .

One or More Character

If a URL has a + sign, then it’ll match one or more of the character preceding it.

For example, if we have:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))
app.get('/ab+c', (req, res) => {  
  res.send('hi');  
})

app.listen(3000, () => console.log('server started'));

Then when making a request to /abc , /abbc , /abbbc and so on, we’ll get hi .

Wildcard

The * in the URL is a wildcard character.

For example, if we have:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))
app.get('/ab\*c', (req, res) => {  
  res.send('hi');  
})

app.listen(3000, () => console.log('server started'));

Then we put anything between ab and c in the URL and get the hi response.

Optional Character Group

A character group is optional if it’s surrounded by parentheses and has a ? after it.

For example, if we have:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))
app.get('/ab(cd)?ef', (req, res) => {  
  res.send('hi');  
})
app.listen(3000, () => console.log('server started'));

Then when we go to /abcdef or /abef , we get hi .

Regular Expressions

We can also use regular expressions as route paths. For example, if we have:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))
app.get(/\/a/, (req, res) => {  
  res.send('hi');  
})
app.listen(3000, () => console.log('server started'));

Then when we make a request to /a , we get hi .

To get hi when we make any request with path that ends with foo , we can write:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))
app.get(/\/.\*foo$/, (req, res) => {  
  res.send('hi');  
})

app.listen(3000, () => console.log('server started'));

Route Parameters

We can designate parameters by writing a colon before the key name of the parameter.

For example, we can write:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))
app.get('/:name/:age', (req, res) => {  
  res.send(req.params);  
})

app.listen(3000, () => console.log('server started'));

Then we get the following response:

{"name":"Mary","age":"10"}

:name and :age are interpreted as route placeholders. req.params will use them as the keys without the colon. Then whatever is set in the place of the placeholders will be the corresponding values.

Route Handlers

The callback in the second argument of the app.get above are route handlers. The same can be done for POST, PUT and DELETE requests by replacing app.get with app.post , app.put and app.delete respectively.

We can chain multiple handlers together with the next function. For example, we can write:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))
app.get('/',  
  (req, res, next) => {  
    console.log('handler 1 called');  
    next()  
  },  
  (req, res) => {  
    console.log('handler 2 called');  
    res.send();  
  })

app.listen(3000, () => console.log('server started'));

Then we should see:

handler 1 called  
handler 2 called

in the console.log output.

We call the next function which originates from the route handler’s parameter to call the next route handler function.

Middlewares

Characteristics of Middleware

Middleware functions can run any code, make changes to the request and response object, end the request-response cycle, and call the next middleware in the stack.

A middleware may look something like the following code:

app.get('/', (req, res, next) => {  
  next();  
});

The code above has the request method in which the middleware will be called, which is the get in app.get .

Then we have the '/' which is the route path.

Finally, the middleware function that we pass in has the request and response objects as the first 2 parameters respectively, and the next function, which we call to run the next middleware.

Example

For example, we can write a middleware to log some output when we make a request to the / route.

We can write this as follows:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))

app.get('/', (req, res, next) => {  
  console.log('middleware called');  
  next();  
});

app.get('/', (req, res) => {  
  res.send();  
})

app.listen(3000, () => console.log('server started'));

next will call our route handler for the / route.

We should get middleware called from the console.log output and an empty response.

To load the middleware for all routes, we can use app.use instead of app.METHOD where METHOD is the request method is in lower case.

For example, we can write an app-wide middleware as follows:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))

app.use((req, res, next) => {  
  console.log('middleware called');  
  next();  
});

app.get('/', (req, res) => {  
  res.send();  
})

app.listen(3000, () => console.log('server started'));

We should get the same thing as before, but if we have:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))

app.use((req, res, next) => {  
  console.log('middleware called');  
  next();  
});

app.get('/', (req, res) => {  
  res.send();  
})

app.post('/foo', (req, res) => {  
  res.send('foo');  
})

app.listen(3000, () => console.log('server started'));

We get middleware called when we make a GET request to / and a POST request to /foo .

Modifying the Request and Response Objects

We can attach new properties and set values for the request and response objects.

For example, we can write:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))

app.use((req, res, next) => {  
  req.requestTime = Date.now();  
  next();  
});

app.get('/', (req, res) => {  
  res.json(req.requestTime);  
})

app.post('/foo', (req, res) => {  
  res.json(req.requestTime);  
})

app.listen(3000, () => console.log('server started'));

Then we get the timestamp of when the request is made when we make a GET request to / and a POST request to /foo .

Likewise, we can do something similar with the response object:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))

app.use((req, res, next) => {  
  res.responseTime = Date.now();  
  next();  
});

app.get('/', (req, res) => {  
  res.json(res.responseTime);  
})

app.post('/foo', (req, res) => {  
  res.json(res.responseTime);  
})

app.listen(3000, () => console.log('server started'));

We also will get the timestamp of when the response is made when we make a GET request to / and a POST request to /foo .

Configurable Middleware

We can make a function that has optional parameters and return a middleware function to create a configurable middleware function.

For example, we can write one as follows:

const express = require('express')  
const app = express()  
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))

const configurableMiddleware = (options) => {  
  return (req, res, next) => {  
    req.options = options;  
    next();  
  }  
}

app.use(configurableMiddleware({ date: new Date(), foo: 'bar' }));

app.get('/', (req, res) => {  
  res.json(req.options);  
})

app.listen(3000, () => console.log('server started'));

The configurableMiddleware function takes an options object as a parameter and then return a middleware function with the req.options property set to the options parameter.

Then when we make a request to the / route then we get:

{"date":"2019-12-23T22:37:04.927Z","foo":"bar"}

as the response.

app.use([path,] callback [, callback…])

We can use the app.use method to mount middleware that’s run app-wide.

They can be run when requests for specific paths are made or for all paths.

It takes the following arguments:

  • path — it can be a string or regex representing paths or patterns of paths. The default is / .
  • callback — a function to handle requests. It can be a middleware function, a series of them, array of them, or a combination of all of the above

We can provide multiple callbacks that behave just like middleware, but they call next('route') to skip the remaining middleware functions.

We can use these to impose preconditions on some routes and pass control to the route if we don’t need to call the remaining middleware.

Since router and app implement the middleware interface, we can use them like any other middleware function.

For example, we can use it as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));  
app.use((req, res, next) => {  
  console.log(`Request made at ${Date.now()}`)  
  next();  
})

app.get('/', (req, res) => {  
  res.send('hi');  
})

app.listen(3000);

Then we get the request time logged with every request.

We call next to call the route handler.

Middlewares are run sequentially, so the order they’re included in the code is important.

For example, if we have:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));  
app.use((req, res, next) => {  
  res.send('End with middleware');  
})

app.get('/', (req, res) => {  
  res.send('hi');  
})

app.listen(3000);

Then we get End with middleware instead of hi since we sent our response with the middleware, which we included first.

Error-Handling Middleware

We can create our own middleware for handling errors instead of using Express’s default error handler.

The error handler takes 4 arguments, which are the error object, request object, response object, and next function.

For instance, we can define one as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
app.get('/', (req, res, next) => {  
  try {  
    throw new Error('foo');  
  }  
  catch (ex) {  
    next(ex);  
  }  
})

app.use((err, req, res, next) => {  
  res.status(500).send('error');  
})

app.listen(3000);

The error should be displayed since we included the error handler after our GET request handler.

Paths

We can path in a string or regex path or path pattern as the first argument of the use method. This way, the middleware will only be called when the route matches the one specified in string or regex.

To match a constant path, we can write the following:

app.use('/abc', (err, req, res, next) => {  
  res.status(500).send('error');  
})

We can match patterns as follows. A ? will make the character preceding it optional:

app.use('/abc?d', (err, req, res, next) => {  
  res.status(500).send('error');  
})

The when requests made to /abcd or /abd has an error, the middleware above will be called.

The + sign means that one or more characters preceding it will be matched. For example, if we have:

app.use('/abc+d', (err, req, res, next) => {  
  res.status(500).send('error');  
})

The when requests made to /abccd,/abcccd, etc. has an error, the middleware above will be called.

* is a wildcard character. We can use it as follows:

app.use('/ab*cd', (err, req, res, next) => {  
  res.status(500).send('error');  
})

Then we get error when we make a request to /abcccd , /abFoocd , and so on if that has an error.

To match a group of optional characters, we can use parentheses and a question mark after it.

For instance, if we have:

app.use('/a(bcd)?e', (err, req, res, next) => {  
  res.status(500).send('error');  
})

Then we get error when we make a request to /abcde or/ae that has an error.

Passing in Middleware

We can pass in various combinations of middleware in addition to just one middleware.

We can pass in a router object as a middleware as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));const router = express.Router();  
router.get('/', (req, res, next) => {  
  res.send();  
});

app.use(router);app.listen(3000);

An Express app is also valid middleware:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
const subApp = express();
subApp.get('/', (req, res, next) => {  
  res.send();  
});

app.use(subApp);app.listen(3000);

We can also pass in a series of middleware as arguments:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));  
const r1 = express.Router();  
r1.get('/', (req, res, next) => {  
  console.log('r1 called');  
  next();  
});

const r2 = express.Router();  
r2.get('/', (req, res, next) => {  
  console.log('r2 called');  
  next();  
});  
app.use(r1, r2);
app.listen(3000);

Then they’ll be called in order, so we get:

r1 called  
r2 called

in the console.

Likewise, we can pass in an array of middlewares as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));  
const r1 = express.Router();  
r1.get('/', (req, res, next) => {  
  console.log('r1 called');  
  next();  
});

const r2 = express.Router();  
r2.get('/', (req, res, next) => {  
  console.log('r2 called');  
  next();  
});  
app.use([r1, r2]);
app.listen(3000);

Then we get the same output since they’re called in the same order that they’re listed.

A combination of them also works:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));  
const foo = (req, res, next) => {  
  console.log('foo called');  
  next();  
}

const r1 = express.Router();  
r1.get('/', (req, res, next) => {  
  console.log('r1 called');  
  next();  
});

const r2 = express.Router();  
r2.get('/', (req, res, next) => {  
  console.log('r2 called');  
  next();  
});

const subApp = express();  
subApp.get('/', (req, res, next) => {  
  console.log('subApp called');  
  next();  
})

app.use(foo, [r1, r2], subApp);app.listen(3000);

Then we get:

foo called  
r1 called  
r2 called  
subApp called

from the console. They’re still called in the order they’re listed.

Static Files

We can expose static folders with the express.static middleware. For example, if we want to expose the public folder in the project folder, we can write:

app.use(express.static(path.join(__dirname, 'public')));

Choosing Template Engines

We can use popular template engines with Express, like Pug, Mustache, and EJS. The default for the Express application generator is Jade, but we can use the other ones listed as well.

To render template files, we can set the application settings. To set the templates folder, we can set the views property by writing the following:

app.set('views', './views')

It’ll set the templates folder to the views directory of our app.

Also, we can set the view engine we want to use by setting the view engine property as follows:

app.set('view engine', 'pug')

To install the template engine library, we can run something like the following:

npm install pug --save

The command above will install pug view engine for us to use.

Rendering Items with Template Engines

We can use the res.render method to render our variables to our templates.

First, we create the views folder to store our templates as we set the ./views folder to be our templates folder.

Since we chose to use Pug as our template engine, the template files should have the .pug extension.

Inside the views folder, create an index.pug and then put:

html  
  head  
    title= title  
  body  
    h1= message

The title= and h1= are our tags and title and message are keys in which Express will use to get the values from in the object that we’ll pass to res.render .

Then we can create index.js in the same level as the views folder and add:

const express = require('express')  
const app = express()  
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))  
app.set('views', './views');  
app.set('view engine', 'pug');

app.get('/', (req, res, next) => {  
  res.render('index', { pageTitle: 'Hello', pageMessage: 'Hello there!' });  
})

app.listen(3000, () => console.log('server started'));

In the code above, we called res.render to render an HTML response. The object:

{ pageTitle: 'Hello', pageMessage: 'Hello there!' }

will be passed to our index.pug template.

title will be the title in:

html  
  head  
    title= pageTitle  
  body  
    h1= pageMessage

message will be replaced by the 'Hello there!' . Anything right of the tag and the equal sign will be replaced by the value with the key given above.

In other words, pageTitle and pageMessage properties’ values’ are the ones that’ll go in between the title and h1 tags respectively.

So we get

and the title of our page will be 'Hello' .

Pug Syntax

We can have dynamic expressions other than variables. Loops and conditional expressions are part of Pug.

HTML Tags

Other HTML tags can be used with Pug.

For example, a link can be written as follows:

a(href='http://medium.com') Medium

In the code above Medium goes between the a tags and href=’http://medium.com' is the href attribute and value respectively.

Loops

We can also write loops with Pug. For example, if we have:

const express = require('express')  
const app = express()  
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))  
app.set('views', './views');  
app.set('view engine', 'pug');

app.get('/', (req, res, next) => {  
  res.render('index', { pageTitle: 'Hello', pageMessages: ['Hi', 'Bye'] });  
})

app.listen(3000, () => console.log('server started'));

in index.js and:

html  
  head  
    title= pageTitle  
  body  
    ul  
      each pageMessage in pageMessages  
        li= pageMessage

in views/index.pug , then we would get:

Conditionals

We can also add conditional expressions to our Pug template. For example, if we have:

const express = require('express')  
const app = express()  
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))  
app.set('views', './views');  
app.set('view engine', 'pug');

app.get('/', (req, res, next) => {  
  res.render('index', { pageTitle: 'Hello', user: { type: 'member' }   });  
})  
app.listen(3000, () => console.log('server started'));

in index.js and:

html  
  head  
    title= pageTitle  
  body  
    if user.type === 'anonymous'  
      p User type is anonyomous  
    else if user.type === 'member'  
      p User type is member  
    else  
      p User type is admin

in views/index.pug , then we get:

'User type is member'

since user.type in res.render is set to 'member' .

The body-parser Package

By default, Express 4.x or later doesn’t come with anything to parse request bodies. Therefore, we need to add something to do this.

Adding the Body-Parser Package

body-parser is a Node package that we can add onto our Express app to parse request bodies.

It doesn’t support multipart bodies, so we have to use other middleware packages to parse those.

However, it can parse JSON bodies, raw request bodies, text, and URL-encoded bodies.

To install the package, we run:

npm install body-parser

Then we can include by adding:

const bodyParser = require('body-parser');

Parsing JSON Bodies

We can parse JSON bodies by calling the JSON method. It takes an optional object with a few properties.

The options include:

  • inflate — compressed request bodies will be inflated when this is set to true . Otherwise, they’ll be rejected.
  • limit — controls the maximum request body size. If it’s number, then it’s measured in bytes. If it’s a string then it can be parsed into a number of bytes.
  • reviver — this is a function that’s passed into JSON.parse as the second argument to map values to what we want
  • strict — only accepts objects and arrays when set to true . Otherwise, it’ll accept anything that JSON.parse accepts. Defaults to true .
  • type — this is used to determine what media type it’ll parse. It can be a string, array of strings or a function. If it’s not a function, then it’s directly passed into the type-is library. Otherwise, the request is parsed if the data type the function is called with returns a truthy value
  • verify — this is a function with signature (req, res, buf, encoding) , where buf is a Buffer object of the raw request body. The parsing can be aborted by throwing an error in the function.

For example, we can use it as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();  
const options = {  
  inflate: true,  
  limit: 1000,  
  reviver: (key, value) => {  
    if (key === 'age') {  
      if (value < 50) {  
        return 'young'  
      }  
      else {  
        return 'old';  
      }  
    }  
    else {  
      return value;  
    }  
  }  
};  
app.use(bodyParser.json(options));
app.post('/', (req, res) => {  
  res.send(req.body);  
});
app.listen(3000);

Then when we make a POST request to / , we get back:

{  
    "name": "foo",  
    "age": "young"  
}

as the response since we check the age field in the reviver function and return 'young' or 'old' depending on the value . Otherwise, we return the value as-is.

The request body is parsed and set as the value of req.body .

Parsing Raw Bodies

We can parse raw bodies as a buffer. It supports the automatic inflation of gzip and deflate encodings.

The parsed body containing the parsed data is populated on the request object, i.e. it’ll be set as the value of req.body .

It takes an optional option object that can take the following properties:

  • inflate — compressed request bodies will be inflated when this is set to true . Otherwise, they’ll be rejected.
  • limit — controls the maximum request body size. If it’s number, then it’s measured in bytes. If it’s a string then it can be parsed into a number of bytes.
  • type — this is used to determine what media type it’ll parse. It can be a string, array of strings or a function. If it’s not a function, then it’s directly passed into the type-is library. Otherwise, the request is parsed if the data type the function is called with returns a truthy value
  • verify — this is a function with signature (req, res, buf, encoding) , where buf is a Buffer object of the raw request body. The parsing can be aborted by throwing an error in the function.

For example, we can use it as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();  
const options = {  
  inflate: true,  
  limit: 1000,  
  type: 'text/plain'  
};  
app.use(bodyParser.raw(options));
app.post('/', (req, res) => {  
  res.send(req.body);  
});
app.listen(3000);

Then when we send a POST request to / with the body foo , we get back foo as the response.

This is because we specified text/plain as the type to parse the raw data.

It also smart enough to parse multiple body types. We can do that by passing in an array of type strings as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();  
const options = {  
  inflate: true,  
  limit: 1000,  
  type: ['text/plain', 'text/html']  
};  
app.use(bodyParser.raw(options));
app.post('/', (req, res) => {  
  res.send(req.body);  
});
app.listen(3000);

Then when we make a POST request to / with the body <b>foo</b> . Then we get back <b>foo</b> .

Upload Files with Multer

Multer is a middleware that lets us process file uploads with our Express app.

We can install it by running:

npm install --save multer

Once it’s installed we have to make a form that has the encoding type multipart/form-data .

It won’t process any form that’s not multipart, i.e. has the enctype set to multipart/form-data .

Simple Usage

The simplest usage is to allow upload of one file at a time. We can do this by calling the multer function and setting the path to store the uploaded file as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const multer = require('multer');  
const upload = multer({ dest: './uploads/' });  
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
app.use(express.static('public'));
app.get('/', (req, res) => {  
  res.sendFile('public/index.html');  
});
app.post('/upload', upload.single('upload'), (req, res) => {  
  res.send('file uploaded')  
});
app.listen(3000, () => console.log('server started'));

Then we can create index.html in the public folder and add:

<!DOCTYPE html>  
<html>
<head>  
  <meta charset="utf-8">  
  <meta name="viewport" content="width=device-width">  
  <title>Upload</title>  
  <link href="style.css" rel="stylesheet" type="text/css" />  
</head>
<body>  
  <form action="/upload" method="post" enctype="multipart/form-data">  
    <input type="file" name="upload" />      
    <input type='submit' value='Upload'>  
  </form>  
</body>
</html>

The code for index.html will make the form submit multipart form data.

Also, the value of thename attribute has to match with the name string that’s passed into the upload.single() method.

Once the file is upload, a random ID will be generated for the uploaded file’s name.

req.files will have the file objects and req.body will have the values of the text fields.

Uploading Multiple Files

We can also process the upload of multiple files with the upload.array method as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const multer = require('multer');  
const upload = multer({ dest: './uploads/' });  
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
app.use(express.static('public'));
app.get('/', (req, res) => {  
  res.sendFile('public/index.html');  
});
app.post('/upload', upload.array('uploads', 5), (req, res) => {  
  res.send('files uploaded')  
});
app.listen(3000, () => console.log('server started'));

Then we have to change index.html to:

<!DOCTYPE html>  
<html>
<head>  
  <meta charset="utf-8">  
  <meta name="viewport" content="width=device-width">  
  <title>Upload</title>  
  <link href="style.css" rel="stylesheet" type="text/css" />  
</head>
<body>  
  <form action="/upload" method="post" enctype="multipart/form-data">  
    <input type="file" name="uploads" multiple />  
    <input type='submit' value='Upload'>  
  </form>  
</body>
</html>

We add the multiple attribute to the form to enable multiple uploads in addition to using the upload.array method.

The 5 in the second argument of the upload.array call is the maximum number of files that we can upload.

Once the file is upload, a random ID will be generated for the uploaded file’s name.

Multiple File Fields

We can enable upload of multiple files as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const multer = require('multer');  
const upload = multer({ dest: './uploads/' });  
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
app.use(express.static('public'));
app.get('/', (req, res) => {  
  res.sendFile('public/index.html');  
});
const multiUpload = upload.fields([  
  { name: 'photos', maxCount: 3 },  
  { name: 'texts', maxCount: 5 }  
])  
app.post('/upload', multiUpload, (req, res) => {  
  console.log(req.files);  
  res.send('files uploaded');  
});
app.listen(3000, () => console.log('server started'));

Then when we have the following in public/index.html :

<!DOCTYPE html>  
<html><head>  
  <meta charset="utf-8">  
  <meta name="viewport" content="width=device-width">  
  <title>Upload</title>  
  <link href="style.css" rel="stylesheet" type="text/css" />  
</head><body>  
  <form action="/upload" method="post" enctype="multipart/form-data">  
    <label>Photos</label>  
    <br>  
    <input type="file" name="photos" multiple />  
    <br>  
    <label>Texts</label>  
    <br>  
    <input type="file" name="texts" multiple />  
    <br>  
    <input type='submit' value='Upload'>  
  </form>  
</body></html>

We get the files uploaded after we select the files and submit them. The code:

const multiUpload = upload.fields([  
  { name: 'photos', maxCount: 3 },  
  { name: 'texts', maxCount: 5 }  
])

Gets the files from the fields photos and texts and upload to 3 and 5 files respectively.

In the console.log , we see the files of the POST /upload route that we uploaded.

Upload Nothing

To skip the processing of upload files, we can use the upload.none() method as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const multer = require('multer');  
const upload = multer();  
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
app.use(express.static('public'));
app.get('/', (req, res) => {  
  res.sendFile('public/index.html');  
});
app.post('/submit', upload.none(), (req, res) => {  
  res.send(req.body)  
});
app.listen(3000, () => console.log('server started'));

Then when have the following for public/index.html :

<!DOCTYPE html>  
<html><head>  
  <meta charset="utf-8">  
  <meta name="viewport" content="width=device-width">  
  <title>Upload</title>  
  <link href="style.css" rel="stylesheet" type="text/css" />  
</head><body>  
  <form action="/submit" method="post" enctype="multipart/form-data">  
    <input type="text" name="name" />      
    <input type='submit' value='Submit'>  
  </form>  
</body></html>

We get the submitted POST body in req.body as we can see from the response after typing in something and clicking Submit.

Checking File Information

We can check the uploaded files’ information by looking at the req.file object for single file upload and the array entries of req.files for multiple file uploads.

These fields are available for each file:

  • fieldname — field name specified in the form
  • originalname — name of the file on the user’s computer
  • encoding — encoding type of the file
  • mimetype — MIME-type of the file
  • size — size of the file in bytes
  • destination — the folder where the file was saved on the server
  • filename — name of the file stored in the destination
  • path — the full path of the uploaded file
  • bufferBuffer object of the whole file

Options for File Upload

The multer function takes an options object that takes a variety of options.

We can do things like set the destination of the files and rename the files.

The following properties can be in the options object:

  • dest or storage — where to store the files
  • fileFilter — controls which files are accepted
  • limits — limits of the uploaded data
  • preservePath — keep the full path of the files instead of just the base name

Storage Options

DiskStorage

We can store files on disk by using the diskStorage method.

There’re 2 options available, destination and filename . They’re both functions that determine the destination where the file is saved and what to rename the file to respectively.

Each function takes the requestion object, file object and callback function. The callback function is called at the end of each function with the first argument being null .

The second argument is the destination that we want to save the file for the destination function and the filename that we want to rename the file to for the filename function.

For example, we can rename a file by keeping the original name and adding a timestamp to the end of the file as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const multer = require('multer');  
const storage = multer.diskStorage({  
  destination: (req, file, cb) => {  
    cb(null, './uploads/')  
  },  
  filename: (req, file, cb) => {  
    cb(null, `${file.originalname}-${+Date.now()}`)  
  }  
})
const upload = multer({ storage });  
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
app.use(express.static('public'));
app.get('/', (req, res) => {  
  res.sendFile('public/index.html');  
});
app.post('/upload', upload.single('upload'), (req, res) => {  
  res.send('file uploaded')  
});
app.listen(3000, () => console.log('server started'));

MemoryStorage

memoryStorage stores files in memory as a Buffer object and doesn’t take any option.

For example, we can use it as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const multer = require('multer');  
const storage = multer.memoryStorage();  
const upload = multer({ storage })  
const app = express();app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
app.use(express.static('public'));
app.get('/', (req, res) => {  
  res.sendFile('public/index.html');  
});
app.post('/upload', upload.single('upload'), (req, res) => {  
  console.log(req.file);  
  res.send('uploaded');  
});
app.listen(3000, () => console.log('server started'));

Then we get the buffer property in req.file with the content of the upload file as the value of it.

Upload Limits

The limits object specifies the size limits of the following optional properties:

  • fieldNameSize — maximum field name size. Defaults to 100 bytes
  • fieldSize — maximum field value size. Defaults to 1MB
  • fields — the maximum number of non-file fields. Defaults to Infinity
  • fileSize — maximum file size in bytes. Defaults to Infinity
  • files — maximum of file fields. Defaults to Infinity
  • parts — the maximum number of parts (fields and files). Defaults to Infinity
  • headerPairs — the maximum number of header key-value pairs to parse. Defaults to 2000.

This is useful for preventing denial of service attacks.

We can set the limits as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const multer = require('multer');  
const upload = multer({  
  limits: {  
    fieldSize: 1024 * 512,  
    fieldNameSize: 200  
  },  
  dest: './uploads/'  
});  
const app = express();app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
app.use(express.static('public'));
app.get('/', (req, res) => {  
  res.sendFile('public/index.html');  
});
app.post('/upload', upload.single('upload'), (req, res) => {  
  res.send('file uploaded')  
});
app.listen(3000, () => console.log('server started'));

We set the field size limit to 512 KB and field name size to 200 bytes in the code above.

Controlling the Files to Process

The fileFilter field is a function that lets us control which files should be uploaded and which should be skipped.

For example, we can throw an error if the file uploaded doesn’t have the MIME-type image/png and then handle the error in our route as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const multer = require('multer');  
const upload = multer({  
  fileFilter: (req, file, cb) => {  
    if (file.mimetype === 'image/png') {  
      cb(null, true);  
    }  
    else {  
      cb(new multer.MulterError('not a PNG'));  
    }  
  },  
  dest: './uploads/'  
})  
  .single('upload')  
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
app.use(express.static('public'));
app.get('/', (req, res) => {  
  res.sendFile('public/index.html');  
});
app.post('/upload', (req, res) => {  
  upload(req, res, (err) => {  
    if (err instanceof multer.MulterError) {  
      res.send('file not uploaded since it\'s not a PNG');  
    }  
    else {  
      res.send('file uploaded');  
    }  
  })  
});
app.listen(3000, () => console.log('server started'));

First, we have the file type check by setting a function with the fileFilter function:

const upload = multer({  
  fileFilter: (req, file, cb) => {  
    if (file.mimetype === 'image/png') {  
      cb(null, true);  
    }  
    else {  
      cb(new multer.MulterError('not a PNG'));  
    }  
  },  
  dest: './uploads/'  
})  
  .single('upload')

Then in our /upload route, we have:

app.post('/upload', (req, res) => {  
  upload(req, res, (err) => {  
    if (err instanceof multer.MulterError) {  
      res.send('file not uploaded since it\'s not a PNG');  
    }  
    else {  
      res.send('file uploaded');  
    }  
  })  
});

to catch the MulterError and respond accordingly. If the file is a PNG, then it’s uploaded. Otherwise, an error is thrown and the error won’t be uploaded.

In either case, we send a response indicating if the file was uploaded.

Storing Sessions

To store confidential session data, we can use the express-session package. It stores the session data on the server and gives the client a session ID to access the session data.

express-session is a Node package. We can install it by running:

npm install express-session

Then we can include it in our app by adding:

const session = require('express-session');

Usage

The session function returns a middleware to let us store sessions. It takes an option object with the following properties:

cookie

Settings object for the session ID cookie. The default value is:

{ path: '/', httpOnly: true, secure: false, maxAge: null }.

It takes the following properties:

cookie.domain

It specifies the value for the Domain Set-Cookie attribute. No domain is set by default. Most clients will consider the cookie to apply only to the current domain.

cookie.expires

This specifies the Date object of the value for the Expires Set-Cookie attribute. No expiration date is set by default. This means the cookie will be deleted when the session ends.

If both expires and maxAge are set, then the last one defined if what’s used. expires shouldn’t be set directly. We should set maxAge .

cookie.httpOnly

Specifies the boolean value for the HttpOnly Set-Cookie attribute. When it’s truthy, the HttpOnly attribute is set. Otherwise, it’s not. By default, only HttpOnly attribute is set.

cookie.maxAge

The number of milliseconds to use when calculating the Expires Set-Cookie attribute. This is calculated by taking the current server time and adding maxAge milliseconds to it to calculate the Expires datetime. No maximum age is set by default.

cookie.path

Sets the Path Set-Cookie attribute value. This is set to / by default which is the root path of the domain.

cookie.sameSite

A boolean or string for the value of the SameSite Set-Cookie attribute. It can be:

  • true — will set the SameSite attribute to Strict for strict same site enforcement
  • false — won’t set the SameSite attribute
  • 'lax' — set the SameSite attribute to Lax for lax same-site enforcement
  • 'strict' set the SameSite attribute to Strict for strict same-site enforcement

cookie.secure

Boolean value for the Secure Set-Cookie attribute. The Secure attribute is set. Otherwise, it’s not.

Cookies won’t be sent over HTTP connects if this is set to true .

However, it’s recommended. If the app is hosted behind a proxy, then trust proxy must be set if secure is set to true .

genid

A function to generate a new session ID. The default value is a function that uses the uid-safe library to generate IDs.

name

The name of the session ID cookie to set in the response. The default value is 'connect.sid' .

proxy

Trust the reverse proxy when setting cookies via the X-Forwarded-Proto header.

The default value is undefined . Other possibles include:

  • true — the X-Forwarded-Proto header will be used
  • false — all headers are ignored and the connection is considered secure only if it’s a direct SSL/TLS connection
  • undefined — uses the 'trust proxy' setting from Express

resave

Forces the session to be saved back to the session store even if the session was never modified during the request.

This may create race conditions when a client makes 2 parallel requests to our server and changes made to the session on one request may be overwritten when the other request ends.

Defaults to true , but the default may change in the future. false makes more sense as the option.

rolling

Force a session identifier cookie to be set on every response. The expiration is reset to the original maxAge , resetting the expiration countdown.

Defaults to false .

When this is set to true and saveUnintialized option is false , the cookie won’t be set on a response with uninitialized session

saveUnitialized

Force a session that’s uninitialized to be saved to the store. A session is uninitialized when it’s new but not modified. Setting this to false reduces server storage usage and comply with laws that require permission before storing cookies.

secret

This is a required option for the secret to sign the session ID cookie. It can be a string or an array of multiple string secrets.

store

The session store instance. It defaults to a new MemoryStore instance.

unset

Controls the result of unsetting req.session through delete , setting to null , etc.

Defaults to 'keep' . The possible values are:

  • 'destroy' — the session will be deleted when the response ends
  • 'keep' — the session will be kept but modifications made during the request aren’t saved.

req.session

We access the session data by using req.session . The data is typically JSON, so nested objects are fine.

req.session comes with the following methods:

regenerate(callback)

We call this to regenerate the session. Once it’s called, a new session ID and Session instance will be initialized at req.session and the callback will be called.

destroy(callback)

Destroys a session. req.session will be unset. Once it’s called, the callback will be called.

reload(callback)

Reloads the session data from the store and re-populates the req.session object. Once it’s done, the callback will be called.

save(callback)

Saves a session back to the store, replacing the contents of the store with the contents in memory.

This usually doesn’t need to be called.

touch()

Updates the maxAge property. This shouldn’t be called usually as this is done automatically.

id

We can get the session ID with the id property.

cookie

This property has the cookie content. This also lets us alter the cookie for the user.

Cookie.maxAge

We can set the maxAge property to change the expiry time in milliseconds.

Cookie.originalMaxAge

We can use this to get the original maxAge value in milliseconds.

req.sessionID

We can get the session ID of the loaded session with sessionId .

Implementing our Own Session Store

Session store must implement EventEmitter with additional methods.

store.all(callback)

An optional method to get all sessions in the store as an array. The callback signature is callback(error, sessions) and it’s called once the session is retrieved.

store.destroy(sid, callback)

A required method to delete a session from the store given the session ID. The callback has the signaturecallback(error) . It’s called once the session is deleted.

store.clear(callback)

An optional method to delete all sessions from the store. The callback signature is callback(error) and it called once the store is cleared.

store.length(callback)

An optional method to count all the sessions in the store. The callback has the signature callback(error, len).

store.get(sid, callback)

A required method to get the session data by the session ID (sid ). The callback has the signature callback(error, session) .

The session is returned if found, otherwise returns null or undefined .

store.set(sid, session, callback)

A required method to set the session given the sid , which is the session ID. The callback has the signature callback(error) once the session is set.

store.touch(sid, session, callback)

This method is suggested to be implemented. This is used to signal to the store that the given session is active. The callback has the signature callback(error) .

Example

An example would be storing the number of views and keep that data for 365 days. We can do that as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const session = require('express-session');  
const app = express();  
app.set('trust proxy', 1)  
app.use(session({  
  secret: 'secret',  
  resave: true,  
  cookie: {  
    maxAge: 24 * 60 * 60 * 365 * 1000  
  }  
}))
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
app.get('/', (req, res) => {  
  if (req.session.views) {  
    req.session.views++;  
  }  
  else {  
    req.session.views = 1;  
  }  
  res.send(`${req.session.views} views`);  
})
app.listen(3000);

In the code, we have the secret so that the cookie will be signed. Then in our route handler, we can get and set the data by getting and setting req.session.views , which we created.

Then we should see the number of views as we keep reloading the / route. It sets to expire in 365 days by setting maxAge, so the number will keep going up.

In the response header, we should get that the Set-Cookie response header has a value like:

connect.sid=s%3Ad6jfPu5awWj3EXPF-MdtU8cPzjOY5NRT.33nyXjKdShPAPyw9WWwY4sywP44IOX5SPSt2WUkH0cs; Path=/; Expires=Mon, 28 Dec 2020 00:47:31 GMT; HttpOnly

We have the value of the session ID and expiry time.

Conclusion

We can use Express to create back end apps easily.

It comes with a router to route requests.

All logic is written as middleware that we can compose.

We use the body-parser to parse text request payloads.

Multer is used for handling file uploads.

Categories
Express JavaScript Nodejs

Tracking Response Time of Express App Responses

To gauge the performance of our apps, we need to measure the response time of our API.

To do this with our Express app, we can use the response-time package.

Installation

response-time is available as a Node package. We can install it by running:

npm install response-time

Then we can import it by writing:

const responseTime = require('response-time');

responseTime([options])

We can set various options by passing an object into the optional options parameter.

It’ll create a middleware that adds a X-Response-Time header to responses.

The options object takes the following properties:

digits

The digits property is a fixed number of digits to include in the output, which is always in milliseconds. The default is 3.

header

The name of the header to be set. It defaults to X-Response-Time.

suffix

The suffix property is a boolean property to indicate if units of measure should be added to the output. The default value is true.

responseTime(fn)

The function creates a middleware that records the response time of a request and makes it available to our own function fn. fn has the signature (req, res, time) where time is a number in milliseconds.

Examples

Simple Example

We can use the responseTime package without any options or function passed in:

const express = require('express');  
const bodyParser = require('body-parser');  
const responseTime = require('response-time')  
const app = express();  
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));

app.use(responseTime());

app.get('/', (req, res) => {  
  res.send('foo');  
});

app.listen(3000);

Then we get a X-Response-Time response header with a value like 0.587ms.

We can check the response header on an HTTP client like Postman.

Passing in Options

We can change the options for the header returned with the response. For example, we can write the following to change the number of digits sent:

const express = require('express');  
const bodyParser = require('body-parser');  
const responseTime = require('response-time')  
const app = express();  
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));

app.use(responseTime({  
  digits: 5  
}))

app.get('/', (req, res) => {  
  req.id = 1;  
  res.send('foo');  
});

app.listen(3000);

Then we get a X-Response-Time response header with a value like 0.71987ms.

Passing in Our Own Function

We can pass a function into the responseTime function as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const responseTime = require('response-time')  
const app = express();  
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));

app.use(responseTime((req, res, time) => {  
  console.log(`${req.method} ${req.url} ${time}`);  
}))

app.get('/', (req, res) => {  
  req.id = 1;  
  res.send('foo');  
});

app.listen(3000);

Then we get something like:

GET / 2.9935419999999997

from the console.log.

It’s useful if we want to manipulate the response time data or log it ourselves.

Conclusion

We can get the response time in the response header of a request with the response-time package.

It has a responseTime function which returns a middleware that we can use with the use method of express or express.Router() .

The function can either take an options object and or a function with the req, res, and time parameters to get the request, response, and response time respectively.

Categories
Express JavaScript Nodejs

Add Basic Authentication to an Express App

To add basic auth capabilities to an Express app, we can use the express-basic-auth package.

With it, we can check for a given username and password in the URL in our protected routes.

In this article, we’ll take a look at how to use it.

How to Install

express-basic-auth is available as a Node package, we can install it by running:

npm install express-basic-auth

Basic Usage

We can use it as follows:

const express = require('express');  
const basicAuth = require('express-basic-auth')  
const app = express();
app.use(basicAuth({  
    users: { 'admin': 'supersecret' }  
}))

app.get('/', (req, res) => {  
  res.send('authorized');  
});

app.listen(3000, () => console.log('server started'));

In the code above, we included the basicAuth middleware from express-basic-auth from in our app.

Then we can call app.use with an object that has the users property with the usernames as the keys and the corresponding passwords as the values.

Then if we make a GET request to a URL like:

https://admin:supersecret@ThistleDarkExponents--five-nine.repl.co

Then we’ll get the authorized response back since the username and password matches what we listed in the object in our app.

Otherwise, we’ll get a 401 response and a configurable body, which is empty by default.

Custom Authorization

We can also pass in our own authorizer function to check for credentials that we want.

We shouldn’t use standard string comparison with == or === when comparing user input with sercet credentials since it’ll make our app vulenrable to timing attacks.

Timing attacks are a side-channel attack where the attacker tries to compromise a system by analyzing the time taken to run cryptographic algorithms.

We should use the safeCompare method provided by the package instead.

Also, we should use bitwise logic operators instead of standard ones for the same reason.

For example, we can use the following:

const express = require('express');  
const basicAuth = require('express-basic-auth')  
const app = express();
app.use(basicAuth({  
  authorizer: (username, password) => {  
    const userMatches = basicAuth.safeCompare(username, 'admin')  
    const passwordMatches = basicAuth.safeCompare(password, 'supersecret')  
    return userMatches & passwordMatches  
  }  
}))

app.get('/', (req, res) => {  
  res.send('authorized');  
});

app.listen(3000, () => console.log('server started'));

In the code above, we have a function that calls safeCompare to match the username and password respectively and then use the AND bitwise operator to combine the 2 to make sure that both are true .

Async Authorization

We can set the authorizeAsync option to true and set a function with a cb parameter for the callback as the authorizer function to check credentials asynchronously:

const express = require('express');  
const basicAuth = require('express-basic-auth')  
const app = express();
app.use(basicAuth({  
  authorizer: (username, password, cb) => {  
    const userMatches = basicAuth.safeCompare(username, 'admin')  
    const passwordMatches = basicAuth.safeCompare(password, 'supersecret')  
    if (userMatches & passwordMatches)  
      return cb(null, true)  
    else  
      return cb(null, false)  
  },  
  authorizeAsync: true,  
}))

app.get('/', (req, res) => {  
  res.send('authorized');  
});

app.listen(3000, () => console.log('server started'));

We return the cb callback call instead of returning the result directly in our authorizer function.

Unauthorized Response Body

We can set the unauthorizedResponse property to a function that returns a response that we want to show when a credential check failed.

For example, we can write:

const express = require('express');  
const basicAuth = require('express-basic-auth')  
const app = express();  
app.use(basicAuth({  
  users: { 'admin': 'supersecret' },  
  unauthorizedResponse: (req) => {  
    return `unauthorized. ip: ${req.ip}`  
  }  
}))  
app.get('/', (req, res) => {  
  res.send('authorized');  
});  
app.listen(3000, () => console.log('server started'));

to return something like:

unauthorized. ip: ::ffff:172.18.0.1

when basic authentication fails.

The function we set unauthorizedResponse to takes the Express request object, so we can use any properties from there in the function.

Challenge

We can make browsers show a popup so that users can enter credentials for authentication by add a challenge: true option to the object.

In addition, we set the realm to identify the system to authenticate against and can be used to save credentials of the challenge by passing a static or a function that gets passed the request object and is expected to return the challenge:

const express = require('express');  
const basicAuth = require('express-basic-auth')  
const app = express();app.use(basicAuth({  
  users: { 'admin': 'supersecret' },  
  challenge: true,  
  realm: 'foo',  
}))

app.get('/', (req, res) => {  
  res.send('authorized');  
});

app.listen(3000, () => console.log('server started'));

With the code above, we should get a dialog box to enter the username and password when we load the / page.

Conclusion

Adding basic auth to an Express app is easy with the express-basic-auth package.

It lets us add a list of valid credentials or use a function to validate credentials.

We can also let users enter the username and password and display custom content when the user is unauthorized.

Categories
Express JavaScript

Compress Express.js Responses with the Compression Middleware

Compressing our responses is a great way to speed up our Express app’s performance.

To do this, we can use the compression middleware.

In this article, we’ll look at the options available to us for compressing data and how to send a compressed response with it.

Adding the Middleware

We can install the package by running:

npm install compression

Then we can include it with:

const compression = require('compression');

Options

The compression middleware takes an options parameter, which is an object that can have the following properties:

chunkSize

The chunk size of the compressed output. Default is 16384 bytes or zlib.Z_DEFAULT_CHUNK , where zlib is the zlib Node package.

filter

A function that decides what kind of responses should be compressed. The function is called as filter(req, res) and should return true if the kind of response should be compressed and false if it shouldn’t.

The default function uses the compressible module to check if the res.getHeader('Content-Type') is compressible.

level

The compression level to apply The higher the level of compression, the longer it’ll take to complete.

The values range from 0 (no compression) to 9 (highest level of compression). -1 can be used to mean the default compression level, which is equivalent to level 6.

memLevel

This sets how much memory should be allocated for compression and is an integer ranging from 1 for the minimum level to 9 for the maximum.

Default is zlib.Z_DEFAULT_MEMLEVEL or 8.

strategy

This sets the compression algorithm. The value only differs in the compression ratio and doesn’t affect the content of the compressed output.

The following are available:

  • zlib.Z_DEFAULT_STRATEGY — used for normal data
  • zlib.Z_FILTERED — used for data produced by a filter. Filtered data consists mostly of small values with a random distribution. This algorithm is tuned to compressed these kinds of data better. This algorithm forces more Huffman coding.
  • zlib.Z_FIXED — use to prevent the use of dynamic Huffman codes
  • zlib.Z_HUFFMAN_ONLY — force Huffman encoding only
  • zlib.Z_RULE — limit match distances to one, This gives a compression ratio as good as Huffman only, but gives better compression for PNG data

threshold

The byte threshold for response body size before compression is considered for the response. The default 1kb . The value can be the number of bytes or any string applied by the bytes module.

windowBits

The default value is zlib.Z_DEFAULT_WINDOWBITS or 15.

.filter

The default filter function, which we can use to construct a custom filter function that extends the default action.

res.flush

A res.flush method is added to force partially compressed response to be flushed to the client.

Example

For example, we can use it as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const compression = require('compression');
const app = express();  
const shouldCompress = (req, res) => {  
  if (req.headers['x-no-compression']) {  
    return false  
  }  
  return compression.filter(req, res);  
}

app.use(bodyParser.json());  
app.use(compression({  
  filter: shouldCompress,  
  level: 7,  
}));

app.get('/', (req, res) => {  
  const foo = 'foo';  
  res.send(foo.repeat(3000));  
});

app.listen(3000);

In the code above, we add the compression middleware and specify some options.

We specified that we compress our responses when the x-no-compression header isn’t present.

Also, we changed the compression level.

With the x-no-compression header present, our response is 9KB. After compression is enabled by removing the x-no-compression header, the response is only 402 bytes.

As we can see, the difference is huge. This is because the text is repeated so it can just keep one part of it and then repeat it instead of storing the whole string.

Server-Sent Events

We can compress responses with server-sent events. To do this, we compress the content after a window of the output has been buffered.

Once we have that, then we can send the data to the client.

We can use the res.flush() method to send whatever is present to the client.

For example, we can write:

const express = require('express');  
const bodyParser = require('body-parser');  
const compression = require('compression');
const app = express();  
const shouldCompress = (req, res) => {  
  if (req.headers['x-no-compression']) {  
    return false  
  }  
  return compression.filter(req, res);  
}

app.use(bodyParser.json());  
app.use(compression({  
  filter: shouldCompress,  
  level: 7,  
}));

app.get('/', (req, res) => {  
  res.setHeader('Content-Type', 'text/event-stream');  
  res.setHeader('Cache-Control', 'no-cache');  
  const timer = setInterval(() => {  
    res.write('foo');  
    res.flush()  
  }, 2000) res.on('close', () => {  
    clearInterval(timer)  
  })  
});

app.listen(3000);

To send a text event stream to the client, then we should see foo on the screen added every 2 seconds.

Conclusion

The compression middleware is useful for compressing regular responses and server-sent event output.

We can set options like compression level, chunk size, etc.

Also, if we want to compress server-side events, we should call res.flush to send what’s what already buffered to the client compressed.