Categories
JavaScript Nodejs

Using the Node.js OS Module (Part 1)

The Node.js OS module has many useful utility functions for getting information about the computer system that the OS module’s program is running on. It can provide information about hardware such as CPUs, endianness, the home directory, IP address, hostname, the platform the program is running on, system uptime, information about the currently logged in user and more.

We can use the OS module by writing const os = require('os'); at the top of a file. There are many useful properties in the OS module. Below are some of the useful properties in the OS module:

os.EOL

The os.EOL property is a string constant that has the operating system specific end of line marker. For POSIX operating system, it’s \n and for Windows, it’s \r\n . We can use it like the following code as an example:

console.log(`End of line market is ${os.EOL}`)

Then we get:

'End of line market is \n'

os.arch()

The os.arch() function returns a string that tells us the operating system CPU architecture that the Node.js binary is compiled on. Possible values are 'arm', 'arm64', 'ia32', 'mips', 'mipsel', 'ppc', 'ppc64', 's390', 's390x', 'x32', and 'x64'. It’s the same as the process.arch function. For example, we can use it like in the following code:

console.log(`Node.js is built in ${os.arch()}`)

The we get:

'Node.js is built in x64'

if Node.js was compiled on an x64 system

os.constants

The os.constants property has a collection of constants for error codes, process signals, etc. The following are the constants from os.constants :

Signal Constants

  • SIGHUP — signal sent to indicate when a controlling terminal is closed or a parent process exits.
  • SIGINT — signal sent to indicate when a user wishes to interrupt a process ((Ctrl+C)).
  • SIGQUIT — signal sent to indicate when a user wishes to terminate a process and perform a core dump.
  • SIGILL — signal sent to a process to notify that it has attempted to perform an illegal, malformed, unknown, or privileged instruction.
  • SIGTRAP — signal sent to a process when an exception has occurred.
  • SIGABRT — signal sent to a process to request that it abort.
  • SIGIOT — same as SIGABRT
  • SIGABRTSIGBUS — signal sent to a process to notify that it has caused a bus error.
  • SIGFPE — signal sent to a process to notify that it has performed an illegal arithmetic operation.
  • SIGKILL — signal sent to a process to terminate it immediately.
  • SIGUSR1 SIGUSR2 — signal sent to a process to identify user-defined conditions.
  • SIGSEGV — signal sent to a process to notify of a segmentation fault.
  • SIGPIPE — signal sent to a process when it has attempted to write to a disconnected pipe.
  • SIGALRM — signal sent to a process when a system timer elapses.
  • SIGTERM — signal sent to a process to request termination.
  • SIGCHLD — signal sent to a process when a child process terminates.
  • SIGSTKFLT — signal sent to a process to indicate a stack fault on a co-processor.
  • SIGCONT — signal sent to instruct the operating system to continue a paused process.
  • SIGSTOP — signal sent to instruct the operating system to halt a process.
  • SIGTSTP — signal sent to a process to request it to stop.
  • SIGBREAK — signal sent to indicate when a user wishes to interrupt a process.
  • SIGTTIN — signal sent to a process when it reads from the teletypewriter while in the background.
  • SIGTTOU — signal sent to a process when it writes to the teletypewriter while in the background.
  • SIGURG — signal sent to a process when a socket has urgent data to read.
  • SIGXCPU — signal sent to a process when it has exceeded its limit on CPU usage.
  • SIGXFSZ — signal sent to a process when it grows a file larger than the maximum allowed.
  • SIGVTALRM — signal sent to a process when a virtual timer has elapsed.
  • SIGPROF — signal sent to a process when a system timer has elapsed.
  • SIGWINCH — signal sent to a process when the controlling terminal has changed its size.
  • SIGIO — signal sent to a process when I/O is available.
  • SIGPOLL — same as SIGIO
  • SIGIOSIGLOST — signal sent to a process when a file lock has been lost.
  • SIGPWR — signal sentto a process to notify of a power failure.
  • SIGINFO — same as SIGPWR
  • SIGPWRSIGSYS — signal sent to a process to notify of a bad argument.
  • SIGUNUSED — same as SIGSYS

Error Constants for POSIX Systems

The following are constants for error indicators that are raised in POSIX systems.

  • E2BIG — list of arguments is longer than expected.
  • EACCES — the operation did not have sufficient permissions.
  • EADDRINUSE — the network address is already in use.
  • EADDRNOTAVAIL — the network address is currently unavailable for use.
  • EAFNOSUPPORT — he network address family is not supported.
  • EAGAIN — there is currently no data available and to try the operation again later.
  • EALREADY — the socket already has a pending connection in progress.
  • EBADF — file descriptor is not valid.
  • EBADMSG — invalid data message.
  • EBUSY — device or resource is busy.
  • ECANCELED — an operation was canceled.
  • ECHILD — there are no child processes.
  • ECONNABORTED — network connection has been aborted.
  • ECONNREFUSED — network connection has been refused.
  • ECONNRESET — network connection has been reset.
  • EDEADLK — resource deadlock has been avoided.
  • EDESTADDRREQ — a destination address is required.
  • EDOM — an argument is out of the domain of the function.
  • EDQUOT — the disk quota has been exceeded.
  • EEXIST — the file already exists.
  • EFAULT — invalid pointer address.
  • EFBIG — the file is too large.
  • EHOSTUNREACH — the host is unreachable.
  • EIDRM — the identifier has been removed.
  • EILSEQ — an illegal byte sequence encountered.
  • EINPROGRESS — an operation is already in progress.
  • EINTR — a function call was interrupted.
  • EINVAL — an invalid argument was provided.
  • EIO — unspecified I/O error.
  • EISCONN — the socket is connected.
  • EISDIR — the path is a directory.
  • ELOOP — too many levels of symbolic links in a path.
  • EMFILE — there are too many open files.
  • EMLINK — there are too many hard links to a file.
  • EMSGSIZE — the provided message is too long.
  • EMULTIHOP — a multihop was attempted.
  • ENAMETOOLONG — he filename is too long.
  • ENETDOWN — the network is down.
  • ENETRESET — the connection has been aborted by the network.
  • ENETUNREACH — the network is unreachable.
  • ENFILE — too many open files in the system.
  • ENOBUFS — no buffer space is available.
  • ENODATA — no message is available on the stream head read queue.
  • ENODEV — that there is no such device.
  • ENOENT — there is no such file or directory.
  • ENOEXEC — an exec format error.
  • ENOLCK — there are no locks available.
  • ENOLINK — a link has been severed.
  • ENOMEM — there is not enough space.
  • ENOMSG — there is no message of the desired type.
  • ENOPROTOOPT — a given protocol is not available.
  • ENOSPC — there is no space available on the device.
  • ENOSR — there are no stream resources available.
  • ENOSTR — a given resource is not a stream.
  • ENOSYS — a function has not been implemented.
  • ENOTCONN — the socket is not connected.
  • ENOTDIR — the path is not a directory.
  • ENOTEMPTY — the directory is not empty.
  • ENOTSOCK — the given item is not a socket.
  • ENOTSUP — a given operation is not supported.
  • ENOTTY — inappropriate I/O control operation.
  • ENXIO — no such device or address.
  • EOPNOTSUPP — an operation is not supported on the socket. While ENOTSUP and EOPNOTSUPP have the same value on Linux, according to POSIX.1 these error values should be distinct.)
  • EOVERFLOW — a value is too large to be stored in a given data type.
  • EPERM — the operation is not permitted.
  • EPIPE — a pipe is broken.
  • EPROTO — a protocol error.
  • EPROTONOSUPPORT — a protocol is not supported.
  • EPROTOTYPE — wrong type of protocol for a socket.
  • ERANGE — the results are too large.
  • EROFS — the file system is read only.
  • ESPIPE — invalid seek operation.
  • ESRCH — there is no such process.
  • ESTALE — the file handle is stale.
  • ETIME — timer expired
  • ETIMEDOUT — the connection timed out.
  • ETXTBSY — a text file is busy.
  • EWOULDBLOCK — the operation would block.
  • EXDEV — improper link.

Error Constants for Windows Systems

The following are constants for error indicators that are raised in Windows systems.

  • WSAEINTR — an interrupted function call.
  • WSAEBADF — an invalid file handle.
  • WSAEACCES — insufficient permissions to complete the operation.
  • WSAEFAULT — an invalid pointer address.
  • WSAEINVAL — an invalid argument was passed.
  • WSAEMFILE — there are too many open files.
  • WSAEWOULDBLOCK — a resource is temporarily unavailable.
  • WSAEINPROGRESS — an operation is currently in progress.
  • WSAEALREADY — an operation is already in progress.
  • WSAENOTSOCK — the resource is not a socket.
  • WSAEDESTADDRREQ — a destination address is required.
  • WSAEMSGSIZE — the message size is too long.
  • WSAEPROTOTYPE — the wrong protocol type for the socket.
  • WSAENOPROTOOPT — a bad protocol option.
  • WSAEPROTONOSUPPORT — the protocol is not supported.
  • WSAESOCKTNOSUPPORT — the socket type is not supported.
  • WSAEOPNOTSUPP — the operation is not supported.
  • WSAEPFNOSUPPORT — the protocol family is not supported.
  • WSAEAFNOSUPPORT — the address family is not supported.
  • WSAEADDRINUSE — the network address is already in use.
  • WSAEADDRNOTAVAIL — the network address is not available.
  • WSAENETDOWN — the network is down.
  • WSAENETUNREACH — the network is unreachable.
  • WSAENETRESET — the network connection has been reset.
  • WSAECONNABORTED — the connection has been aborted.
  • WSAECONNRESET — the connection has been reset by the peer.
  • WSAENOBUFS — there is no buffer space available.
  • WSAEISCONN — the socket is already connected.
  • WSAENOTCONN — the socket is not connected.
  • WSAESHUTDOWN — data cannot be sent after the socket has been shut down.
  • WSAETOOMANYREFS — there are too many references.
  • WSAETIMEDOUT — the connection has timed out.
  • WSAECONNREFUSED — the connection has been refused.
  • WSAELOOP — a name cannot be translated.
  • WSAENAMETOOLONG — a name was too long.
  • WSAEHOSTDOWN — a network host is down.
  • WSAEHOSTUNREACH — there is no route to a network host.
  • WSAENOTEMPTY — the directory is not empty.
  • WSAEPROCLIM — there are too many processes.
  • WSAEUSERS — the user quota has been exceeded.
  • WSAEDQUOT — the disk quota has been exceeded.
  • WSAESTALE — a stale file handle reference encountered.
  • WSAEREMOTE — the item is remote.
  • WSASYSNOTREADY — the network subsystem is not ready.
  • WSAVERNOTSUPPORTED — the winsock.dll version is out of range.
  • WSANOTINITIALISED — successful WSAStartup has not yet been performed.
  • WSAEDISCON — graceful shutdown is in progress.
  • WSAENOMORE — there are no more results.
  • WSAECANCELLED — an operation has been canceled.
  • WSAEINVALIDPROCTABLE — the procedure call table is invalid.
  • WSAEINVALIDPROVIDER — invalid service provider.
  • WSAEPROVIDERFAILEDINIT — the service provider failed to initialize.
  • WSASYSCALLFAILURE — a system call failure.
  • WSASERVICE_NOT_FOUND — service was not found.
  • WSATYPE_NOT_FOUND — a class type was not found.
  • WSA_E_NO_MORE — there are no more results.
  • WSA_E_CANCELLED — the call was canceled.
  • WSAEREFUSED — a database query was refused.

dlopen Constants

The result of the dlopen command are also included in the os.constants object. The dlopen command is for dynamically loading libraries into memory.

  • RTLD_LAZY — Perform lazy binding. Node.js sets this flag by default.
  • RTLD_NOW — Resolve all undefined symbols in the library before dlopen(3) returns.
  • RTLD_GLOBAL — Symbols defined by the library will be made available for symbol resolution of subsequently loaded libraries.
  • RTLD_LOCAL — Opposite of RTLD_GLOBAL. This is the default behavior if neither RTLD_GLOBAL or RTLD_LOCAL is specified.
  • RTLD_DEEPBIND — Make a self-contained library use its own symbols in preference to symbols from previously loaded libraries.

Priority Constants

The constants below are for setting the priority of scheduled processes. The nice value refers to the integer value for CPU scheduling priority which are used the nice program in Unix and Linux. However, the some of the values are also the same in Windows. The default value is 0. 19 indicates the lowest CPU priority while -20 indicates the highest.

  • PRIORITY_LOW — The lowest process scheduling priority. This corresponds to IDLE_PRIORITY_CLASS on Windows, and a nice value of 19 on all other platforms.
  • PRIORITY_BELOW_NORMAL — This corresponds to BELOW_NORMAL_PRIORITY_CLASS on Windows and a nice value of 10 on all other platforms.
  • PRIORITY_NORMAL — The default process scheduling priority. This corresponds to NORMAL_PRIORITY_CLASS on Windows and a nice value of 0 on all other platforms.
  • PRIORITY_ABOVE_NORMAL — This corresponds to ABOVE_NORMAL_PRIORITY_CLASS on Windows and a nice value of -7 on all other platforms.
  • PRIORITY_HIGH — . This corresponds to HIGH_PRIORITY_CLASS on Windows and a nice value of -14 on all other platforms.
  • PRIORITY_HIGHEST — The highest process scheduling priority. This corresponds to REALTIME_PRIORITY_CLASS on Windows and a nice value of -20 on all other platforms.

The schedule priorities ordered from lowest to highest are — PRIORITY_LOW, PRIORITY_BELOW_NORMAL, PRIORITY_NORMAL, PRIORITY_ABOVE_NORMAL, PRIORITY_HIGH, PRIORITY_HIGHEST .

The Node.js OS module has many useful utility functions for getting information about the computer system that the OS module’s program is running on. The modules have many more properties containing useful information like CPUs, endianness, home directory, IP address, hostname, the platform the program is running on, system uptime, information about the currently logged in user and more.

Categories
JavaScript Nodejs

How To Make a Simple Back End With User Accounts and Authentication

With single-page front-end apps and mobile apps being more popular than ever, the front end is decoupled from the back end. Since almost all web apps need authentication, there needs to be a way for front-end or mobile apps to store user identity data in a secure fashion.

JSON Web Tokens (JWT) is one of the most common ways to store authentication data on front-end apps. With Node.js, there are popular libraries that can generate and verify the JWT by checking for its authenticity. They do this by checking against a secret key stored in the back end and also by checking for an expiry date.

The token is encoded in a standard format that’s understood by most apps. It usually contains user identity data like user ID, user name, etc. It’s given to the user when the user successfully completes authentication.

In this piece, we will build an app that uses JWT to store authentication data.


Overview

For the back end, we’ll use the Express framework, which runs on Node.js, and for the front end, we’ll use the Angular framework. Both have their own JWT add-ons. On the back end, we have the jsonwebtoken package for generating and verify the token.

On the front end, we have the @auth0/angular-jwt module for Angular. In our app, when the user enters user name and password and they are in our database, then a JWT will be generated from our secret key, returned to the user, and stored on the front-end app in local storage. Whenever the user needs to access authenticated routes on the back end, they’ll need the token.

There will be a function in the back-end app called middleware to check for a valid token. A valid token is one that is not expired and verifies as valid against our secret key. There will also be a sign-up and user credential settings pages, in addition to a login page.


Building the App

With this plan, we can begin.

First, we create the front- and back-end app folders. Make one for each.

Then we start writing the back-end app. First, we install some packages and generate our Express skeleton code. We run npx express-generator to generate the code. Then we have to install some packages. We do that by running npm i @babel/register express-jwt sequelize bcrypt sequelize-cli dotenv jsonwebtoken body-parser cors . @babel/register allows us to use the latest JavaScript features.

express-jwt generates the JWT and verifies it against a secret key.bcrypt does the hashing and salting of our passwords. sequelize is our ORM for doing CRUD. cors allows our Angular app to communicate with our back end by allowing cross-domain communication. dotenv allows us to store environment variables in an .env file. body-parser is needed for Express to parse JSON requests.

Then we make our database migrations. First, we run npx sequelize-cli init to generate skeleton code for our database-to-object mapping. Then we run:

npx sequelize-cli model:generate --name User --attributes username:string, password:string, email:string

We make another migration and put:

'use strict';module.exports = {  
  up: (queryInterface, Sequelize) => {  
    return Promise.all([  
      queryInterface.addConstraint(  
        "Users",  
        ["email"],  
        {  
          type: "unique",  
          name: 'emailUnique'  
        }),

      queryInterface.addConstraint(  
        "Users",  
        ["userName"],  
        {  
          type: "unique",  
          name: 'userNameUnique'  
        }),  
  },

  down: (queryInterface, Sequelize) => {  
    return Promise.all([  
      queryInterface.removeConstraint(  
        "Users",  
        'emailUnique'  
      ),

      queryInterface.removeConstraint(  
        "Users",  
        'userNameUnique'  
      ),  
    ])  
  }  
};

This makes sure we don’t have two entries with the same username or email.

This creates the User model and will create the Users table once we run npx sequelize-cli db:migrate .

Then we write some code. First, we put the following in app.js :

require("[@babel/register](http://twitter.com/babel/register)");  
require("babel-polyfill");  
require('dotenv').config();  
const express = require('express');  
const bodyParser = require('body-parser');  
const cors = require('cors');  
const user = require('./controllers/userController');  
const app = express();app.use(cors())  
app.use(bodyParser.urlencoded({ extended: true }));  
app.use(bodyParser.json());

app.use((req, res, next) => {  
  res.locals.session = req.session;  
  next();  
});

app.use('/user', user);app.get('*', (req, res) => {  
  res.redirect('/home');  
});

app.listen((process.env.PORT || 8080), () => {  
  console.log('App running on port 8080!');  
});

We need:

require("@babel/register");  
require("babel-polyfill");

to use the latest features in JavaScript.

And we need:

require('dotenv').config();

to read our config in an .env file.

This is the entry point. We will create userController in the controllers folder shortly.

app.use(‘/user’, user); routes any URL beginning with user to the userController file.

Next, we add the userController.js file:

const express = require('express');  
const bcrypt = require('bcrypt');  
const router = express.Router();  
const models = require('../models');  
const jwt = require('jsonwebtoken');  
import { saltRounds } from '../exports';  
import { authCheck } from '../middlewares/authCheck';

router.post('/login', async (req, res) => {  
    const secret = process.env.JWT_SECRET;  
    const userName = req.body.userName;  
    const password = req.body.password;  
    if (!userName || !password) {  
        return res.send({  
            error: 'User name and password required'  
        })  
    }  
    const users = await models.User.findAll({  
        where: {  
            userName  
        }  
    }) 

    const user = users[0];  
    if (!user) {  
        res.status(401);  
        return res.send({  
            error: 'Invalid username or password'  
        });  
    } 

    try {  
        const compareRes = await bcrypt.compare(password, user.hashedPassword);  
        if (compareRes) {  
            const token = jwt.sign(  
                {  
                    data: {  
                        userName,  
                        userId: user.id  
                    }  
                },  
                secret,  
                { expiresIn: 60 * 60 }  
            );  
            return res.send({ token });  
        }  
        else {  
            res.status(401);  
            return res.send({  
                error: 'Invalid username or password'  
            });  
        }  
    }  
    catch (ex) {  
        logger.error(ex);  
        res.status(401);  
        return res.send({  
            error: 'Invalid username or password'  
        });  
    }});

router.post('/signup', async (req, res) => {  
    const userName = req.body.userName;  
    const email = req.body.email;  
    const password = req.body.password;  
    try {  
        const hashedPassword = await bcrypt.hash(password, saltRounds)  
        await models.User.create({  
            userName,  
            email,  
            hashedPassword  
        })  
        return res.send({ message: 'User created' });  
    }  
    catch (ex) {  
        logger.error(ex);  
        res.status(400);  
        return res.send({ error: ex });  
    }  
});

router.put('/updateUser', authCheck, async (req, res) => {  
    const userName = req.body.userName;  
    const email = req.body.email;  
    const token = req.headers.authorization;  
    const decoded = jwt.verify(token, process.env.JWT_SECRET);  
    const userId = decoded.data.userId;  
    try {  
        await models.User.update({  
            userName,  
            email  
        }, {  
                where: {  
                    id: userId  
                }  
            })  
        return res.send({ message: 'User created' });  
    }  
    catch (ex) {  
        logger.error(ex);  
        res.status(400);  
        return res.send({ error: ex });  
    }});

router.put('/updatePassword', authCheck, async (req, res) => {  
    const token = req.headers.authorization;  
    const password = req.body.password;  
    const decoded = jwt.verify(token, process.env.JWT_SECRET);  
    const userId = decoded.data.userId;  
    try {  
        const hashedPassword = await bcrypt.hash(password, saltRounds)  
        await models.User.update({  
            hashedPassword  
        }, {  
                where: {  
                    id: userId  
                }  
            })  
        return res.send({ message: 'User created' });  
    }  
    catch (ex) {  
        logger.error(ex);  
        res.status(400);  
        return res.send({ error: ex });  
    }});module.exports = router;

The login route searches for the User entry. If it’s found, it then checks for the hashed password with the compare function of bcrypt. If both are successful, then a JWT is generated. The signup route gets the JSON payload of username and password and saves it.

Note that there is hashing and salting on the password before saving. Passwords should not be stored as plain text.

This first is the plain text password, and the second is a number of salt rounds.

updatePassword route is an authenticated route. It checks for the token, and if it’s valid, it will continue to save the user’s password by searching for the User with the user ID from the decoded token.

We will add the authCheck middleware next. We create a middlewares folder and create authCheck.js inside it.

const jwt = require('jsonwebtoken');  
const secret = process.env.JWT_SECRET;export const authCheck = (req, res, next) => {  
    if (req.headers.authorization) {  
        const token = req.headers.authorization;  
        jwt.verify(token, secret, (err, decoded) => {  
            if (err) {  
                res.send(401);  
            }  
            else {  
                next();  
            }  
        });  
    }  
    else {  
        res.send(401);  
    }  
}

You should use the same process.env.JWT_SECRET for generating and verifying the token. Otherwise, verification will fail. The secret shouldn’t be shared anywhere and shouldn’t be checked in to version control.

This allows us to check for authentication in authenticated routes without repeating code. We place it in between the URL and our main route code in each authenticated route by importing and referencing it.

We make an .env file of the root of the back-end app folder, with the following content. (This shouldn’t be checked in to version control.)

DB_HOST='localhost'  
DB_NAME='login-app'  
DB_USERNAME='db-username'  
DB_PASSWORD='db-password'  
JWT_SECRET='secret'

The back-end app is now complete. Now can we can use a front-end app, mobile app, or any HTTP client to sign in.

Categories
Express JavaScript Testing

Adding Tests to Express Apps with Jest and Supertest

Automated tests are essential to the apps we write since modern apps have so many moving parts.

In this piece, we’ll look at how to write apps to test an Express app that interacts with a database with Jest and SuperTest.


Creating the App We’ll Test

We create a project folder by creating an empty folder and running the following to create a package.json file with the default answers:

npm init -y

Then we run the following to install the packages for our apps:

npm i express sqlite3 body-parser

Then, we create the app.js file for our app and write:

const express = require('express');  
const sqlite3 = require('sqlite3').verbose();  
const bodyParser = require('body-parser');  
const app = express();  
const port = process.env.NODE_ENV === 'test' ? 3001 : 3000;  
let db;  
if (process.env.NODE_ENV === 'test') {  
    db = new sqlite3.Database(':memory:');  
}  
else {  
    db = new sqlite3.Database('db.sqlite');  
}

db.serialize(() => {  
    db.run('CREATE TABLE IF NOT EXISTS persons (id INTEGER PRIMARY KEY, name TEXT, age INTEGER)');  
});

app.use(bodyParser.json());  
app.get('/', (req, res) => {  
    db.serialize(() => {  
        db.all('SELECT * FROM persons', [], (err, rows) => {  
            res.json(rows);  
        });  
    })  
})

app.post('/', (req, res) => {  
    const { name, age } = req.body;  
    db.serialize(() => {  
        const stmt = db.prepare('INSERT INTO persons (name, age) VALUES (?, ?)');  
        stmt.run(name, age);  
        stmt.finalize();  
        res.json(req.body);  
    })  
})

app.put('/:id', (req, res) => {  
    const { name, age } = req.body;  
    const { id } = req.params;  
    db.serialize(() => {  
        const stmt = db.prepare('UPDATE persons SET name = ?, age = ? WHERE id = ?');  
        stmt.run(name, age, id);  
        stmt.finalize();  
        res.json(req.body);  
    })  
})

app.delete('/:id', (req, res) => {  
    const { id } = req.params;  
    db.serialize(() => {  
        const stmt = db.prepare('DELETE FROM persons WHERE id = ?');  
        stmt.run(id);  
        stmt.finalize();  
        res.json(req.body);  
    })  
})

const server = app.listen(port);  
module.exports = { app, server };

The code above has the app we’ll test.

To make our app easier to test, we have:

const port = process.env.NODE_ENV === 'test' ? 3001 : 3000;  
let db;  
if (process.env.NODE_ENV === 'test') {  
    db = new sqlite3.Database(':memory:');  
}  
else {  
    db = new sqlite3.Database('db.sqlite');  
}

So we can set the process.env.NODE_ENV to 'test' to make our app listen to a different port than it does when the app is running in a nontest environment.

We’ll use the 'test' environment to run our tests.

Likewise, we want our app to use a different database when running unit tests than when we aren’t running them.

This is why we have:

let db;  
if (process.env.NODE_ENV === 'test') {  
    db = new sqlite3.Database(':memory:');  
}  
else {  
    db = new sqlite3.Database('db.sqlite');  
}

We specified that when the app is running in a 'test' environment we want to use SQLite’s in-memory database rather than a database file.


Writing the Tests

Initialization the code

With the app made to be testable, we can add tests to it.

We’ll use the Jest test runner and SuperTest to make requests to our routes in our tests. To add Jest and SuperTest, we run:

npm i jest supertest

Then, we add app.test.js to the same folder as the app.js file we had above.

In app.test.js, we start by writing the following:

const { app } = require('./app');  
const sqlite3 = require('sqlite3').verbose();  
const request = require('supertest');  
const db = new sqlite3.Database(':memory:');
beforeAll(() => {  
    process.env.NODE_ENV = 'test';  
})

In the code above, we included our Express app from our app.js. Then we also included the SQLite3 and SuperTest packages.,

Then, we connected to our in-memory database with:

const db = new sqlite3.Database(':memory:');

Next, we set all the tests to run in the 'test' environment by running:

beforeAll(() => {  
    process.env.NODE_ENV = 'test';  
})

This will make sure we use port 3001 and the in-memory database we specified in app.js for each test.

To make our tests run independently and with consistent results, we have to clean our database and insert fresh data every time.

To do this, we create a function we call on each test:

const seedDb = db => {  
    db.run('CREATE TABLE IF NOT EXISTS persons (id INTEGER PRIMARY KEY, name TEXT, age INTEGER)');  
    db.run('DELETE FROM persons');  
    const stmt = db.prepare('INSERT INTO persons (name, age) VALUES (?, ?)');  
    stmt.run('Jane', 1);  
    stmt.finalize();  
}

The code above creates the persons table if it doesn’t exist and deletes everything from there afterward.

Then we insert a new value in there to have some starting data.


Adding Tests

With the initialization code complete, we can write the tests.

GET request test

First, we write a test to get the existing seed data from the database with a GET request.

We do this by writing:

test('get persons', () => {  
    db.serialize(async () => {  
        seedDb(db);  
        const res = await request(app).get('/');  
        const response = [  
            { name: 'Jane', id: 1, age: 1 }  
        ]  
        expect(res.status).toBe(200);  
        expect(res.body).toEqual(response);  
    })  
});

We put everything inside the callback of db.serialize so the queries will be run sequentially.

First, we call seedDb, which we created above to create the table if it doesn’t exist, to clear out the database, and to add new data.

Then, we call the GET request by writing:

await request(app).get('/');

This gets us the res object with the response resolved from the promise.

request(app) will start the Express app so we can make the request.

Next, we have the response for us to check against for correctness:

const response = [  
  { name: 'Jane', id: 1, age: 1 }  
]

Then, we check the responses to see if we get what we expect:

expect(res.status).toBe(200);  
expect(res.body).toEqual(response);

The toBe method checks for shallow equality, and toEqual checks for deep equality. So we use toEqual to check if the whole object structure is the same.

res.status checks the status code returned from the server, and res.body has the response body.

POST request test

Next, we add a test for the POST request. It’s similar to the GET request test.

We write the following code:

test('add person', () => {  
    db.serialize(async () => {  
        seedDb(db);  
        await request(app)  
            .post('/')  
            .send({ name: 'Joe', age: 2 }); 
        const res = await request(app).get('/');  
        const response = [  
            { name: 'Jane', id: 1, age: 1 },  
            { name: 'Joe', id: 2, age: 2 }  
        ]  
        expect(res.status).toBe(200);  
        expect(res.body).toEqual(response);  
    })  
});

First, we reset the database with:

seedDb(db);

We made our POST request with:

await request(app)  
  .post('/')  
  .send({ name: 'Joe', age: 2 });

This will insert a new entry into the in-memory database.

Finally, to check for correctness, we make the GET request — like in our first test — and check if both entries are returned:

const res = await request(app).get('/');  
const response = [  
  { name: 'Jane', id: 1, age: 1 },  
  { name: 'Joe', id: 2, age: 2 }  
]  
expect(res.status).toBe(200);  
expect(res.body).toEqual(response);

PUT and DELETE tests

The test for the PUT request is similar to the POST request. We reset the database, make the PUT request with our payload, and then make the GET request to get the returned data, as follows:

test('update person', () => {  
    db.serialize(async () => {  
        seedDb(db);  
        await request(app)  
            .put('/1')  
            .send({ name: 'Joe', age: 2 }); 
        const res = await request(app).get('/');  
        const response = [  
            { name: 'Jane', id: 1, age: 1 }  
        ]  
        expect(res.status).toBe(200);  
        expect(res.body).toEqual(response);  
    })  
});

Then we can replace the PUT request with the DELETE request and test the DELETE request:

test('delete person', () => {  
    db.serialize(async () => {  
        seedDb(db);  
        const res = await request(app).delete('/1');  
        const response = [];  
        expect(res.status).toBe(200);  
        expect(res.body).toEqual(response);  
    })  
});

Running the Tests

To run the tests, we add the following to the scripts section:

"test": "jest --forceExit"

We have to add the --forceExit option so Jest will exist after the tests are run. There’s no fix for the issue where Jest tests using SuperTest don’t exit properly yet.

Then we run the following to run the tests:

npm test

And we should get:

PASS  ./app.test.js  
  √ get persons (11ms)  
  √ add person (2ms)  
  √ update person (2ms)  
  √ delete person (6ms)Test Suites: 1 passed, 1 total  
Tests:       4 passed, 4 total  
Snapshots:   0 total  
Time:        2.559s  
Ran all test suites.  
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?

We should get the same thing no matter how many times we run the tests since we reset the database and made all database queries run sequentially.

Also, we used a different database and port for our tests than other environments, so the data should be clean.


Conclusion

We can add tests run with the Jest test runner. To do this, we have to have a different port and database for running the tests. Then we create the tables if they don’t already exist, clear all the data, and add seed data so we have the same database structure and content for every test.

With SuperTest, we can run the Express app automatically and make the request we want. Then, we can check the output.

Categories
Express JavaScript Testing

Adding Database Interactions to Express Apps

Most back end apps need to interact with a database to do something useful. With Express apps, we can add database interactivity easily.

In this article, we’ll look at how to get and save data into an SQLite database with an Express app.

Getting Started

We get started by creating a project folder, then go in it and run:

npm init -y

to create a package.json file.

The -y flag just answers all the question from npm init with the default options.

Next, we install Express, body-parser to parse request bodies, and sqlite3 for interacting with our SQLite database by running:

npm i express sqlite3 body-parser

Building Our App

Initialization Code

The app we build will get and save data to the persons database.

To start, we create an app.js for the app. Then we create the app by first importing the packages we need to use and setting the port that our app will listen to connections to.

To do this, we add:

const express = require('express');  
const sqlite3 = require('sqlite3').verbose();  
const bodyParser = require('body-parser');  
const app = express();  
const port = 3000;

We include the packages we installed earlier, including Express, body-parser , and sqlite3 .

Then we create an instance of our Express app and set the port to 3000 to listen to requests on port 3000.

Next, we create our database initialization code. We do this by writing:

const db = new sqlite3.Database('db.sqlite');

The code above tells us that our app will use db.sqlite to get and save data. If it doesn’t exist, it’ll be created on the fly so we don’t have to create it beforehand. Also, we don’t need any credentials to use the database file.

Then we create the person table if it doesn’t already exist by writing:

db.serialize(() => {  
    db.run('CREATE TABLE IF NOT EXISTS persons (id INTEGER PRIMARY KEY, name TEXT, age INTEGER)');  
});

We have an id auto-incrementing integer column, and name and age columns for saving some personal data.

We include IF NOT EXISTS in our CREATE TABLE query so that the database won’t be attempted to be created every time the app restarts. Since if it already exists, the query will fail if the table already exists.

Then we include our body-parser middleware so that JSON request bodies will be parsed into the req.body object as follows:

app.use(bodyParser.json());

Adding the Routes

Now we add the routes for getting and saving data.

First, we add a route to handle GET requests for getting data from the persons table, we add the following:

app.get('/', (req, res) => {  
    db.serialize(() => {  
        db.all('SELECT * FROM persons', [], (err, rows) => {  
            res.json(rows);  
        });  
    })  
})

We use app.get to handle the GET request to the / route.

db.serialize makes sure that each query inside the callback runs in sequence.

Then we select all the rows from the persons table and send the array returned as the response.

Next, we add our POST route to handle the POST request for saving a new entry to the database:

app.post('/', (req, res) => {  
    const { name, age } = req.body;  
    db.serialize(() => {  
        const stmt = db.prepare('INSERT INTO persons (name, age) VALUES (?, ?)');  
        stmt.run(name, age);  
        stmt.finalize();  
        res.json(req.body);  
    })  
})

We start by using the app.post to indicate that we’re handling a POST request. The / means that we’ll handle requests to the / path.

Next, we get the parsed JSON request body returned by body-parser by writing:

const { name, age } = req.body;

We decomposed the fields into variables with the destructing assignment operator.

Next, we create a prepared statement to insert data to the persons table with db.prepare. This lets us set data for the placeholders marked by the ? and also sanitizes the data to avoid SQL injection attacks.

Then we run:

stmt.run(name, age);  
stmt.finalize();

to run the statement and return the response with res.json(req.body); .

Next, we add the route to let us update an entry:

app.put('/:id', (req, res) => {  
    const { name, age } = req.body;  
    const { id } = req.params;  
    db.serialize(() => {  
        const stmt = db.prepare('UPDATE persons SET name = ?, age = ? WHERE id = ?');  
        stmt.run(name, age, id);  
        stmt.finalize();  
        res.json(req.body);  
    })  
})

It’s similar to the POST request route above, except that we’re handling PUT requests.

The difference is that we have an :id in the URL parameter, which we get by writing:

const { id } = req.params;

So when we make a request to /1 , id will be set to 1.

Then we ran our update statement with name , age , and id and return the request body as the response.

Finally, we have our DELETE route to let us delete items from the persons table as follows:

app.delete('/:id', (req, res) => {  
    const { id } = req.params;  
    db.serialize(() => {  
        const stmt = db.prepare('DELETE FROM persons WHERE id = ?');  
        stmt.run(id);  
        stmt.finalize();  
        res.json(req.body);  
    })  
})

It’s similar to the other routes except that we have a delete request and we run a delete statement with the ID.

Then we return the request body as the response.

Finally, we add the following line:

const server = app.listen(port);

So that when we run node app.js , our app will run.

Conclusion

We run our app by running node app.js .

In the end, we have the following code if we put everything together:

const express = require('express');  
const sqlite3 = require('sqlite3').verbose();  
const bodyParser = require('body-parser');  
const app = express();  
const port = 3000;  
const db = new sqlite3.Database('db.sqlite');
db.serialize(() => {  
    db.run('CREATE TABLE IF NOT EXISTS persons (id INTEGER PRIMARY KEY, name TEXT, age INTEGER)');  
});

app.use(bodyParser.json());  
app.get('/', (req, res) => {  
    db.serialize(() => {  
        db.all('SELECT * FROM persons', [], (err, rows) => {  
            res.json(rows);  
        });  
    })  
})

app.post('/', (req, res) => {  
    const { name, age } = req.body;  
    db.serialize(() => {  
        const stmt = db.prepare('INSERT INTO persons (name, age) VALUES (?, ?)');  
        stmt.run(name, age);  
        stmt.finalize();  
        res.json(req.body);  
    })  
})

app.put('/:id', (req, res) => {  
    const { name, age } = req.body;  
    const { id } = req.params;  
    db.serialize(() => {  
        const stmt = db.prepare('UPDATE persons SET name = ?, age = ? WHERE id = ?');  
        stmt.run(name, age, id);  
        stmt.finalize();  
        res.json(req.body);  
    })  
})

app.delete('/:id', (req, res) => {  
    const { id } = req.params;  
    db.serialize(() => {  
        const stmt = db.prepare('DELETE FROM persons WHERE id = ?');  
        stmt.run(id);  
        stmt.finalize();  
        res.json(req.body);  
    })  
})

const server = app.listen(port);

Once we include a database library, we can create an Express app that interacts with a database to make our apps more useful.

Categories
Express JavaScript Nodejs

Configuring Express Multer Middleware and Checking File Information

Multer is a middleware that lets us process file uploads with our Express app.

In this article, we’ll look at the settings that we can change to upload files our way, and also check the file information.

Checking File Information

We can check the uploaded files’ information by looking at the req.file object for single file upload and the array entries of req.files for multiple file uploads.

These fields are available for each file:

  • fieldname — field name specified in the form
  • originalname — name of the file on the user’s computer
  • encoding — encoding type of the file
  • mimetype — MIME-type of the file
  • size — size of the file in bytes
  • destination — the folder where the file was saved on the server
  • filename — name of the file stored in the destination
  • path — the full path of the uploaded file
  • bufferBuffer object of the whole file

Options for File Upload

The multer function takes an options object that takes a variety of options.

We can do things like set the destination of the files and rename the files.

The following properties can be in the options object:

  • dest or storage — where to store the files
  • fileFilter — controls which files are accepted
  • limits — limits of the uploaded data
  • preservePath — keep the full path of the files instead of just the base name

Storage Options

DiskStorage

We can store files on disk by using the diskStorage method.

There’re 2 options available, destination and filename . They’re both functions that determine the destination where the file is saved and what to rename the file to respectively.

Each function takes the requestion object, file object and callback function. The callback function is called at the end of each function with the first argument being null .

The second argument is the destination that we want to save the file for the destination function and the filename that we want to rename the file to for the filename function.

For example, we can rename a file by keeping the original name and adding a timestamp to the end of the file as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const multer = require('multer');  
const storage = multer.diskStorage({  
  destination: (req, file, cb) => {  
    cb(null, './uploads/')  
  },  
  filename: (req, file, cb) => {  
    cb(null, `${file.originalname}-${+Date.now()}`)  
  }  
})
const upload = multer({ storage });  
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
app.use(express.static('public'));
app.get('/', (req, res) => {  
  res.sendFile('public/index.html');  
});
app.post('/upload', upload.single('upload'), (req, res) => {  
  res.send('file uploaded')  
});
app.listen(3000, () => console.log('server started'));

MemoryStorage

memoryStorage stores files in memory as a Buffer object and doesn’t take any option.

For example, we can use it as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const multer = require('multer');  
const storage = multer.memoryStorage();  
const upload = multer({ storage })  
const app = express();app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
app.use(express.static('public'));
app.get('/', (req, res) => {  
  res.sendFile('public/index.html');  
});
app.post('/upload', upload.single('upload'), (req, res) => {  
  console.log(req.file);  
  res.send('uploaded');  
});
app.listen(3000, () => console.log('server started'));

Then we get the buffer property in req.file with the content of the upload file as the value of it.

Upload Limits

The limits object specifies the size limits of the following optional properties:

  • fieldNameSize — maximum field name size. Defaults to 100 bytes
  • fieldSize — maximum field value size. Defaults to 1MB
  • fields — the maximum number of non-file fields. Defaults to Infinity
  • fileSize — maximum file size in bytes. Defaults to Infinity
  • files — maximum of file fields. Defaults to Infinity
  • parts — the maximum number of parts (fields and files). Defaults to Infinity
  • headerPairs — the maximum number of header key-value pairs to parse. Defaults to 2000.

This is useful for preventing denial of service attacks.

We can set the limits as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const multer = require('multer');  
const upload = multer({  
  limits: {  
    fieldSize: 1024 * 512,  
    fieldNameSize: 200  
  },  
  dest: './uploads/'  
});  
const app = express();app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
app.use(express.static('public'));
app.get('/', (req, res) => {  
  res.sendFile('public/index.html');  
});
app.post('/upload', upload.single('upload'), (req, res) => {  
  res.send('file uploaded')  
});
app.listen(3000, () => console.log('server started'));

We set the field size limit to 512 KB and field name size to 200 bytes in the code above.

Controlling the Files to Process

The fileFilter field is a function that lets us control which files should be uploaded and which should be skipped.

For example, we can throw an error if the file uploaded doesn’t have the MIME-type image/png and then handle the error in our route as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const multer = require('multer');  
const upload = multer({  
  fileFilter: (req, file, cb) => {  
    if (file.mimetype === 'image/png') {  
      cb(null, true);  
    }  
    else {  
      cb(new multer.MulterError('not a PNG'));  
    }  
  },  
  dest: './uploads/'  
})  
  .single('upload')  
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
app.use(express.static('public'));
app.get('/', (req, res) => {  
  res.sendFile('public/index.html');  
});
app.post('/upload', (req, res) => {  
  upload(req, res, (err) => {  
    if (err instanceof multer.MulterError) {  
      res.send('file not uploaded since it\'s not a PNG');  
    }  
    else {  
      res.send('file uploaded');  
    }  
  })  
});
app.listen(3000, () => console.log('server started'));

First, we have the file type check by setting a function with the fileFilter function:

const upload = multer({  
  fileFilter: (req, file, cb) => {  
    if (file.mimetype === 'image/png') {  
      cb(null, true);  
    }  
    else {  
      cb(new multer.MulterError('not a PNG'));  
    }  
  },  
  dest: './uploads/'  
})  
  .single('upload')

Then in our /upload route, we have:

app.post('/upload', (req, res) => {  
  upload(req, res, (err) => {  
    if (err instanceof multer.MulterError) {  
      res.send('file not uploaded since it\'s not a PNG');  
    }  
    else {  
      res.send('file uploaded');  
    }  
  })  
});

to catch the MulterError and respond accordingly. If the file is a PNG, then it’s uploaded. Otherwise, an error is thrown and the error won’t be uploaded.

In either case, we send a response indicating if the file was uploaded.

Conclusion

Multer has lots of options for us to control how file upload is done. We can check the file type, set the destination, control how much we can upload, etc.

Also, we can catch errors in our routes by using the upload function inside our route handler instead of using it as a middleware.

We can also check for file information with req.file for single file upload and the req.files array for multiple file upload.

Finally, we can change where files are stored by changing the destination and storage type.