Category: Node.js Tips

As with any kind of app, there are difficult issues to solve when we write Node apps. In this article, we’ll look at some solutions to common problems when writing Node apps.

Overwrite a File Using fs in Node.js

fs.writeFileSync and fs.writeFile both overwrite the file by default.

Therefore, we don’t have to add any extra checks.

Also, we can set the 'w' flag to make sure we write to the file:

fs.writeFileSync(path, content, {
  encoding: 'utf8',
  flag: 'w'
})

We set the option in the 3rd argument.

Defining an Array as an Environment Variable in Node.js

We can set environment variables as a comma-separated string as its value.

Then we can get the string and call split to split the environment variable string with a comma.

For example, we can write:

app.js

const names = process.env.NAMES.split(',');

Then when we run:

NAMES=bar,baz,foo node app.js

Then process.env.NAMES will be 'bar,baz,foo' .

And then we can call split as we did above to convert it to an array.

Make POST Request Using Node.js

We can make a POST request using the http module.

For instance, we can write:

const http = require('http')

const body = JSON.stringify({
  foo: "bar"
})

const request = new http.ClientRequest({
  hostname: "SERVER_NAME",
  port: 80,
  path: "/some-path",
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "Content-Length": Buffer.byteLength(body)
  }
})

request.end(body);

We use the http.ClientRequest constructor to create our request.

We specify the hostname which is the hostname.

port is the port.

path is the path relative to the hostname.

method is the request method, which should be 'POST' to make a POST request.

headers have the request headers.

Then we call request.end to make a request with the body , which is the request body.

Then to listen to the request-response, we listen to the response event.

For instance, we can write:

request.on('response', (response) => {
  console.log(response.statusCode);
  console.log(response.headers);
  response.setEncoding('utf8');
  response.on('data', (chunk) => {
    console.log(chunk);
  });
});

We listen to the response event on the request object.

The callback has the response object which is the read stream with the response .

It also has the statusCode to get the status code.

headers have the response headers.

We listen to the data event on the response to get the response in the chunk parameter.

Together, we have:

const http = require('http')

const body = JSON.stringify({
  foo: "bar"
})

const request = new http.ClientRequest({
  hostname: "SERVER_NAME",
  port: 80,
  path: "/some-path",
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "Content-Length": Buffer.byteLength(body)
  }
})

request.end(body);

request.on('response', (response) => {
  console.log(response.statusCode);
  console.log(response.headers);
  response.setEncoding('utf8');
  response.on('data', (chunk) => {
    console.log(chunk);
  });
});

Running Async Code in Series

We can use the async module’s series method to run multiple pieces of async code in series.

For instance, we can write:

const async = require('async');

const foo = (callback) => {
  setTimeout(() => {
    callback(null, 'foo');
  }, 5000);
}

const bar = (callback) => {
  setTimeout(() => {
    callback(null, 'bar');
  }, 2000);
}

async.series([
  foo,
  bar
], (err, results) => {
  console.log(results);
});

We have 2 functions foo and bar which runs setTimeout and takes a Node-style callback.

The callback parameter in each function takes an error object and the result.

Then we can pass the functions to the async.series method after putting them in an array.

Since the signature of the functions match with async.series is looking for, we’ll get the results of each function in the results parameter, which is an array.

It has all the results of each function that we passed as the 2nd argument of callback .

This means results is ['foo', 'bar'] .

Set Navigation Timeout with Node Puppeteer

We can set the navigation timeout with the setDefaultNavigationTimeout method.

For instance, we can write:

await page.setDefaultNavigationTimeout(0);

to set the default timeout in milliseconds.

The timeout will affect goBack , goForward , goto , reload , setContent , and waitForNavigation .

Conclusion

We can set the navigation timeout with Puppeteer. To make a POST request, we can use the http.ClientRequest constructor. Also, we can use the async.series method to run functions that run asynchronously in the Node format sequentially. Environment variables are always strings. writeFile and writeFileSync always overwrite files.

As with any kind of app, there are difficult issues to solve when we write Node apps. In this article, we’ll look at some solutions to common problems that we might encounter when writing Node apps.

Log Inside page.evaluate with Puppeteer

We can log output when page.evaluate is run by listening to the console event.

For instance, we can write:

const page = await browser.newPage();
page.on('console', consoleObj => console.log(consoleObj.text()));

We call browser.newPage to create a new page object.

Then we listen to the console event with it.

It takes a callback, which we can convert to text with the text method.

Testing Asynchronous Function with Mocha

We can run async functions with Mocha tests to test them.

For instance, we can write:

it('should do something', async function() {
  this.timeout(40000);
  const result = await someFunction();
  assert.isBelow(result, 3);
});

We call this.timeout to set the timeout before the test times out.

Then we use await to run our async function, which returns a promise.

Finally, we get the result and use assert to check it.

Acknowledgment for socket.io Custom Event

We can listen to events after a connection is established.

On the server-side, we can listen to the connection event to see if a connection is established.

Once it is, then we emit an event to acknowledge the connection is made.

Likewise, we can do the same on the client-side.

For instance, in our server-side code, we write:

io.sockets.on('connection', (sock) => {
   sock.emit('connected', {
     connected: 'yes'
   });

   sock.on('message', (data, callback) => {
     console.log('received', data);
     const responseData = {
       hello: 'world'
     };

     callback(responseData);
   });
 });

We listen to the connection event to listen to connections.

Then we emit the connected event to acknowledge the connection.

We also listen to the message event which takes data and a callback, which we call to send data back to the client.

The on the client-side, we write:

const socket = io.connect('http://localhost:3000');
socket.on('error', (err) => {
  console.error(err);
});

socket.on('connected', (data) => {
  console.log('connected', data);
  socket.emit('message', {
    payload: 'hello'
  }, (responseData) => {
    console.log(responseData);
  });
});

We connect to the server with io.connect .

And we listen to the error even which runs a callback if there’s an error.

We also listen to the connected event for any data that’s sent.

And we emit a message event with some data and a callback which the server calls to get the data we passed into the callback function in the server-side code.

responseData is the responseData passed into callback in the server-side code.

Download Large File with Node.js while Avoiding High Memory Consumption

We can make an HTTP request with the http module.

Then we listen to the response event, which has the downloaded file’s content.

We create a write stream with the file path.

Then we listen to the data event emitted on the response object to get the chunks of data.

We also listen to the end event emitted from response so that we can stop writing to the stream.

For example, we can write:

const http = require('http');
const fs = require('fs');

const download = (url, dest, cb) => {
  const file = fs.createWriteStream(dest);
  const request = http.get(url, (response) => {
    response.pipe(file);
    file.on('finish', () => {
      file.close(cb);
    });
  }).on('error', (err) => {
    fs.unlink(dest);
    if (cb) cb(err.message);
  });
};

We created a write stream with fs.createWriteStream .

Then we make our request with the http.get method.

We call response.pipe to pipe the response to our write stream, which writes it to the file.

We listen to the finish event and close the write stream in the callback.

If there’s an error, we delete what’s written so far with unlink .

We listen to the error event to watch for errors.

cb is a callback that we call to deliver the results.

Since the data is obtained in chunks, memory usage shouldn’t be an issue.

Conclusion

We can download a file and write it to disk with a write stream. We can log the output with Puppeteer. Mocha can run async functions in tests. We can send events to acknowledge the emission of custom events.

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Use the Middleware to Check the Authorization Before Entering Each Route in Express

We can add a middleware to a route to check for the proper credentials before entering a route.

For instance, we can write:

const protectedMiddlewares = [authChecker, fetchUser];
const unprotectedMiddlewares = [trackVisistorCount];

app.get("/", unprotectedMiddlewares, (req, res) => {
  //...
})

app.get("/dashboard", protectedMiddlewares, (req, res) => {
  // ...
})

app.get("/auth", unprotectedMiddlewares, (req, res) => {
  //...
})

app.put("/auth", unprotectedMiddlewares, (req, res) => {
  //...
})

We have an array of middleware for the protected dashboard route.

And we have an array of middleware for the unprotected routes.

A middleware may look like the following:

const authChecker = (req, res, next) => {
  if (req.session.auth || req.path === '/auth') {
    next();
  } else {
    res.redirect("/auth");
  }
}

We have the authChecker middleware that checks the path of the route and the session.

If there’s a session set, then we call the next middleware which should lead to a protected route handler in the end.

Otherwise, we redirect to the auth route.

Testing Requests that Redirect with Mocha and Supertest in Node

To test requests that redirect to another route with Supertest running in the Mocha test runner, we can write;

it('should log out the user', (done) => {
  request(app)
    .post('/login')
    .type('form')
    .field('user', 'username')
    .field('password', 'password')
    .end((err, res) => {
      if (err) { return done(err); }
      request(app)
        .get('/')
        .end((err, res) => {
          if (err) { return done(err); }
          res.text.should.include('profile');
          done();
        });
    });
});

We test a login route for redirection by checking the response text in the second end callback.

To do that, we made a POST request with the username and password.

Then that callback is called after the redirect is done.

We can get the text from the response to see if it’s the text in the profile route.

Send Additional HTTP Headers with Express

We can send whatever response headers we want with the setHeader method of the response object.

For instance, we can write:

app.use((req, res, next) => {
  res.setHeader("Access-Control-Allow-Origin", "*");
  return next();
});

We set the Access-Control-Allow-Origin to * to allow requests from all origins.

Working with Sessions in Express

To work with sessions in an Express app, we can use the expression-middleware.

For instance, we can use it by writing:

const express = require('express');
const session = require('express-session');

const app = express();

app.use(session({
  resave: false,
  saveUninitialized: false,
  secret: 'secret'
}));

app.get('/', (req, res) => {
  if (req.session.views) {
    ++req.session.views;
  } else {
    req.session.views = 1;
  }
  res.send(`${req.session.views} views`);
});

app.listen(3000);

We used the express-session package with a few options.

session is a function that returns a middleware to let us work with sessions.

resave means we don’t save a session if it’s not modified is it’s false .

saveUninitialized means that we don’t create a session until something is stored if it’s false .

secret is the secret for signing the sessions.

Then we can store whatever data we want in the req.session property.

It’ll persist until the session expires.

We just keep increasing the views count as we hit the / route.

The default is the memory store, which should only be used if we don’t have multiple instances.

Otherwise, we need to use persistent sessions.

Get the List of Connected Clients Username using Socket.io

We can get the list of clients connected to a namespace or not by using the clients method.

For instance, we can write:

const clients = io.sockets.adapter.rooms[roomId];
for (const clientId of Object.keys(clients)) {
  console.log(clientId);
  const clientSocket = io.sockets.connected[clientId];
}

We get an object with the client IDs as the keys with the io.socket.adapter.rooms object.

roomId is the ID of the room we’re in.

Then we can get the socket for the client with the io.socket.connected object.

Conclusion

We can get the clients with Socket.io.

Also, we can write our own middlewares to check for credentials before proceeding to the route.

express-session lets us work with sessions in our Express app.

We can test redirects with Supertest by checking the response content in the end callback.

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Programmatically Stop and Restart Express Servers to Change Ports

We can stop the server and restart it by writing:

server.on('close', () => {
  server.listen(3000);
});

server.listen(8000, () => {
  server.close();
});

We added a listener for the close event that’s run when the server is closed.

Then we call listen with a port.

We have the callback that calls the close method that triggers the close event.

And then the callback we passed to server.on will run.

Automatically Add Header to Every “render” Response

We can add a header to every render response by creating our own middleware to add the response header.

For instance, we can write:

app.get('/*', (req, res, next) => {
  res.header('X-XSS-Protection' , 0);
  next();
});

We add the X-XSS-Protection custom response header with value 0.

We can make the middleware run app-wide by writing:

app.use((req, res, next) => {
  res.set('X-XSS-Protection', 0);
  next();
});

Then we can add a test by installing Mocha and the hipper package by running:

npm install --save-dev mocha hippie

Also, we can test that by writing:

describe('Response Headers', () => {
  it('responds with header X-XSS-Protection with value 0', (done) => {
    hippie(app)
      .get('/foo')
      .expectHeader('X-XSS-Protection', 0)
      .end(done);
  });
});

We import the app object which we have in the previous example.

Then we pass that to hippie and call a route and then call expectHeader to check that the header returned.

And then we call end with done to finish the test.

Set Cache Control Header for Dynamic Data Express

We can set the Cache-Control header with our own value.

For instance, we can write:

res.set('Cache-Control', 'public, max-age=86400');

which sets the Cache-Control header to one day, which is 86400 seconds.

We can use it in a middleware by writing:

app.use((req, res, next) => {
  res.set('Cache-Control', 'public, max-age=86400');
  next();
})

We just put the line in and call next to call the next middleware.

Express Serve Static Folder Relative to Parent Directory

We can serve a folder as a static folder relative to a directory with path.join .

For instance, we can write:

const path = require('path');
const express = require('express');
const app = express();

app.use(express.static(path.join(__dirname, '../public')));

We call express.static to set the public folder which is one level above the current folder as a folder for serving static files.

Send Multiple Response Chunks in one Route

If need to send our responses in chunks, we can use the response.write method.

For instance, we can write:

response.write("foo");
response.write("bar");
response.end();

We called response.write twice to send different parts of the response,

Then we call response.end to finish sending the response.

Unit Test with a File Upload in Mocha

We can test file upload by writing:

const should = require('should');
const supertest = require('supertest');
const request = supertest('localhost:3000');

describe('upload', () => {
  it('a file', (done) => {
    request
      .post('/upload')
      .field('foo', 'bar')
      .attach('image', 'pic.jpg')
      .end((err, res) => {
        res.should.have.status(200);
        done();
      });
  });
});

We make a POST request with supertest and call attach to attach the file with the given name.

field has the text fields that we want to submit with the file upload.

Then we check that the response is 200 with the callback.

We can attach more than one field or file attachment.

Listen to socket.io Errors in an Express App

We can listen to socket.io in an Express app by passing in the server object returned by listen into the socket.io function.

For instance, we can write:

const express = require('express');
const app = express();
const server = app.listen(port);
const io = require('socket.io')(server);

We pass the server into the function returned by the socket.io package,.

Also, we can write:

const http = require('http');
const express = require('express');
const app = express();

const server = http.createServer(app);
consr io = require('socket.io').listen(server);
server.listen(80);

We pass the server returned from the http module into the listen method instead.

Conclusion

We can set the header by using the res.header method.

There are multiple ways to listen to errors from socket.io in an Express app.

We can also test file uploads with Supertest.

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Force Parse Request Body as Plain Text Instead of JSON in Express

We can remove the body-parser middleware to parse the request body as raw text.

For instance, we can write:

const express = require('express');

const app = express();

const parseRawBody = (req, res, next) => {
  req.setEncoding('utf8');
  req.rawBody = '';
  req.on('data', (chunk) => {
    req.rawBody += chunk;
  });
  req.on('end', () => {
    next();
  });
}

app.use(parseRawBody);

app.post('/test', (req, res) => {
  res.send(req.rawBody);
});

app.listen(3000);

We created our own parseRawBody middleware that takes the request, response objects and the next function.

req is the request, which is a read stream.

We can get the data from it by listening to the data event.

In the handler for the data event, we concatenate the chunks of text to the req.rawBody property.

Then when the end event is emitted, then the stream has no more data to emit, so we call next to move to the next middleware.

Then we can access req.rawBody in the routes that come after it.

If we use the body-parser package, then we can use the bodyParser.text() middleware to keep the request body as plain text.

For instance, we can write:

app.use(bodyParser.text());

Then we can access the request body by writing:

app.get('/', (req, res) => {
  res.send(req.body);
})

req.body has the string with the request body.

Sending Message to a Specific Client with Socket.io and Empty Message Queue

To send a message to a specific client and empty the message queue with socket.io, we can listen to the connection event and then set the socket object as the property of an object.

For instance, we can write:

const socketio = require('socket.io');
const clients = {};
const io = socketio.listen(app);

io.sockets.on('connection', (socket) => {
  clients[socket.id] = socket;
});

The property name is the socket’s ID, indicated by the socket.id property.

We saved each client’s socket so that we can call emit to send a message to the client later.

Then we can send a message by writing:

const socket = clients[socketId];
socket.emit('show', {});

Where socketId is the ID of the socket that we set earlier.

We call emit with our own event and data we want to send.

Modifying Express Request Object

To modify the Express’s request object, we can assign our properties in the request object.

For instance, we can write:

app.use((req, res, next) => {
  req.root = `${req.protocol}://${req.get('host')}`;
  next();
});

We set the req.root property to the protocol and host of the URL combined.

The callback is a middleware that we can use anywhere.

next is a function that calls the next middleware in the chain.

Find the Size of the File in Node.js

We can find the size of a file with stateSync .

For instance, we can write:

function getFileSize(filename) {
  const stats = fs.statSync(filename);
  const fileSizeInBytes = stats.size;
  return fileSizeInBytes;
}

We call statsSync on the filename parameter, which is the string path to a file.

Then we can get the size in bytes with the size property.

We can also use the filesize package to get the file size.

For instance, we can write:

const filesize = require("filesize");
const stats = fs.statSync("foo.txt");
const fileSizeInMb = filesize(stats.size, {round: 0});

The benefit of this package is that we can conveyor the size to whatever format we like.

For instance, we can change the base, convert it to an object, and much more.

The example above converted the size to the file size in megabytes.

We can do other conversions by writing:

const filesize = require("filesize");
const stats = fs.statSync("foo.txt");
const fileSizeMb = filesize(stats.size, {round: 0});
const fileSizeArray = filesize(stats.size, {output: "array"});
const fileSizeObj = filesize(stats.size, {output: "object"});

The array has the size and unit as separate entries of the array.

object puts the size, union, exponent in their own properties.

Conclusion

We can keep the request body text in various ways.

There are libraries to help us get the file size in the format we want.

We can send messages to a specific client with Socket.io.