Category: Node.js Tips

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Share Constants in NodeJS Modules

We can share constants in Node modules by exporting a frozen object.

For instance, we can write:

constants.js

module.exports = Object.freeze({
  FOO: 'foo',
  BAR: '123'
});

We call the Object.freeze method to freeze the object with the constants.

Then in another module, we can require by writing:

const constants = require('./constants');

Send Responses to All Clients Except Sender with Socket.io

There are multiple ways to send a response to all clients except the sender with Socket.io.

Yo send to the sender only, we can write:

socket.emit('message', "hello");

To send to all clients except the sender, we can write:

socket.broadcast.emit('message', "hello");

To send to all clients in a room except for the sender, we can write:

socket.broadcast.to('chatRoom').emit('message', 'hello');

To send to all clients including the sender, we write:

io.emit('message', "hello");

To send to all clients in a room including the sender, we can write:

io.in('chatRoom').emit('message', 'hello');

To send to the sender only in a room, we can write:

`socket.to('`chatRoom`').emit('message', 'hello');`

To send to all clients in a namespace, we can write:

io.of('namespace').emit('message', 'hello');

To send to a socket ID, we can write:

socket.broadcast.to(socketId).emit('message', 'hello');

Reasons to use Promise Libraries like Q or BlueBird

We can use promise libraries like Q or Bluebird to convert existing async functions to return promises.

For instance, we can write:

const Promise = require('bluebird');
const fs = Promise.promisifyAll(require('fs'));

fs.readFileAsync('foo.text').then((data) => {
   //...
});

We used Bluebird to convert everything in the fs module to promises.

So we can use methods like readFileAsync to read files asynchronously.

There are also methods exclusive to Bluebird that aren’t available with the ES6 Promise constructor.

Promise.promisify() converts Node callbacks to promises.

Promise.map() maps arrays to promises.

Promise.reduce() lets us map values to promises and invoke them sequentially and reduce the resolved values to a single value.

Promise.mapSeries() takes values, map them to promises, and run them in series.

Then we get the values of them at the end.

Promise.delay() lets us dealt a promise by a given number of milliseconds.

Node.js Global Variables

We can add a property to global to add a global variable.

For instance, we can write:

global._ = `require('underscore');`

Node.js Get File Extension

We can get the file extension of a file name with the extname method.

For instance, we can write:

const path = require('path');

path.extname('index.html');

Run Function in Script from Command Line

We can run a function in a script from the command line with node -e .

If we have a module, we can write:

hi.js

module.exports.hello = () => {
  console.log('hello');
};

For instance, we can write:

node -e 'require("./hi").hello()'

Difference Between res.send and res.json in Express.js

res.send and res.json are the same when arrays or objects are passed in.

However, if non-objects are passed in, then res.json also converts those values to something that can be returned.

res.json calls res.send in the end.

Passing Variables to the Next Middleware Using next() in Express.js

We add a property to the req variable to pass data to the next middleware.

For instance, we can write:

req.foo = 'bar';

Then we can access req.foo in the next middleware.

Also, we can add a property to the res.locals property to do the same thing.

Use Node.js Crypto to Create an HMAC-SHA1 Hash

We can use the Node crypto module to create an HMAC-SHA1 hash.

For instance, we can write:

const crypto = require('crypto')

const text = 'hello world';
const key = 'secret';

crypto.createHmac('sha1', key)
  .update(text)
  .digest('hex')

We call the createHmac method to create a hash from the key .

And then we hash the text with it with update and convert it to hex with update .

Conclusion

We can use the crypto module to hash text.

There are many ways to send data with socket.io.

We can get the extension of a file name with the path module.

Promise libraries are still useful.

global is the global object in Node.

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Pass Variable from Jade Template File to a Script File

To pass a variable in a Jade template to a script, we can interpolate the script code.

For instance, we can write:

app.get('/', (req, res) => {
  res.render( 'index', { layout: false, name: 'bar' } );
});

index.jade

script.
  name="#{name}";

We use the script tag and interpolated the name variable in index.jad .

Mongoose Select a Specific Field with find

We can select a specific field with find by passing the field name as the argument.

For instance, we can write:

app.get('/name', (req, res, next) => {
  dbSchemas.Foo.find({}, 'name', (err, val) => {
    if(err) {
      return next(err);
    }
    res.send(val);
  });
});

We select a Foo document’s name field by passing the field name as the 2nd argument.

Difference Between async.waterfall and async.series

async.waterfall lets us pas results to the next function.

async.series pass all results to the final callback.

Write Asynchronous Functions for Node.js

We write callbacks that have the err for the error object and the result aS parameters.

For instance, we can write:

file.read('/foo.txt', (err, result) => {
  if (err) {
    return console.log(err);
  }
  doSomething(result);
});

The arguments before the callback is anything we want to use with read .

And the callback is a Node style callback with the err object as the first parameter.

result has the computed results in the 2nd parameter.

Configuring Region in Node.js AWS SDK

We can configure the AWS region with the AWS.config.update method.

For example, we can write:

const AWS = require('aws-sdk');
AWS.config.update({ region: 'us-east-1' });

to configure the region.

Pass Parameter to a Promise Function

To pass parameters to a promise function, we create a function that takes the parameter we want and returns the promise inside it.

For instance, we can write:

const login = (username, password) => {
  return new Promise((resolve, reject) => {
    //...
    if (success) {
      resolve("success");
    } else {
      reject(new Error("error"));
    }
  });
}

We return the promise by using the Promise constructor.

username and password are the parameters we passed in.

Then we can use it by writing:

login(username, password)
  .then(result => {
    console.log(result);
  })

We use then to get the value we passed into resolve .

Shell Command Execution with Node.js

To run shell commands in Node apps, we can use the child_process module.

For instance, we can write:

const spawn = require('child_process').spawn;
const child = spawn('ls', ['-l']);
let resp = "";

child.stdout.on('data', (buffer) => {
  resp += buffer.toString()
});

child.stdout.on('end', () => {
  console.log(resp)
});

We use the child_process ‘s spawn method.

Then we pass in the command as the first argument to spawn.

And the command line arguments as the 2nd argument.

Then we can watch for the results with the child.stdout.on method.

We listen to the 'data' event to get the output and concatenate the parts that are sent.

And we listen to 'end' to get the whole result when it ends.

Monitor Memory Usage of Node.js Apps

We can use the node-memwatch package to watch for memory usage of a Node app.

We can install it by running:

npm install memwatch

Then we can use it by writing:

const memwatch = require('memwatch');
memwatch.on('leak', (info) => {
  //...
});

We watch for memory leaks with the on method.

info has information about the leak.

It has something like:

{
  start: Fri, 29 Jun 2020 15:12:13 GMT,
  end: Fri, 29 Jun 2020 16:12:33 GMT,
  growth: 67984,
  reason: 'heap growth over 5 consecutive GCs (20s) - 11.67 mb/hr'
}

It’ll show us the growth in memory usage.

There also methods for watching heap usage and diffing memory usage in different times.

The built-in process module has the memoryUsage method to get the memory usage of our app.

For instance, we can write:

process.memoryUsage();

to get the memory usage.

Conclusion

We can watch for memory usage with the process module or 3rd party modules.

To pass data to a script tag in a Jade template, we can interpolate the code.

We can return a promise in a function if we want a promise with our own parameters.

Mongoose can be used to find data with a given field.

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Render Error When Page Not Found

We can render an error message with res.render .

For instance, we can write:

app.use((req, res, next) => {
  res.render('404', { status: 404, url: req.url });
});

to render a 404 error.

'404' is the template name.

the object has the stuff we want to display on the 404 template.

We can also make a generic error handling middleware by writing;

app.use((err, req, res, next) => {
  res.render('500', {
    status: err.status || 500,
    error: err
  });
});

We have err as the first parameter.

We can get the error status code with err.status and we can pass the object off to the 500 page template.

Writing Files in Node.js

We can write files with Node.js with the writeFile , writeFileSync , or a write stream.

For instance, we can write:

const fs = require('fs');

fs.writeFile("/tmp/foo.txt", "hello world", (err) => {
  if (err) {
    return console.log(err);
  }
  console.log("The file was saved!");
});

Or we can writ:

const fs = require('fs');

fs.writeFileSync('/tmp/foo.txt', 'hello world');

They both take the file path and the content as the first 2 arguments.

writeFile also takes a callback that’s called with err if it exists.

To create a write stream, we can write:

const fs = require('fs');
const  stream = fs.createWriteStream("/tmp/foo.txt");
stream.once('open', (fd) => {
  stream.write("hellon");
  stream.write("worldn");
  stream.end();
});

We call createWriteStream to with the file path to write to.

Then we open the stream with stream.once .

Then we start writing with stream.write .

We’ve to close the stream with stream.end .

Get Query String Variables in Express

We can get query string variables with the req.query property.

It has the query string parsed into an object of the key-value pairs.

For instance, we can write:

const express = require('express');
const app = express();

app.get('/', (req, res) => {
  res.send(req.query.id);
});

app.listen(3000);

If we have a query string like ?id=10 , then req.query.id is 10.

Retrieve POST Query Parameters in an Express App

We can get POST query parameters with the body-parser package.

To use it, we’ve to install it by running:

npm install --save body-parser

Then we can write:

const bodyParser = require('body-parser');
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({
  extended: true
}));

We import the module so that we can use it.

Then we call bodyParser.json() which returns a middleware to use the middleware.

urlencoded parses URL encoded bodies.

extended set to true means that we can parse JSON bodies in the URL encoded string with it.

Adding CORS Headers to an OPTIONS Route

We can do all the CORDS stuff easily with our own middleware.

For instance, we can write:

const allowCrossDomain = (req, res, next) => {
  res.header('Access-Control-Allow-Origin', 'example.com');
  res.header('Access-Control-Allow-Methods', 'GET,PUT,POST,DELETE');
  res.header('Access-Control-Allow-Headers', 'Content-Type');
  next();
}

The Access-Control-Allow-Origin header lets the browser know what domains can access the API.

Access-Control-Allow-Methods tells the browser what kind of request it’s allowed to be called.

Access-Control-Allow-Headers tells the browser what kind of content it can send.

An easier way is to use the cors middleware.

To use it, we can run:

npm install cors --save

to install it.

Then we can write:

const cors = require('cors');
const express = require('express');
const app = express();
app.use(cors());
app.options('*', cors());

We use the cors middleware on the whole app with:

app.use(cors());

and we allow the OPTIONs route to be called with:

app.options('*', cors());

How to Get the Full URL in Express

We can use the req.protocol property to get the protocol, which is the http or https part.

Also, we can get the hostname with req.get('host') .

And we get the full URL with the req.originalUrl .

To parse combine the parts into the full URL, we can use the url module.

For instance, we can write:

const requrl = url.format({
  protocol: req.protocol,
  host: req.get('host'),
  pathname: req.originalUrl,
});

We can do that inside the route handler or anywhere else.

Conclusion

We can render an error page by passing in a template and get the status or error we want to display.

The url module can combine the URL parts together.

There are many ways to write files.

CORS can be done by setting a few response headers or using the cors middleware with Express.

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Avoid the async/await Inside of a new Promise() Constructor Antipattern

We should avoid the asyhc and await inside of the Promise constructor antipattern.

For instance, instead of writing:

const createPromise = () => {
  return new Promise(async (resolve, reject) => {
    const val = await Promise.resolve(100);
    //..
  });
}

We don’t have to have an async function inside a promise because we shouldn’t have promises inside the Promise constructor.

async functions already return promises, so we don’t need the Promise constructor to turn the code into a promise.

We just move the async function out of the Promise and use that as is.

Update All Clients Using Socket.io

We can update all clients with socket.io by using the emit method.

For instance, we can write:

socket.emit('message', 'hello');

We emit the message event to all clients with the content 'hello' .

We can also use the io.sockets.emit method to emit to all sockets.

For instance, we can write:

io.sockets.emit('`message`', 'hello');

And we can use the broadcast property to do the same thing.

It sends a message to everyone except for the socket that’s sending the event:

socket.broadcast.emit('`message`', 'hello');

Get Response from S3 getObject in Node.js

We can call the getObject method to get the item we want as follows:

const aws = require('aws-sdk');
const s3 = new aws.S3();

const getParams = {
  Bucket: 'abc',
  Key: 'abc.txt'
}

s3.getObject(getParams, (err, data) => {
  if (err) {
    return err;
  }

  const objectData = data.Body.toString('utf-8');
});

We use the aws.S3 constructor to create a new client.

Then we set the parameters for setting the bucket and key, which is the file path.

Next, we call getObject to get the file with the given path in that bucket.

err is the error object.

data is the content of the file.

We convert it to a string with toString and the correct encoding.

Properly Close the Express Server

We call close on the http server instance rather than the app instance.

For instance, we can write:

app.get('/foo', (req, res) => {
  res.redirect('/');
  setTimeout(() => {
    server.close();
  }, 3000)
});

const server = app.listen(3000);

We call close on server , which is the http server instance.

app.listen returns the http server instance.

Calling a JSON API with Node.js

We can use the http module’s get method to make a GET request to a server.

For instance, we can write:

const url = 'https://api.agify.io/?name=michael';

http.get(url, (res) => {
  let body = '';
  res.on('data', (chunk) => {
    body += chunk;
  });

  res.on('end', () => {
    const response = JSON.parse(body);
    console.log(response.name);
  });
})
.on('error', (e) => {
  console.log(e);
});

We called get by passing in a callback.

The callback listens to the data event, which gets the data.

We listen to the end event to parse the JSON when the stream is done emitting data.

In the end callback, we parse the JSON with JSON.parse .

We also listen to the error event to log any errors if they exist.

Selecting Fields for Documents Populated in MongoDB

We can call the populate method to populate the field that we want and return it.

For instance, we can write:

Model
.findOne({ _id: '...' })
.populate('age', 'name')

We get the item by the _id , populate the age and return the name .

Check if a Function is async

We can check if a function is async by using the Symbol.toStringTag property.

For instance, we can write:

asyncFn[Symbol.toStringTag] === 'AsyncFunction'

We can also check if it’s an instance of the AsyncFunction with instanceof .

For instance, we can write:

const AsyncFunction = (async () => {}).constructor;
const isAsyncFunc = asyncFn instanceof AsyncFunction;

We get the constructor of the async function with the constructor property.

Then we use the instanceof operator to do the check.

Conclusion

We shouldn’t put async functions in the Promise constructor.

We can read a file from the S3 bucket with the getObject method.

Also, we can make HTTP requests with the http module.

We can call close on the http server instance returned by app.listen .

There are a few ways to check if a function is async.

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Node.JS async.parallel

The async module is the parallel method to let us run async code in parallel.

For instance, we can write:

async.parallel({
  one(callback) {
    callback(null, 'foo');
  },
  two(callback) {
    callback(null, 'bar');
  }
}, (err, results) => {
  //...
});

We have 2 methods that take a callback in the object that we pass into async.parallel .

They both call callback with the error and result objects as the arguments.

Then we get both in the results object.

results would be:

{ one: 'foo', two: 'bar' }

The method names are used as the keys of the results.

Reading XML File in Node.js

We can read XML files in Node.js with the readFile method.

Then we can parse the content with the xml2json library.

For instance, we can write:

fs = require('fs');
const parser = require('xml2json');

fs.readFile('./data.xml', (err, data) => {
  const json = parser.toJson(data);
  console.log(json);
});

We call readFile with the path of the XML file.

Then we get the XML text with data .

Next we call parser.toJson to parse the XML string stored in data .

Node MySQL Escape LIKE Statement

We can escape the characters in a LIKE statement by writing:

mysql.format("SELECT * FROM persons WHERE name LIKE CONCAT('%', ?,  '%')", searchString)

We just concatenate the % and our searchString together.

Delete Files Older than an Hour

To get all the file older than an hour and delete them, we can use readdir to get the files in the directory.

Then we can use fs.stat to get the time and we can use that to compare with the current time to see if an hour has passed since the file has been created.

When the file is created an hour ago or earlier, we can use rimraf to delete it.

For instance, we can write:

const uploadsDir = path.join( __dirname, '/uploads');

fs.readdir(uploadsDir, (err, files) => {
  files.forEach((file, index) => {
    fs.stat(path.join(uploadsDir, file), (err, stat) => {
      let endTime, now;
      if (err) {
        return console.error(err);
      }
      now = new Date().getTime();
      endTime = new Date(stat.ctime).getTime() + 3600000;
      if (now > endTime) {
        return rimraf(path.join(uploadsDir, file), (err) => {
          if (err) {
            return console.error(err);
          }
          console.log('successfully deleted');
        });
      }
    });
  });
});

We read the folder with readdir .

Then we loop the files obtained in the callback.

Then we call fs.stat to get the file information.

We use the ctime property to get the time when the file is created.

Then we use getTime to turn it into a timestamp.

And we add 3600000 which is an hour in milliseconds.

Then if now > endTime is true , we know that more than an hour has passed since the file is created.

Then we can use rimraf to remove the file.

We use the full path.

Calling a Web Service using Node.js

We can call a web server in a Node app by making HTTP requests as we do on the client-side.

For instance, we can write:

const http = require('http');
const data = JSON.stringify({
  'id': '2'
});

const options = {
  host: 'host.com',
  port: '80',
  path: '/some/url',
  method: 'POST',
  headers: {
    'Content-Type': 'application/json;',
    'Content-Length': data.length
  }
};

const req = http.request(options, (res) => {
  let msg = '';

  res.setEncoding('utf8');
  res.on('data', (chunk) => {
    msg += chunk;
  });
  res.on('end', () => {
    console.log(JSON.parse(msg));
  });
});

req.write(data);
req.end();

We use the http.request method to make a request.

We call req.write to make the request with the body

The options object has the headers, hostname, and request method.

Once the request is finished, the callback is called and res has the stream with the response.

We listen to the data event to get the chunks of data and concatenated to the msg string.

Then we listen to the end event to parse the chunks that are retrieved.

Conclusion

async.parallel can run callbacks in parallel.

We can delete files with rimraf.

Also, we can make HTTP requests with the http module.

To parse XML to JSON we can use the xml2json package.