Category: Node.js Tips

As with any kind of app, there are difficult issues to solve when we write Node apps. In this article, we’ll look at some solutions to common problems when writing Node apps.

Wait for an Element to be Visible with Puppeteer

To wait for an element to be visible, we can use the waitForSelector method.

For example, we can write:

page.waitForSelector('#myId', { visible: true })

This waits for the element with the ID myId to be in the DOM.

visible is true ensures that it’s visible before waiting stops.

Minimize or Close Window to the System Tray and Restore Window Back from Tray

To minimize the window to the system tray, we can write:

const BrowserWindow = require('browser-window'),
`
const mainWindow = new BrowserWindow({
  width: 650,
  height: 650,
  title: "window",
  icon:'./icon.png'
});
`
mainWindow.on('minimize',(event) => {
  event.preventDefault();
  mainWindow.hide();
});
`
mainWindow.on('close', (event) => {
  if(!application.isQuiting){
    event.preventDefault();
    mainWindow.hide();
  }
  return false;
});

We call event.preventDefault to prevent the default minimization action when we listen to the minimize event. Then we call hide to hide it in the tray.

We do the same thing when we listen to the close event. If the app isn’t quitting, then we call preventDefault to prevent it from quitting. Then we call mainWindow.hide() to hide it instead. To restore from the tray, we add new menu entries that we can click to restore the window when we click.

For example, we write:

const contextMenu = Menu.buildFromTemplate([
  {
    label: 'restore app',
    click(){
      mainWindow.show();
    }
  },
  {
    label: 'exit',
    click(){
      application.isQuiting = true;
      application.quit();
    }
  }
]);

We call buildFromTemplate with an array to create a context menu.

It has one entry to call mainWindow.show() to show the mainWindow .

And we have an entry that we click to exit the app with application.quit() .

OAuth Authentication with the Twitter API and Express

We can add Twitter authentication to our Express app by using the oauth library with Express. First, we fill in the callback URL with the URL that we want to invoke when authentication is done. It should be set to /session/callback if the example below is used.

Then we write:

const express = require('express');
const oauth = require('oauth');
const session = require('express-session');
const cookieParser = require('cookie-parser');
const app = express.createServer();
`
const twitterConsumerKey = "1";
const twitterConsumerSecret = "2";
const consumer = new oauth.OAuth(
  "https://twitter.com/oauth/request_token",
  "https://twitter.com/oauth/access_token",
  twitterConsumerKey,
  twitterConsumerSecret,
  "1.0A",
  "http://127.0.0.1:8080/sessions/callback",
  "HMAC-SHA1"
);
`
app.use(cookieParser());
app.use(session({
  secret: "very secret"
}));
`
app.use((req, res, next) => {
  res.locals.user = req.session.user;
  next();
});
`
app.get('/sessions/connect', (req, res) =>{
  consumer.getOAuthRequestToken((error, oauthToken, oauthTokenSecret, results) => {
    if (error) {
      res.send(error, 500);
    } else {
      req.session.oauthRequestToken = oauthToken;
      req.session.oauthRequestTokenSecret = oauthTokenSecret;
      res.redirect(`https://twitter.com/oauth/authorize?oauth_token=${req.session.oauthRequestToken}`);
    }
  });
});
`
app.get('/sessions/callback', (req, res) => {
  consumer.getOAuthAccessToken(
    req.session.oauthRequestToken,
    req.session.oauthRequestTokenSecret,
    req.query.oauth_verifier,
    (error, oauthAccessToken, oauthAccessTokenSecret, results) => {
      if (error) {
        res.send('Error', 500);
      } else {
        req.session.oauthAccessToken = oauthAccessToken;
        req.session.oauthAccessTokenSecret = oauthAccessTokenSecret;
        res.redirect('/home');
      }
    });
});
`
app.get('/home', (req, res) => {
  consumer.get(
    "http://twitter.com/account/verify_credentials.json",
    req.session.oauthAccessToken,
    req.session.oauthAccessTokenSecret,
    (error, data, response) => {
      if (error) {
        res.redirect('/sessions/connect');
      } else {
        const parsedData = JSON.parse(data);
        res.send(parsedData.screen_name);
      }
    });
});
`
app.get('*', (req, res) => {
  res.redirect('/home');
});
`
app.listen(8080);

We use the oauth package’s oauth.OAuth constructor to set the keys and secrets required for Twitter OAuth. Those can be gotten from the Twitter developer website. The /sessions/connect route is where we go to log in with Twitter. We call the getOAuthRequestToken to get the OAuth request token. The callback in that method redirects us to the Twitter auth page.

Once we log in, then we’re redirected to the /session/callback route if we set the callback URL to that. It sets the OAuth access token secret and the access token so that we can authenticate with the Twitter API.

Then we when go to the /home route, we authenticate with the access token that we obtained since they’re set as the properties of req.session in the /session/callback route. We used Express session and Cookie Parser to manage the session.

Conclusion

We can do OAuth with Twitter by using the oauth package. Also, we can wait for elements to appear with Puppeteer before proceeding. And we can minimize a window to the system tray and restore it.

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Remove Event Listeners in Node.js Event Emitter

We can remove listeners with Node event emitter by using the removeListener method.

For instance, we can write:

const events = require('events');
const EventEmitter = events.EventEmitter;
const rr = new EventEmitter();
const refreshHandler = () => {
  console.log('refresh');
}
rr.on("refresh", refreshHandler);
rr.removeListener("refresh", refreshHandler);

We created our EventEmitter instance.

Then we attach the refreshHandler event listener to the refresh event.

If we don’t need to listen to the refresh event anymore, then we can call removeListener to clear the event listener from memory.

We pass in the event name as the first argument of removeListener .

And the 2nd is the event handler we want to remove.

The Meaning of the “_” (underscore) Symbol in Node.js REPL

The _ symbol returns the result of the last logged expression in the REPL,

For instance, if we typed:

> 2 * 3
6

Then _ returns 6:

> _
6

Access Variable from an exec Callback Function in Node.js

exec takes a callback function with the result of running a shell command.

For instance, we can write:

const execChild = (callback) => {
  const child = exec(cmd, (error, stdout, stderr) => {
    callback(stdout);
  });
}

We get the stdout from the callback, which has the results of our cmd command.

Then we call callback to get the result outside of the exec callback.

We’ve to do this because exec is async, so we won’t know when we’ll get the result.

Now we can use the execChild function by writing:

execChild((result) => console.log(result));

The stdout that we passed into the callback would be the result .

Socket.io —Listen for Connection Event

We can listen for connections on server side.

And we then make connections from the client-side.

On our server-side code, we write:

const io = require('socket.io').listen(8001);
io.sockets.on('connection', (socket) => {
  console.log('user connected');
});

to listen to the connection event, which will be emitted from the client if we call io.connect .

Then in our client-side code, we write:

const socket = io.connect('http://localhost:8001');

to connect to the server we created.

Get a Full File Path in Node.js

We can get the __dirname to get the full path of the current directory.

Then w can write:

const path = require("path");
const fileName = "file.txt";
const fullPath = path.join(__dirname, "/uploads/", fileName);

We use path.join to join all the path segments together in a platform-agnostic way.

Also, we can use the path.resolve method to return the full path to a file.

For instance, we can write:

const path = require("path");
const fullPath = path.resolve("./uploads/file.csv");

We call resolve with the relative path to get the full path of the file.

Get Client’s IP with Express

If we’re writing an Express app, then we can use req.ip to get the client’s IP address.

We can also get the value of the x-forward-for header by writing:

req.headers['x-forwarded-for']

to get the header’s value, which has the IP address.

We can also check req.connection.remoteAddress to do the same thing.

Connect to Socket.Io Server with Specific Path and Namespace

We can connect to a socket.io server with a specific path and namespace by using the of method and the path property.

For instance, we can write:

const io = require('socket.io')(http, {
  path: '/foo/bar'
});

io
  .of('/namespace')
  .on('connection', (socket) => {
    socket.on('message', (data) => {
      io.of('namespace').emit('message', data);
    });
  });

We pass in the http server instance to the socket.io function.

The 2nd argument lets us specify the path with the path property.

Then in the io.of method, we specify the namespace.

The on method lets us watch for connections from clients.

We can listen to events from the client with socket.on .

The first argument is the event name.

The 2nd is a callback that has the data emitted with the event.

Conclusion

We can use removeListener to clear event listeners.

We can connect to a specific path and namespace with socket.io.

It’s easy to get various file and network information with Node.

We can use _ to get the last result that’s returned in the Node REPL.

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Call an Async Function within map

We can use Promise.all to map functions to async functions.

For instance, we can write:

const asyncMap = async (teachers) => {
  const data = await Promise.all(teachers.map(async teacher => {
    return {
      ...teacher,
      image: `${teacher.name}.jpg`
   }
  }));
  //...
}

We map the teachers array to an array of promises with map .

An async function returns a promise, so we can just add the async keyword in front of the callback for map .

Then we can use await and Promise.all to invoke all the promises in parallel.

And we can do whatever we want with the results afterward.

Watch Directory for Changes with Nodemon

We can use the --watch option to watch multiple files and directories.

For example, we can run:

nodemon --watch src app.js

to watch the src folder and app.js .

Copy All Objects in Amazon S3 from One prefix to Another Using the AWS SDK for Node.js

To copy objects from one key to another, we can use the listObjects to list the entries.

Then we can use copyObject to copy the entries to the key location.

For example, we can write:

const AWS = require('aws-sdk');
const async = require('async');
const bucketName = 'foo';
const oldPrefix = 'bar/';
const newPrefix = 'baz/';
const s3 = new AWS.S3({ params: { Bucket: bucketName }, region: 'us-west-2' });

const done = (err, data) => {
  if (err) { console.log(err); }
  else { console.log(data); }
};

s3.listObjects({ Prefix: oldPrefix }, (err, data) => {
  if (data.Contents.length) {
    async.each(data.Contents, (file, cb) => {
      const params = {
        Bucket: bucketName,
        CopySource: `${bucketName}/${file.Key}`,
        Key: file.Key.replace(oldPrefix, newPrefix)
      };
      s3.copyObject(params, (copyErr, copyData) => {
        if (copyErr) {
          console.log(copyErr);
        }
        else {
          console.log(params.Key);
          cb();
        }
      });
    }, done);
  }
});

We call listObjects with the oldPrefix to get the items in the lol path.

The entries are stored in data.Contents .

Then we use async.each to iterate through each object.

Bucket has the bucket name.

CopySource has the original path.

Key has the new key, which we call replace to with the oldPrefix and newPrefix to replace it with the oldPrefix with newPrefix .

We pass the params object to the copyObject method.

Then we get the results afterward.

Once we did that, we call cb to complete the iteration.

We call done which we defined earlier to print any errors or results.

Sequential MongoDB Query in Node.js

We can use async and await to make MongoDB queries in a synchronous style.

It looks synchronous in that it’s sequential, but it’s asynchronous.

For instance, we can write:

const getPerson = async () => {
  const db = await mongodb.MongoClient.connect('mongodb://server/db');
  if (await db.authenticate("username", "password")) {
    const person = await db.collection("Person").findOne({ name: "james" });
    await db.close();
    return person;
  }
}

return returns a promise that resolves to person .

It doesn’t actually return person .

db.collection gets the collection we want to query.

findOne finds one entry that matches the condition in the object.

We look for an entry with name value 'james' .

The resolved value of the query is set to person .

Then we call db.close when we’re done the query.

Then we return person to resolve the returned promise to that value.

Automatically Add the Header to Every “render” Response in an Express App

We can make our own middleware to add a header to every response.

For instance, we can write:

app.use((req, res, next) => {
  res.header('foo', 'bar');
  next();
});

With res.header , we add the 'foo' response header with value 'bar' to all responses.

Then we call next to run the next middleware.

This piece of code should be added before the routes so that it runs before each route.

Conclusion

We can call map to map each entry to a promise.

Then we call Promise.all to invoke them in parallel.

We can use the AWS SDK to copy files in S3.

MongoDB methods support promises.

We can add response headers to all routes with a middleware.

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

MongoDB Driver async/await Queries

The MongoDB Node drive supports promises, so we can use async and await in our queries.

For instance, we can write:

const MongoClient = require('mongodb').MongoClient;
const connectionString = 'mongodb://localhost:27017';

(async () => {
  const client = await MongoClient.connect(
    connectionString, {
      useNewUrlParser: true
    }
  );
  const db = client.db('dbName');
  try {
    const res = await db.collection("collectionName")
      .updateOne({
        "foo": "bar"
      }, {
        $set: {}
      }, {
        upsert: true
      });
    console.log(res);
  } finally {
    client.close();
  }
})()

We call MongoClient.connect with the connectionString .

Then we get the database to manipulate with client.db .

We then use the db.collection method to get the collection to query.

Then we call updateOne to update the entry with the given field with the value.

Pressing Enter Button in Puppeteer

We can press the Enter button with Puppeteer by writing:

await page.type(String.fromCharCode(13));

13 is the key code for the Enter key.

Also, we can use the press method on an element.

For example, we can write:

await (await page.$('input[type="text"]')).press('Enter');

We select an input with type text and call press with 'Enter' as the argument.

'\n' and '\r' are also alternatives for 'Enter' , so we can write:

await page.keyboard.press('n');

or:

await page.keyboard.press('r');

To press the Numpad Enter key, we can write:

await (await page.$('input[type="text"]')).press('NumpadEnter');

We select the input with type text and call press with 'NumpadEnter' .

There’s also the sendCharacter method:

await page.keyboard.sendCharacter(String.fromCharCode(13));

Check for undefined Property in EJS for Node.js

We can use the typeof operator to check for an undefined property.

For instance, we can write:

const template = '<% (typeof foo !== "undefined" ? %>foo defined<% : %>foo undefined<% ) %>';

We just use the typeof operator just like in JavaScript.

Then we show foo defined if it’s defined.

Otherwise, we show foo undefined .

Listen to All Interfaces Instead of Localhost Only with Express

We can specify the IP address with app.listen to listen to.

To listen to all address, we can pass in '0.0.0.0' .

For example, we can write:

const express = require('express');
const app = express();

//...
app.listen(3000, '0.0.0.0');

How to Find the Size of the File in Node.js

To find the size of a file, we can use the statSync method.

For instance, we can write:

const fs = require("fs");
const stats = fs.statSync("foo.txt");
const fileSizeInBytes = stats.size;

We call statSync with the file path to get data about the file.

Then we use the size property to get the file size in bytes.

List Available Crypto Algorithms

We can use the crypto module to get the list of algorithms.

To get them, we use the getCiphers to get available ciphers.

And we can also use getHashes to get the available hashes.

So we can write:

const crypto = require('crypto');
console.log(crypto.getCiphers());
console.log(crypto.getHashes());

Set _id to DB Document with Mongoose

To let us set the _id of the database document with Mongoose, we can put the _id property in the Schema.

Or we can ser the _id option to false .

For instance, we can write:

const Person = new mongoose.Schema({
  _id: Number,
  name: String
});

or:

const Person = new mongoose.Schema({
  name: String
}, { _id: false });

All a Where Clause Conditionally to a Knex Query

To add a where clause conditionally to a Knex query, we can store the general query in a variable.

Then we can write:

const query = knex('persons')
  .select('name', 'james')
  .limit(50);

if (age) {
  query.where('age', age);
}

We add the where clause for age only if age is defined.

Or, we can use the modify method.

For instance, we can write:

knex('persons')
  .select('name', 'james')
  .limit(50);
  .modify((queryBuilder) => {
    if (age) {
      queryBuilder.where('age', age);
    }
  });

We make our age query in the callback instead of storing the query in a variable.

Conclusion

The MongoDB Node driver supports promises.

We can press Enter with Puppeteer in many ways.

We can make Knex queries conditionally.

Also, we can listen to all IP addresses with Express apps.

_id can be set if we specify it in the Mongoose schema.

As with any kind of app, there are difficult issues to solve when we write Node apps. In this article, we’ll look at some solutions to common problems when writing Node apps.

Route All Requests to index.html with Express

We can route all requests to index.html in an Express app by using the * wildcard to listen to all requests.

For instance, we can write:

const express = require('express');
const path= require('path');
const server = express();

server.use('/public', express.static(path.join(__dirname, '/assets')))

server.get('/hello', (req, res) => {
  res.send('hello');
})

server.get('/*', (req, res) => {
  res.sendFile(path.join(__dirname + '/index.html'));
})

const port = 8000;
server.listen(port);

We have an Express app that serves the /assets folder with the /public route. Then we define a GET route that’s not a catch-all route to listen to GET requests with that path. Then we define our catch-all route to render index.html . Anything that doesn’t match /public or /hello rendersindex.html .

Re-throwing Exception and not Losing Stack Trace

We can rethrow an exception and not lose the stack trace by rethrowing the error object in the catch clause.

For instance, we can write:

try {
  //...
  throw new Error("error");
  //...
} catch(err) {
  err.message = `error: ${err.message}`;
  throw err;
}

We run something that throws an error in the try block.

Then in the catch block, we change the err.message as we want to and keep the stack trace.

Then we throw the err object again after it’s modified.

Read JSON File Content with require vs fs.readFile

require reads the JSON file content synchronously.

The file is parsed without any extra work.

For instance, if we have:

const obj = require('./someJson');

then we read the someJson.json file and assign the returned value to obj .

We don’t need the .json extension and it’s parsed without any code.

If we use readFile , then we read the JSON file asynchronously.

Also, we’ve to parse the text with JSON.parse to convert it to an object.

For example, we can write:

const fs = require('fs');

fs.readFile('/path/test.json', 'utf8', (err, data) => {
  if (err) {
    return console.error(err);
  }
  const obj = JSON.parse(data);
});

We pass in the path as the first argument.

The encoding is the 2nd argument.

The callback is the 3rd argument.

data has the data, which is in a string.

We used JSON.parse to parse into an object.

We can also do the same thing synchronously with readFileSync .

For instance, we can write:

const fs = require('fs');
const json = JSON.parse(fs.readFileSync('/path/test.json', 'utf8'));

We pass in the path and encoding and it’ll return the JSON string.

Then we call JSON.parse to parse the string.

We can use require to cache the object.

readFileSync is faster out of the 3 choices according to https://github.com/jehy/node-fastest-json-read/runs/416637981?check_suite_focus=true

readFile is the slowest but doesn’t block the event loop.

Basic Web Server with Node.js and Express for Serving HTML File and Assets

We can use the connect package to make a basic web serve to serve static files.

To install it, we run:

npm install connect

Then in our code file, we can write:

const connect = require('connect');
const port = 1234;

connect.createServer(connect.static(__dirname)).listen(port);

We call createServer to create the webserver.

connect.static is a middleware to let us serve the static files.

listen selects the port that we listen to requests from.

With Express, we can write:

const express = require('express');
const app = express();
const http = require('http');
const path = require('path');
const httpServer = http.Server(app);

app.use(express.static(path.join(__dirname, '/assets')));

app.get('/', (req, res) => {
  res.sendfile(path.join(__dirname, '/index.html'));
});
app.listen(3000);

We use the express.static middleware to serve static files.

Then we can access them from index.html , where we access the assets folder.

Get a List of Files in Chronological Order

We can sort a list of files that are returned from readdirSync by using the sort method.

For instance, we can write:

const fs = require('fs');
const path = require('path');
const dir = './';
const files = fs.readdirSync(dir);
files.sort((a, b) => {
  return fs.statSync(path.join(dir, a)).mtime.getTime() -   fs.statSync(path.join(dir, b).mtime.getTime();
});

In the sort callback, we use getTime to get the timestamp of when the files are modified.

We subtract them to sort in ascending order.

We can do the same thing with readFile in the callback.

Conclusion

We can serve static files and route requests to static pages. Files can be sorted once they’re read. Error objects can be modified and rethrown. JSON can be read with various methods.