Categories
Node.js Tips

Node.js Tips — Mail, Mocha Tests, S3 and JSON

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

‘UnhandledPromiseRejectionWarning: This error originated either by throwing inside of an async function without a catch block’ Error in Express Apps

We’ve to handle all promise errors manually in our Express apps.

For instance, we should write:

router.get("/fetch-email", authCheck, async (req, res, next) => {
  try {
    const emailFetch = await gmaiLHelper.getEmails(req.user.id, '/messages', req.user.accessToken)
    emailFetch = emailFetch.data
    res.send(emailFetch);
  } catch (err) {
    next(err);
  }
})

In our async route handler function, we have a catch block to catch errors.

Then we call next to call our error handling middleware.

The error handler middleware should be added after the routes so that next will call it.

Sort Sequelize.js Query by Date

We can sort a Sequelie query by date by adding an order property to the object we pass into findAll .

For instance, we can write:

Post.findAll({ limit: 50, order: '"updatedAt" DESC' })

Then we sort the updateAt column in descending order.

Sending an Email to Multiple Recipients with nodemailer

We can send an email to multiple recipients with nodemailer .

To do that, we can set the to property to an array.

For instance, we can write:

const mailList = [
  'foo@bar.com',
  'baz@bar.com',
  'qux@bar.com',
];

const msg = {
  from: "from@bar.com",
  subject: "Hello",
  text: "Hello",
  cc: "cc@bar.com",
  to: mailList
}

smtpTransport.sendMail(msg, (err) => {
  if (err) {
    return console.log(err);
  }
});

We just set to to our mailList array and send the message with sendMail .

Also, we can pass in a comma-separated string to do the same thing.

For instance, we can write:

const mailList = [
    'foo@bar.com',
    'baz@bar.com',
    'qux@bar.com',
  ]
  .join(',');
const msg = {
  from: "from@bar.com",
  subject: "Hello",
  text: "Hello",
  cc: "cc@bar.com",
  to: mailList
}
smtpTransport.sendMail(msg, (err) => {
  if (err) {
    return console.log(err);
  }
});

We called join to join the email address by commas.

Then the rest is the same.

Socket.io custom client ID

We can generate a custom ID for a client with socket.io.

We create our own generateId method to do so.

For instance, we can write:

const app = require('express')();
const http = require('http').Server(app);
const io = require('socket.io')(http);

io.engine.generateId = (req) => {
  return 1
}

io.on('connection', (socket) => {
  console.log(socket.id);
})

We add the generateId method, then we can access the returned result with socket.id .

We can replace 1 with our own auto-generated ID.

Expect Mocha to Fail a Test

We can expect Mocha to fail a test with assert.fail .

For instance, we can write:

it("should fail", () => {
  assert.fail("actual", "expected", "Error message");
});

We pass in the actual result, expected result, and error message if something isn’t expected.

Wait for All Images to load with Puppeteer Then Take Screenshot

We can call goto with the waitUntil option.

For instance, we can write:

await page.goto('https://www.example.com/', {"waitUntil" : "networkidle0" });

'networkidle0' means navigation is considered finished when there are no more than 0 network connections for at least 500 ms.

There’s also 'networkidle2' which means no more than 2 network connections for at least 500 ms.

Then we can call page.screenshot to take the screenshot after this line.

Parse Stream to Object with S3 Download

If we have a download from S3, we can parse it to an object with data.Body.toString() .

For example, we can write:

const options = {
  BucketName: 'mybucket',
  ObjectName: 'path/to/my.json',
  ResponseContentType: 'application/json'
};

s3.GetObject(options, (err, data) => {
  const fileContents = data.Body.toString();
  const json = JSON.parse(fileContents);
  console.log(json);
});

We pass in the bucket, pathname, and response data type.

We use s3.GetObject to do the download with the given options.

Since we have JSON, we can convert it to a string with data.Body.toString() .

Then we can use JSON.parse to parse it to an object.

Conclusion

We can convert an item downloaded from S3 into a string.

We’ve to handle errors in Express route handlers ourselves.

Also, we can send email to multiple recipients with nodemailer.

We can sort by date with Sequelize queries.

With socket.io, we can generate our own IDs.

Categories
Node.js Tips

Node.js Tips — Related Documents, Unzipping Files, and Parsing HTML

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Creating Rooms in Socket.io

To create rooms in socket.io, we just have to join a room by name.

It doesn’t have to be created.

For instance, we can write:

socket.on('create', (room) => {
  socket.join(room);
});

We listen to the create event with the room name.

We get the room parameter with the room we want to join.

Then we can emit the event so join is run:

socket.emit('create', 'room');

'create' is the event name.

'room' is the room name.

Use populate with Multiple Schemas in Mongoose

We can chain the populate calls to select multiple related documents.

For instance, we can write:

Person.find()
  .populate('address')
  .populate('phone')
  .exec((err, results) => {
    //...
  });

to get the Person with the address and phone models.

Then we get all the related documents in the results .

Also, we can call populate with a space-delimited string to specify multiple schemas without call populate multiple times.

For instance, we can write:

Person.find()
  .populate('address phone')
  .exec((err, results) => {
    //...
  });

This is available since version 3.6 of Mongoose.

Download and Unzip a Zip File in Memory in a Node App

We can download and unzip a zip file in memory with the adm-zip package.

To install it, we run:

npm install adm-zip

Then we can download and unzip a file as follows:

const fileUrl = 'https://github.com/notepad-plus-plus/notepad-plus-plus/releases/download/v7.8.6/npp.7.8.6.bin.zip'

const AdmZip = require('adm-zip');
const http = require('http');

http.get(fileUrl, (res) => {
  let data = [];
  let dataLen = 0;

res.on('data', (chunk) => {
  data.push(chunk);
  dataLen += chunk.length;

})
.on('end', () => {
  const buf = Buffer.alloc(dataLen);

  for (let i = 0, pos = 0; i < data.length,; i++) {
    data[i].copy(buf, pos);
    pos += data[i].length;
  }

  const zip = new AdmZip(buf);
  const zipEntries = zip.getEntries();

  for (const zipEntry of zipEntries) {
    if (zipEntry.entryName.match(/readme/))
      console.log(zip.readAsText(zipEntry));
    }
  };
});

We use the http module’s get method to make a GET request to get the zip file.

Then we put the response data chunks into the data array.

Once we did that, we create a new buffer with Buffer.alloc .

Then we put the chunks in data into the buffer according to the position.

Once we copied everything into the buffer, we create a new AdmZip instance with the buffer.

Then we can get the entries from it.

We looped through the entries in the for-of loop.

Use a DOM Parser with Node.js

There are several DOM parser libraries made for Node apps.

One is the jsdom library.

We can use it by writing:

const jsdom = require("jsdom");
const dom = new jsdom.JSDOM(`<p>Hello world</p>`);
dom.window.document.querySelector("p").textContent;

We pass in an HTML string into the JSDOM constructor.

Then we can use the dom.window.document.querySelector method to get an element from the parsed DOM tree.

There’s also the htmlparser2 library.

It’s faster and more flexible than jsdom but the API is more complex.

To use it, we can write:

const htmlparser = require("htmlparser2");
const parser = new htmlparser.Parser({
  onopentag(name, attrib){
    console.log(name, attrib);
  }
}, { decodeEntities: true });
parser.write(`<p>Hello world</p>`);
parser.end();

We use the htmlparser.Parser constructor, which takes an object with the onopentag handler that’s run when the HTML string is parsed.

name has the tag name and attrib has the attributes.

decodeEntities means that we decode the entities within the document.

cheerio is another DOM parser library. It has an API similar to jQuery, so it should be familiar to many people.

We can use it by writing:

const cheerio = require('cheerio');
const $ = cheerio.load(`<p>Hello world</p>`);
$('p').text('bye');
$.html();

cheerio.load loads the HTML string into a DOM tree.

Then we can use the returned $ function to select elements and manipulate it.

Conclusion

We don’t have to create a room explicitly to join it.

We can zip files with a library.

There are many libraries to let us parse HTML strings into a DOM tree.

We can use Mongoose’spopulate to get related documents.

Categories
Node.js Tips

Node.js Tips — Body Parser Errors, Static Files, and Update Collections

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Register Mongoose Models

We can register Mongoose Models by exporting them and then requiring them in our Express app’s entry point.

For instance, we can write the following in models/User.js:

const mongoose = require('mongoose');
const Schema = mongoose.Schema;

const userSchema = new Schema({
  name: String
});

mongoose.model('User', userSchema);

Then in our app.js file, we can write:

mongoose.connect("mongodb://localhost/news");
require("./models/User");

const users = require('./routes/users');

const app = express();

Requiring the ./models/User.js will make it run the code to register the model.

Catch Express body-parser Error

We can catch Express body-parser errors with our own middleware.

For instance, we can write:

app.use( (error, req, res, next) => {
  if (error instanceof SyntaxError) {
    sendError(res, 'syntax error');
  } else {
    next();
  }
});

We can check if error is an instance of SyntaxError and do something with it.

Otherwise, we call next to continue to the next middleware.

Route All Requests to index.html in an Express App

We can route all requests to index.html in an Express app by using rthe sendFile method in one route handler.

For instance, we can write:

const express = require('express')
const server = express();
const path = require('path');

server.use('/public', express.static(path.join(__dirname, '/assets')));

server.get('/foo', (req, res) => {
  res.send('ok')
})

server.get('/*', (req, res) => {
  res.sendFile(path.join(__dirname, '/index.html'));
})

const port = 8000;
server.listen(port)

We redirect all requests to index.html with a catch-all route at the end.

This way, we can still have some routes that do something else.

The foo route responds with ok .

express.static serves static content.

Basic Web Server with Node.js and Express for Serving HTML File and Assets

We can use the express.static middleware to serve static assets.

For instance, we can write:

const express = require('express');
const app = express();
const http = require('http');
const path= require('path');
const httpServer = http.Server(app);

app.use(express.static(path.join(__dirname, '/assets'));

app.get('/', (req, res) => {
  res.sendfile(path.join(__dirname, '/index.html'));
});
app.listen(3000);

We create a server with the express.static middleware to serve static files from the assets folder.

Then we create a GET route to render index.html .

We can also use connect to do the same thing:

const util = require('util');
const connect = require('connect');
const port = 1234;

connect.createServer(connect.static(__dirname)).listen(port);

We call createServer to create a web server.

Then we use connect.static to serve the current app folder as a static folder.

Gzip Static Content with Express

We can use the compression middleware to serve static content with Gzip.

For instance, we can write:

const express = require('express');
const compress = require('compression');
const app = express();
app.use(compress());

Find the Size of a File in Node.js

We can use the statSync method to get the size pf a file.

For instance, we can write:

const fs = require("fs");
const stats = fs.statSync("foo.txt");
const fileSizeInBytes = stats.size

We call statSync to return an object with some data about the file.

The size in bytes is in the size property.

Automatically Add Header to Every “render” Response

We can use the res.header method to return a header for every response.

For instance, we can write:

app.get('/*', (req, res, next) => {
  res.header('X-XSS-Protection', 0);
  next();
});

to create a middleware to set the X-XSS-Protection header to 0.

Update Child Collection with Mongoose

We get the updated collection after the update in the callback.

For instance, we can write:

Lists.findById(listId, (err, list) => {
  if (err) {
    ...
  } else {
    list.items.push(task)
    list.save((err, list) => {
      ...
    });
  }
});

We called findById to get the collection.

Then we call push on the items child collection.

Finally, we call list.save to save the data and get the latest list in the list array of the save callback.

Also, we can use the $push operator to do the push outside the callback:

Lists.findByIdAndUpdate(listId, {$push: { items: task }}, (err, list) => {
  //...
});

We call findByIdAndUpdate with the listId to get the list.

Then we set items with our task object and use $push to push that.

Then we get the latest data from the list parameter in the callback.

Conclusion

We can register Mongoose models by running the code to register the schema.

There’s more than one way to add an item to a child collection.

We can serve static files with Express or Connect.

We can catch body-parser errors.

Categories
Node.js Tips

Node.js Tips — Auto-Scrolling, Delays, Hashing, and Get the Latest Mongo Document

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Using Q delay in Node.js

The Q promise library has a delay method that can delay the execution of code asynchronously.

For instance, we can use it by wiring:

const Q = require('q');

const asyncDelay = (ms) => {
  return Q.delay(ms);
}

asyncDelay(1000)
.then(() => {
  //...
});

We have the asyncDelay function which returns the Q.delay(ms) promise.

It returns a promise that delays the execution of something by the given number of milliseconds.

We can call it and then call then and run the code we want after asyncDelay is done in the then callback.

Also, we can pass in a 2nd argument which can be called a resolved value.

For instance, we can write:

const Q = require('q');

Q.delay(1000, "Success")
  .then((value) => {
      console.log(value);
  });

Then value would be 'Success' since that’s what we passed in as the 2nd argument of delay .

Convert an Image to a base64-Encoded Data URL

To convert an image to a base64-encoded data URL, we can use the readFileSync method to read the file’s data.

Then we pass the content into the Buffer constructor.

And then we can call toString to convert it to a base64 string.

For instance, we can write:

const fs = require('fs');

const file = 'img.jpg';
const bitmap = fs.readFileSync(file);
const dataUrl = new Buffer(bitmap).toString('base64');

We pass the file path into readFileSync .

Then we pass the returned content into the Buffer constructor.

Then we call toString with 'base64' to convert it to a base64 string.

Return the Updated Document with MongoDB findOneAndUpdate

We should set the returnOriginal property to false in the options object to return the updated document in the callback.

For instance, we can write:

db.collection('user_setting').findOneAndUpdate({
  user_id: data.userId
}, {
  $set: data
}, {
  returnOriginal: false
}, (err, res) => {
  if (err) {
    return console.log(err);
  } else {
    console.log(res);
  }
});

The first argument is the query to find the object to update.

The 2nd argument is the data to update the object with.

The 3rd is the options for updating.

We have the returnOriginal option set to false in the 3rd argument.

The last argument is the callback.

res should have the updated results.

Use jQuery Installed with NPM in Express App

To use jQuery installed with NPM in an Express app, we can serve the jQuery’s dist folder with express.static .

For instance, we can write:

const path = require('path');

app.use('/jquery', express.static(path.join(__dirname,  '/node_modules/jquery/dist/')));

Then we can use it in our HTML file by writing:

<script src="/jquery/jquery.js"></script>

Scroll Down Until We Can’t Scroll with Puppeteer

We can check if we’re at the end of the page and use the scrollBy method to scroll.

For instance, we can write:

const distance = 100;
const delay = 100;
while (document.scrollingElement.scrollTop + window.innerHeight < document.scrollingElement.scrollHeight) {
  document.scrollingElement.scrollBy(0, distance);
  await new Promise(resolve => { setTimeout(resolve, delay); });
}

We check if we’re can scroll with:

document.scrollingElement.scrollTop + window.innerHeight < document.scrollingElement.scrollHeight

When scrollTop and innerHeight is less than scrollHeight , we can scroll.

Find OS Username in Node.js

We can get the current user’s username with the os module.

We write:

require("os").userInfo().username

to get the username

In Windows 10, it returns the first name of the owner account that has been used.

There’s also the username module.

We can install it by running:

npm install username

Then we can use it by writing:

const username = require('username');

(async () => {
  console.log(await username());
})();

Hash JavaScript Objects

To hash JavaScript objects, we can use the object-hash module.

For instance, we can write:

const hash = require('object-hash');

const obj = {a: 1, b: 2};
const hashed = hash(obj);

We just call the hash function from the package to return a hashed string.

Conclusion

We can hash JavaScript objects with the object-hash module.

Also, we can get the username with os or a 3rd party module.

The Q promise library has a delay method to delay execution of functions.

We can get the updated object with findOneAndUpdate .

We can scroll until we can’t scroll with Puppeteer by checking the heights.

Categories
Node.js Tips

Node.js Tips — Socket.io, MongoDB and Nested Objects, and Express Issues

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Updating a Nested Array with MongoDB

We can update nested arrays with the native MongoDB driver.

To do that, we can use the update method.

We can write:

Model.update({
    "questions.answers._id": "123"
  }, {
    "$push": {
      "questions.answers.$.answeredBy": "success"
    }
  },
  (err, numAffected) => {
    // ...
  }
);

We called update with a few arguments.

The first is the query for the nested array entry to update.

The 2nd is the $push appends a new entry to the questions.answers array with the answeredBy value.

The $ sign is the positional operator, and it’s a placeholder for a single value.

Then the callback has the result after the push operation.

Express Static Relative to Parent Directory

We can use path.join to get the absolute path of the directory that we want to serve as the static folder.

For example, we can write:

const path = require('path');
const express = require('express');
const app = express();
app.use(express.static(path.join(__dirname, '../public')));

The ../ symbols move one directory level up and we get the public directory from there.

Fix ‘TypeError: Router.use() requires middleware function but got an Object’ Error in Express Apps

To fix this error, we’ve to make sure the routeing module is exported.

For instance, we can write:

users.js

const express = require('express');
const router = express.Router();

router.get('/', (req, res, next) => {
  //Do whatever...
});

module.exports = router;

We have:

module.exports = router;

to export the router .

Then we can use it in another file by writing:

const users = require('./users');

//...
app.use('/users', users);

Socket.io Error Hooking into Express.js

We can incorporate socket.io in our Express app by call the socket.io function.

For instance, we can write:

const express = require('express');
const app = express();
const server = app.listen(port);
const io = require('socket.io')(server);

We just require the socket.io module and call the required function with our server .

server is an http server instance.

Wait for Multiple Async Calls with Node.js

We can use Underscore’s or Lodash’s after method to wait for multiple async calls to be done before calling another function.

For instance, we can write:

const finished = _.after(2, baz);

foo(data, (err) => {
  //...
  finished();
});

bar(data, (err) => {
  //...
  finished();
})

const baz = () => {
  //...
}

The after method returns a finish function that we can call when the async callbacks are called.

Since we want to wait for 2 functions to finish, we pass 2 into after as the first argument.

We call finished in each callback to indicate that it’s finished.

Use Aliases with Node.js’s require Function

We can create aliases for required members by using the destructuring syntax.

For example, we can write:

const { foo: a, bar: b } = require('module');

We import a and b with require .

Then we alias a as foo and b as bar with the destructuring assignment syntax.

Then they’re accessed as foo and bar .

How to Send JavaScript Object with Socket.io

We can send JavaScript objects with socket.io in a few ways.

We can use the send method:

socket.send({ foo: 'b' });

This would convert the object automatically to JSON.

Also, we can use socket.json.send :

socket.json.send({ foo: 'b' });

And we can also use emit :

socket.emit('someEvent', { foo: 'b' });

emit emits an event with name someEvent and an object in the 2nd argument.

Get Error Status Code from HTTP GET

To get the error code for a GET request made with the http module, we can get it from the statusCode property of res .

For instance, we can write:

const https = require('https');

https.get('https://example.com/error', (res) => {
  console.log(res.statusCode);

  res.on('data', (d) => {
    process.stdout.write(d);
  });
})
.on('error', function(e) {
  console.error(e);
});

The status code is in the res.statusCode property to get the status code.

Conclusion

We can use update to update a nested array with MongoDB.

The status code of a request made with http is in the statusCode property.

We can send objects with socket.io.

Static folders should be served with absolute paths with Express.

We can use after to wait for multiple async calls to complete before calling another function.