Node.js Tips — Async and Map, Async and MongoDB, and Copying Files with S3

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Call an Async Function within map

We can use Promise.all to map functions to async functions.

For instance, we can write:

const asyncMap = async (teachers) => {
  const data = await Promise.all(teachers.map(async teacher => {
    return {
      ...teacher,
      image: `${teacher.name}.jpg`
   }
  }));
  //...
}

We map the teachers array to an array of promises with map .

An async function returns a promise, so we can just add the async keyword in front of the callback for map .

Then we can use await and Promise.all to invoke all the promises in parallel.

And we can do whatever we want with the results afterward.

Watch Directory for Changes with Nodemon

We can use the --watch option to watch multiple files and directories.

For example, we can run:

nodemon --watch src app.js

to watch the src folder and app.js .

Copy All Objects in Amazon S3 from One prefix to Another Using the AWS SDK for Node.js

To copy objects from one key to another, we can use the listObjects to list the entries.

Then we can use copyObject to copy the entries to the key location.

For example, we can write:

const AWS = require('aws-sdk');
const async = require('async');
const bucketName = 'foo';
const oldPrefix = 'bar/';
const newPrefix = 'baz/';
const s3 = new AWS.S3({ params: { Bucket: bucketName }, region: 'us-west-2' });

const done = (err, data) => {
  if (err) { console.log(err); }
  else { console.log(data); }
};

s3.listObjects({ Prefix: oldPrefix }, (err, data) => {
  if (data.Contents.length) {
    async.each(data.Contents, (file, cb) => {
      const params = {
        Bucket: bucketName,
        CopySource: `${bucketName}/${file.Key}`,
        Key: file.Key.replace(oldPrefix, newPrefix)
      };
      s3.copyObject(params, (copyErr, copyData) => {
        if (copyErr) {
          console.log(copyErr);
        }
        else {
          console.log(params.Key);
          cb();
        }
      });
    }, done);
  }
});

We call listObjects with the oldPrefix to get the items in the lol path.

The entries are stored in data.Contents .

Then we use async.each to iterate through each object.

Bucket has the bucket name.

CopySource has the original path.

Key has the new key, which we call replace to with the oldPrefix and newPrefix to replace it with the oldPrefix with newPrefix .

We pass the params object to the copyObject method.

Then we get the results afterward.

Once we did that, we call cb to complete the iteration.

We call done which we defined earlier to print any errors or results.

Sequential MongoDB Query in Node.js

We can use async and await to make MongoDB queries in a synchronous style.

It looks synchronous in that it’s sequential, but it’s asynchronous.

For instance, we can write:

const getPerson = async () => {
  const db = await mongodb.MongoClient.connect('mongodb://server/db');
  if (await db.authenticate("username", "password")) {
    const person = await db.collection("Person").findOne({ name: "james" });
    await db.close();
    return person;
  }
}

return returns a promise that resolves to person .

It doesn’t actually return person .

db.collection gets the collection we want to query.

findOne finds one entry that matches the condition in the object.

We look for an entry with name value 'james' .

The resolved value of the query is set to person .

Then we call db.close when we’re done the query.

Then we return person to resolve the returned promise to that value.

Automatically Add the Header to Every “render” Response in an Express App

We can make our own middleware to add a header to every response.

For instance, we can write:

app.use((req, res, next) => {
  res.header('foo', 'bar');
  next();
});

With res.header , we add the 'foo' response header with value 'bar' to all responses.

Then we call next to run the next middleware.

This piece of code should be added before the routes so that it runs before each route.

Conclusion

We can call map to map each entry to a promise.

Then we call Promise.all to invoke them in parallel.

We can use the AWS SDK to copy files in S3.

MongoDB methods support promises.

We can add response headers to all routes with a middleware.