Categories
Node.js Tips

Node.js Tips — Mongoose Relationships, Body Parser, and Reading and Writing Files

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Disable Express BodyParser for File Uploads

We can bypass body-parser by setting the value of the body.parse('multipart/form-data') function to a different value.

For instance, we can write:

const express = require('express');
const bodyParser = express.bodyParser;

bodyParser.parse('multipart/form-data') = (req, options, next) => {
  next();
}

We set a new value to the body.parse('multipart/form-data') function to do nothing and just call next .

Then no parsing will be done when multipart/form-data data types are encountered.

A better way is to not use bodyParser directory.

Instead, we can just use it to parse the data type we want by writing:

const bodyParser = require("body-parser");
app.use(bodyParser.json());
app.use(bodyParser.urlencoded());

We call the json and urlencoded methods to only parse JSON and URL encoded data.

Node.js — Creating Relationships with Mongoose

We can create relationships with Mongoose by using the ref property.

For instance, we can write:

const Schema = mongoose.Schema;
const ObjectId = Schema.ObjectId;

const PersonSchema = new Schema({
  name : String
})

const AddressSchema = new Schema({
  street: String
  person: { type: ObjectId, ref: 'PersonSchema' }
})

In AddressSchema , we set the ref property to the string of the schema name for PersonSchema .

Then we can use it by writing:

const person = new PersonSchema({ name: 'james' });
person.save();

const address = new AddressSchema({
  phone: '123 A St.',
  person: person._id
});
address.save();

We create the person and address , which is related to person .

The person._id if the object ID for the person which we want to reference in address .

Then we can get the address when we query PersonSchema buy writing:

PersonSchema
  .findOne({})
  .populate('address')
  .exec((err, address) => {
    //...
  })

We query the PersonSchema with findOne , then call populate to get the address from the person.

Looping Through Files in a Folder Node.js

We can loop through files in a folder in Node apps by using the opendir method.

For instance, we can write:

const fs = require('fs')

const listFiles = async (path) => {
  const dir = await fs.promises.opendir(path);
  for await (const dirent of dir) {
    console.log(dirent.name);
  }
}

listFiles('.').catch(console.error)

We use the promise version of the opendir method.

The path is the path to the folder.

Then we use the for-await-of loop to read the entry’s name property to log their name.

Since an async function returns a promise, we can use catch to handle errors when we read the directory’s entries.

We can also read a directory’s entries synchronously.

For instance, we can write:

const fs = require('fs');

const dir = fs.opendirSync('.');
let dirent;
while ((dirent = dir.readSync()) !== null) {
  console.log(dirent.name)
}
dir.closeSync();

We call openDirSync to read the entries with the given path.

Then we use the while loop to read each entry with readSync .

Then we close the handle with closeSync .

Checking if writeFileSync Successfully Wrote the File

writeFileSync doesn’t return anything, so we won’t know whether the file is successfully written.

Instead, we can use the async version, which takes a callback that brings us the error if there’s an error.

For instance, we can write:

fs.exists(file, (exists) => {
  if (exists) {
    fs.writeFiles(file, content, 'utf-8', (err) => {
      if (err) {
        console.log("failed to save");
      } else {
        console.log("success");
      }
  } else {
    console.log('file not found');
  }
}

We check if the exists. If it does exist, then we write what we want to it.

file is the file path. content is the content that we want to write.

'utf-8' is the encoding.

err is the error that’s encountered when the file failed to save.

Otherwise, we don’t do anything.

Conclusion

We can use parts of the body-parser package to parse the request payload we want.

Mongoose can create schemas with relationships.

We can loop through folders with various methods.

Also, we can check if a file exists before we write to them.

writeFileSync doesn’t guarantee that file writing operations succeed.

Categories
Node.js Tips

Node.js Tips — Send Message, Adding End of Line, and S3 Uploads

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Send a Message to a Particular Client with Socket.io

We can send a message to a particular client with socket.io by using the emit method.

For instance, we can write:

io.to(socket.id).emit("event", data);

We call the io.to method with the ID of the client we want to send our message to.

Responding with a JSON object in Node.js

We can use the res.json method with Express to return JSON response.

For instance, we can write:

res.json({ foo: 'bar' });

to return the JSON object that we passed in.

Check if Node.js is Installed or Not

We can check if Node is installed by using the node -v command to display the Node version.

Select and Update Document by _id with node-mongodb-native

To select and update a document by its _id , we can use the mongo.ObjectID constructor to create the ID object.

For instance, we can write:

const mongo = require('mongodb');
const id = new mongo.ObjectID(theId);
collection.update({ '_id': id });

theId is the string of the ID.

Then we use that as the value of the '_id' property.

How to Append to New Line in Node.js

We can append a new line to a file in a Node app by using the open and write methods.

To add the new line, we use the os.EOL property to add the new line character.

This way, the new line will be inserted correctly with all platforms.

For instance, we can write:

const os = require("os");

const addNewLine = (text) => {
  fs.open('/path/to/file', 'a', 666, (e, id) => {
    fs.write(id, `${text}${os.EOL}`, null, 'utf8', () => {
      fs.close(id, () => {
       console.log('file is updated');
      });
    });
  });
}

We created the addNewLine method to open the file.

We open it with append permission as indicated by 'a' .

666 is the file permission which includes a read and writes.

Then we get the file ID in the callback so we can use it to update the file.

In the fs.write call, id is the file ID. The 2nd argument is text content.

os.EOL is the platform-agnostic new line constant.

'utf8' is the UTF-8 encoding.

Then callback is called when writing is done.

Then we clean up the filehandle with fs.close .

Uploading base64 Encoded Image to Amazon S3 via Node.js

To upload a base64 image with S3, we can convert it to the buffer.

Then we can call putObject to upload the file.

For instance, we can write:

const AWS = require('aws-sdk');
AWS.config.loadFromPath('./config.json');
const s3Bucket = new AWS.S3( { params: { Bucket: 'someBucket'} } );

const buffer = new Buffer(req.body.imageBinary.replace(/^data:image/w+;base64,/, ""),'base64');

const data = {
  Key: 123,
  Body: buffer,
  ContentEncoding: 'base64',
  ContentType: 'image/jpeg'
};

s3Bucket.putObject(data, (err, data) => {
  if (err) {
    console.log(err, data);
  } else {
    console.log('success');
  }
});

We firs get the credentials from the config.json file to authenticate for the AWS SDK.

Then we create a buffer from the base64 string.

We’ve to take out the data:image/w+;base64 part.

Then we create a data object with the Key , Body , ContentEncoding and ContentType properties.

Key is the path to the file.

Body has the content of the file.

ContentEncoding has the encoding.

ContentType has the data type.

Then we call putObject to upload the file.

config.json has:

{
  "accessKeyId":"xxxxxxxxxxxxxxxx",
  "secretAccessKey":"xxxxxxxxxxxxxx",
  "region":"us-east-1"
}

Stdout Buffer Issue Using Node child_process’s exec — maxBuffer Exceeded

If we get the ‘maxBuffer exceeded’ error, we can use set the maxBuffer option on exec to increase the buffer size.

For instance, we can write:

exec('ls', { maxBuffer: 1024 ** 2 }, (error, stdout, stderr) => {
  console.log(error, stdout);
});

We set the maxBuffer size in the object in the 2nd argument.

Conclusion

We can use the io.to method to send the message to a given client.

To send JSON response, we can use res.json .

We can update with ID.

To max buffer size of exec can be increased.

We use the os.EOL constant to add a platform-agnostic end of line character.

Categories
Node.js Tips

Node.js Tips —MongoDB Connections, Fetch, and Exports

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Retrieve Data from a ReadableStream Object Returned From the Fetch API

We can retrieve data from a ReadableStream object by calling conversion functions to convert it to the data type we want.

For instance, we write:

fetch('https://api.agify.io/?name=michael')
  .then(response => response.json())
  .then((data) => {
    console.log(data);
  });

We did the conversion to JSON with the response.json() call.

Then we can get the data in the next then callback.

Share Variables Between Files in Node.js

We can share variables by creating modules.

For instance, we can write:

module.js

const name = "foo";
exports.name = name;

app.js

const module = require('./module');
const name = module.name;

We export the name constant by setting it as a property of exports .

Then we import it with require in app.js .

And we can use it like any other property.

Using fs in Node.js with async / await

We can use async and await with fs methods if we convert them to promises first.

For instance, we can write:

const fs = require('fs');
const util = require('util');

const readdir = util.promisify(fs.readdir);

const read = async () => {
  try {
    const names = await readdir('/foo/bar/dir');
    console.log(names);
  } catch (err) {
    console.log(err);
  }
}

We convert the fs.readdir method, which reads a directory, to a promise version with util.promisify .

Then we can use it with async and await .

Mongoose and Multiple Database in Single Node.js Project

We can connect to multiple MongoDB databases in a single Node project.

For instance, we can write:

fooDbConnect.js

const mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/foo');
module.exports = exports = mongoose;

bazDbConnect.js

const mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/baz');
module.exports = exports = mongoose;

We connect to different databases with connect in each file and export the connections.

Then in db.js , we write:

const mongoose = require("./`fooDbConnect`");

to connect to the foo database.

Use of module.exports as a Constructor

We can export a constructor with module.exports .

For example, we can write:

person.js

function Person(name) {
  this.name= name;
}

Person.prototype.greet = function() {
  console.log(this.name);
};

module.exports = Person;

Then we can import it by writing:

app.js

const Person = require("./`person`");
const person = new Person("james");

We import the Person constructor in app.js and then invoked it in the last line.

Read a Text File Using Node.js

We can read a text file with the fs.readFile method.

For instance, we can write:

const fs = require('fs');

fs.readFile('./foo.txt', 'utf8', (err, data) => {
  if (err) {
    console.log(err);
  }
  console.log(data);
});

We pass in the file path as the first argument.

The encoding is the 2nd argument.

The callback that’s called when the file read operation is done is the 3rd argument.

data has the file and err has the error object is an error is encountered.

Reuse Connection to MongoDB Across Node Modules

We can export the MongoDB connection so that we can use them in other Node app modules.

For example, we can write:

dbConnection.js

const MongoClient = require( 'mongodb' ).MongoClient;
const url = "mongodb://localhost:27017";

let _db;

module.exports = {
  connect(callback) {
    MongoClient.connect( url, { useNewUrlParser: true }, (err, client) => {
      _db  = client.db('some_db');
      return callback(err, client);
    });
  },

  getDb() {
    return _db;
  }
};

We created the connect function to connect to the MongoDB database.

The connection object is set to a variable.

It takes a callback that we can use to get any errors is there is any.

Then we created the getDb function to get the MongoDB connection.

Then we can use connect.js by writing:

const dbConnection = require('dbConnection');

dbConnection.connect((err, client) => {
  if (err) {
    console.log(err);
  }
} );

We connect to the database in our file by importing the dbConnection module we created earlier.

Then we call connect with the callback we want.

We called it with err and client in dbConnection.js so that’s what we have as the parameter of the callback.

Conclusion

MongoDB connections can be shared between modules.

We’ve to convert the raw response from the fetch function to the data type we want before we can use the data.

fs methods can be converted to promises.

We can export constructors with module.exports.

Categories
Node.js Tips

Node.js Tips — Try/Catch, Express and Clusters, and Getting Mime Types

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Getting Line Number in try-catch

We can get the line number with try-catch by using the stack property.

For instance, we can write:

try {
  //...
} catch(err) {
  console.log(err.stack);
}

Display PDF in the Browser Using Express

To display a PDF in the browser, we set a few headers to make sure that our Express route returns a PDF.

For instance, we can write:

app.post('/url/to/hit', (req, res, next) => {
  const stream = fs.readStream('/location/of/pdf');
  const filename = "doc.pdf";

  filename = encodeURIComponent(filename);

  res.setHeader('Content-Disposition', `inline; filename=${filename}`);
  res.setHeader('Content-Type', 'application/pdf');
  stream.pipe(res);
});

We set the Content-Disposition header to set the header to the PDF file’s file name.

inline means display the data in the browser within a web page or part of a page.

Also, we’ve to set the Content-type header to application/pdf to make sure that the client knows that we’re responding with a PDF.

Accept POST Body from Request Using Express

To make sure that we can send POST requests to an Express sewer, we should use the body-parser middleware.

For instance, we write:

app.use(bodyParser.json());
app.use(bodyParser.urlencoded());

Then we have the routes below these 2 lines.

Now we can parse JSON payloads on the server-side.

Then on the client-side, we should have the Content-Type request header set to application/json .

Mixing JavaScript and TypeScript in Node.js

There are a few ways to let us mix JavaScript and TypeScript in Node apps.

We can set the --allowJs compiler option.

We also need to set the --outDir option to set the folder for storing the compiled files.

--checkJs lets us check the types of JavaScript files.

Node.js Clusters with a Simple Express App

We can use an Express app in a cluster.

For example, we can write:

const cluster = require('cluster');
const express = require('express');
const app = express.createServer();

if (cluster.isMaster) {
  for (let i = 0; i < 2; i++) {
    cluster.fork();
  }
} else {
  app.configure(() => {
    app.set('view options', {
      layout: false
    });
    app.set('view engine', 'jade');
    app.set('views', __dirname + '/views');
    app.use(express.bodyParser());
  });
  app.get('/', (req, res) => {
    res.render('index');
  });
  app.listen(8080);
}

We use the cluster module create our cluster by calling cluer.fork if the process is a master process.

Other, we create an instance of the Express app.

Sequelize Update with Association

We can use Sequelize to update a database entry and its association.

For instance, we can write:

const filter = {
  where: {
    id
  },
  include: [{
    model: Profile
  }]
};

Product.findOne(filter).then((product) => {
  if (product) {
    return product.Profile.updateAttributes(updateProfile).then((result) => {
      return result;
    });
  } else {
    throw new Error("product not found");
  }
});

We get the Product with a filter object.

Then we get its child association with product.Profile to get the Profile model.

And we call the updateAttributes method on it.

In the filter object, we’ve to ad the include property with the child model we want to include.

Exclude Route from Express Middleware

To exclude a route from running an Express middleware, we can pass in the regex of the pattern of the route that we want to run.

For instance, we write:

app.use(//((?!foo).)*/, routerHandler);

We exclude the route with path /foo with our pattern.

Then routerHandler runs on everything except when the path is /foo .

Render Raw HTML

We can render raw HTML by writing:

app.get('/', (req, res) => {
  res.sendFile(__dirname + '/public/layout.html');
});

We use res.sendFile to render the HTML located in the absolute path.

Get File Type of File

To get the type of the file, we can use the mime module.

For instance, we can write:

const mime = require('mime');

const type = mime.getType('/path/to/file.txt');

We just call the getType method.

There’s also the mime-types method:

const mime = require('mime-types');

const type = mime.lookup(fileImageTmp);

Conclusion

We can use the body-parser to parse the request body.

Also, we can use a package to parse the mime type of a file.

Sequelize can update associations.

The cluster module can update child associations.

We can respond with PDFs with a cluster.

Categories
Node.js Tips

Node.js Tips — Try/Catch, Express and Clusters, and Getting Mime Types

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Getting Line Number in try-catch

We can get the line number with try-catch by using the stack property.

For instance, we can write:

try {
  //...
} catch(err) {
  console.log(err.stack);
}

Display PDF in the Browser Using Express

To display a PDF in the browser, we set a few headers to make sure that our Express route returns a PDF.

For instance, we can write:

app.post('/url/to/hit', (req, res, next) => {
  const stream = fs.readStream('/location/of/pdf');
  const filename = "doc.pdf";

  filename = encodeURIComponent(filename);

  res.setHeader('Content-Disposition', `inline; filename=${filename}`);
  res.setHeader('Content-Type', 'application/pdf');
  stream.pipe(res);
});

We set the Content-Disposition header to set the header to the PDF file’s file name.

inline means display the data in the browser within a web page or part of a page.

Also, we’ve to set the Content-type header to application/pdf to make sure that the client knows that we’re responding with a PDF.

Accept POST Body from Request Using Express

To make sure that we can send POST requests to an Express sewer, we should use the body-parser middleware.

For instance, we write:

app.use(bodyParser.json());
app.use(bodyParser.urlencoded());

Then we have the routes below these 2 lines.

Now we can parse JSON payloads on the server-side.

Then on the client-side, we should have the Content-Type request header set to application/json .

Mixing JavaScript and TypeScript in Node.js

There are a few ways to let us mix JavaScript and TypeScript in Node apps.

We can set the --allowJs compiler option.

We also need to set the --outDir option to set the folder for storing the compiled files.

--checkJs lets us check the types of JavaScript files.

Node.js Clusters with a Simple Express App

We can use an Express app in a cluster.

For example, we can write:

const cluster = require('cluster');
const express = require('express');
const app = express.createServer();

if (cluster.isMaster) {
  for (let i = 0; i < 2; i++) {
    cluster.fork();
  }
} else {
  app.configure(() => {
    app.set('view options', {
      layout: false
    });
    app.set('view engine', 'jade');
    app.set('views', __dirname + '/views');
    app.use(express.bodyParser());
  });
  app.get('/', (req, res) => {
    res.render('index');
  });
  app.listen(8080);
}

We use the cluster module create our cluster by calling cluer.fork if the process is a master process.

Other, we create an instance of the Express app.

Sequelize Update with Association

We can use Sequelize to update a database entry and its association.

For instance, we can write:

const filter = {
  where: {
    id
  },
  include: [{
    model: Profile
  }]
};

Product.findOne(filter).then((product) => {
  if (product) {
    return product.Profile.updateAttributes(updateProfile).then((result) => {
      return result;
    });
  } else {
    throw new Error("product not found");
  }
});

We get the Product with a filter object.

Then we get its child association with product.Profile to get the Profile model.

And we call the updateAttributes method on it.

In the filter object, we’ve to ad the include property with the child model we want to include.

Exclude Route from Express Middleware

To exclude a route from running an Express middleware, we can pass in the regex of the pattern of the route that we want to run.

For instance, we write:

app.use(//((?!foo).)*/, routerHandler);

We exclude the route with path /foo with our pattern.

Then routerHandler runs on everything except when the path is /foo .

Render Raw HTML

We can render raw HTML by writing:

app.get('/', (req, res) => {
  res.sendFile(__dirname + '/public/layout.html');
});

We use res.sendFile to render the HTML located in the absolute path.

Get File Type of File

To get the type of the file, we can use the mime module.

For instance, we can write:

const mime = require('mime');

const type = mime.getType('/path/to/file.txt');

We just call the getType method.

There’s also the mime-types method:

const mime = require('mime-types');

const type = mime.lookup(fileImageTmp);

Conclusion

We can use the body-parser to parse the request body.

Also, we can use a package to parse the mime type of a file.

Sequelize can update associations.

The cluster module can update child associations.

We can respond with PDFs with a cluster.