Categories
Node.js Tips

Node.js Tips —MongoDB Connections, Fetch, and Exports

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Retrieve Data from a ReadableStream Object Returned From the Fetch API

We can retrieve data from a ReadableStream object by calling conversion functions to convert it to the data type we want.

For instance, we write:

fetch('https://api.agify.io/?name=michael')
  .then(response => response.json())
  .then((data) => {
    console.log(data);
  });

We did the conversion to JSON with the response.json() call.

Then we can get the data in the next then callback.

Share Variables Between Files in Node.js

We can share variables by creating modules.

For instance, we can write:

module.js

const name = "foo";
exports.name = name;

app.js

const module = require('./module');
const name = module.name;

We export the name constant by setting it as a property of exports .

Then we import it with require in app.js .

And we can use it like any other property.

Using fs in Node.js with async / await

We can use async and await with fs methods if we convert them to promises first.

For instance, we can write:

const fs = require('fs');
const util = require('util');

const readdir = util.promisify(fs.readdir);

const read = async () => {
  try {
    const names = await readdir('/foo/bar/dir');
    console.log(names);
  } catch (err) {
    console.log(err);
  }
}

We convert the fs.readdir method, which reads a directory, to a promise version with util.promisify .

Then we can use it with async and await .

Mongoose and Multiple Database in Single Node.js Project

We can connect to multiple MongoDB databases in a single Node project.

For instance, we can write:

fooDbConnect.js

const mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/foo');
module.exports = exports = mongoose;

bazDbConnect.js

const mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/baz');
module.exports = exports = mongoose;

We connect to different databases with connect in each file and export the connections.

Then in db.js , we write:

const mongoose = require("./`fooDbConnect`");

to connect to the foo database.

Use of module.exports as a Constructor

We can export a constructor with module.exports .

For example, we can write:

person.js

function Person(name) {
  this.name= name;
}

Person.prototype.greet = function() {
  console.log(this.name);
};

module.exports = Person;

Then we can import it by writing:

app.js

const Person = require("./`person`");
const person = new Person("james");

We import the Person constructor in app.js and then invoked it in the last line.

Read a Text File Using Node.js

We can read a text file with the fs.readFile method.

For instance, we can write:

const fs = require('fs');

fs.readFile('./foo.txt', 'utf8', (err, data) => {
  if (err) {
    console.log(err);
  }
  console.log(data);
});

We pass in the file path as the first argument.

The encoding is the 2nd argument.

The callback that’s called when the file read operation is done is the 3rd argument.

data has the file and err has the error object is an error is encountered.

Reuse Connection to MongoDB Across Node Modules

We can export the MongoDB connection so that we can use them in other Node app modules.

For example, we can write:

dbConnection.js

const MongoClient = require( 'mongodb' ).MongoClient;
const url = "mongodb://localhost:27017";

let _db;

module.exports = {
  connect(callback) {
    MongoClient.connect( url, { useNewUrlParser: true }, (err, client) => {
      _db  = client.db('some_db');
      return callback(err, client);
    });
  },

  getDb() {
    return _db;
  }
};

We created the connect function to connect to the MongoDB database.

The connection object is set to a variable.

It takes a callback that we can use to get any errors is there is any.

Then we created the getDb function to get the MongoDB connection.

Then we can use connect.js by writing:

const dbConnection = require('dbConnection');

dbConnection.connect((err, client) => {
  if (err) {
    console.log(err);
  }
} );

We connect to the database in our file by importing the dbConnection module we created earlier.

Then we call connect with the callback we want.

We called it with err and client in dbConnection.js so that’s what we have as the parameter of the callback.

Conclusion

MongoDB connections can be shared between modules.

We’ve to convert the raw response from the fetch function to the data type we want before we can use the data.

fs methods can be converted to promises.

We can export constructors with module.exports.

Categories
Node.js Tips

Node.js Tips — Try/Catch, Express and Clusters, and Getting Mime Types

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Getting Line Number in try-catch

We can get the line number with try-catch by using the stack property.

For instance, we can write:

try {
  //...
} catch(err) {
  console.log(err.stack);
}

Display PDF in the Browser Using Express

To display a PDF in the browser, we set a few headers to make sure that our Express route returns a PDF.

For instance, we can write:

app.post('/url/to/hit', (req, res, next) => {
  const stream = fs.readStream('/location/of/pdf');
  const filename = "doc.pdf";

  filename = encodeURIComponent(filename);

  res.setHeader('Content-Disposition', `inline; filename=${filename}`);
  res.setHeader('Content-Type', 'application/pdf');
  stream.pipe(res);
});

We set the Content-Disposition header to set the header to the PDF file’s file name.

inline means display the data in the browser within a web page or part of a page.

Also, we’ve to set the Content-type header to application/pdf to make sure that the client knows that we’re responding with a PDF.

Accept POST Body from Request Using Express

To make sure that we can send POST requests to an Express sewer, we should use the body-parser middleware.

For instance, we write:

app.use(bodyParser.json());
app.use(bodyParser.urlencoded());

Then we have the routes below these 2 lines.

Now we can parse JSON payloads on the server-side.

Then on the client-side, we should have the Content-Type request header set to application/json .

Mixing JavaScript and TypeScript in Node.js

There are a few ways to let us mix JavaScript and TypeScript in Node apps.

We can set the --allowJs compiler option.

We also need to set the --outDir option to set the folder for storing the compiled files.

--checkJs lets us check the types of JavaScript files.

Node.js Clusters with a Simple Express App

We can use an Express app in a cluster.

For example, we can write:

const cluster = require('cluster');
const express = require('express');
const app = express.createServer();

if (cluster.isMaster) {
  for (let i = 0; i < 2; i++) {
    cluster.fork();
  }
} else {
  app.configure(() => {
    app.set('view options', {
      layout: false
    });
    app.set('view engine', 'jade');
    app.set('views', __dirname + '/views');
    app.use(express.bodyParser());
  });
  app.get('/', (req, res) => {
    res.render('index');
  });
  app.listen(8080);
}

We use the cluster module create our cluster by calling cluer.fork if the process is a master process.

Other, we create an instance of the Express app.

Sequelize Update with Association

We can use Sequelize to update a database entry and its association.

For instance, we can write:

const filter = {
  where: {
    id
  },
  include: [{
    model: Profile
  }]
};

Product.findOne(filter).then((product) => {
  if (product) {
    return product.Profile.updateAttributes(updateProfile).then((result) => {
      return result;
    });
  } else {
    throw new Error("product not found");
  }
});

We get the Product with a filter object.

Then we get its child association with product.Profile to get the Profile model.

And we call the updateAttributes method on it.

In the filter object, we’ve to ad the include property with the child model we want to include.

Exclude Route from Express Middleware

To exclude a route from running an Express middleware, we can pass in the regex of the pattern of the route that we want to run.

For instance, we write:

app.use(//((?!foo).)*/, routerHandler);

We exclude the route with path /foo with our pattern.

Then routerHandler runs on everything except when the path is /foo .

Render Raw HTML

We can render raw HTML by writing:

app.get('/', (req, res) => {
  res.sendFile(__dirname + '/public/layout.html');
});

We use res.sendFile to render the HTML located in the absolute path.

Get File Type of File

To get the type of the file, we can use the mime module.

For instance, we can write:

const mime = require('mime');

const type = mime.getType('/path/to/file.txt');

We just call the getType method.

There’s also the mime-types method:

const mime = require('mime-types');

const type = mime.lookup(fileImageTmp);

Conclusion

We can use the body-parser to parse the request body.

Also, we can use a package to parse the mime type of a file.

Sequelize can update associations.

The cluster module can update child associations.

We can respond with PDFs with a cluster.

Categories
Node.js Tips

Node.js Tips — Try/Catch, Express and Clusters, and Getting Mime Types

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Getting Line Number in try-catch

We can get the line number with try-catch by using the stack property.

For instance, we can write:

try {
  //...
} catch(err) {
  console.log(err.stack);
}

Display PDF in the Browser Using Express

To display a PDF in the browser, we set a few headers to make sure that our Express route returns a PDF.

For instance, we can write:

app.post('/url/to/hit', (req, res, next) => {
  const stream = fs.readStream('/location/of/pdf');
  const filename = "doc.pdf";

  filename = encodeURIComponent(filename);

  res.setHeader('Content-Disposition', `inline; filename=${filename}`);
  res.setHeader('Content-Type', 'application/pdf');
  stream.pipe(res);
});

We set the Content-Disposition header to set the header to the PDF file’s file name.

inline means display the data in the browser within a web page or part of a page.

Also, we’ve to set the Content-type header to application/pdf to make sure that the client knows that we’re responding with a PDF.

Accept POST Body from Request Using Express

To make sure that we can send POST requests to an Express sewer, we should use the body-parser middleware.

For instance, we write:

app.use(bodyParser.json());
app.use(bodyParser.urlencoded());

Then we have the routes below these 2 lines.

Now we can parse JSON payloads on the server-side.

Then on the client-side, we should have the Content-Type request header set to application/json .

Mixing JavaScript and TypeScript in Node.js

There are a few ways to let us mix JavaScript and TypeScript in Node apps.

We can set the --allowJs compiler option.

We also need to set the --outDir option to set the folder for storing the compiled files.

--checkJs lets us check the types of JavaScript files.

Node.js Clusters with a Simple Express App

We can use an Express app in a cluster.

For example, we can write:

const cluster = require('cluster');
const express = require('express');
const app = express.createServer();

if (cluster.isMaster) {
  for (let i = 0; i < 2; i++) {
    cluster.fork();
  }
} else {
  app.configure(() => {
    app.set('view options', {
      layout: false
    });
    app.set('view engine', 'jade');
    app.set('views', __dirname + '/views');
    app.use(express.bodyParser());
  });
  app.get('/', (req, res) => {
    res.render('index');
  });
  app.listen(8080);
}

We use the cluster module create our cluster by calling cluer.fork if the process is a master process.

Other, we create an instance of the Express app.

Sequelize Update with Association

We can use Sequelize to update a database entry and its association.

For instance, we can write:

const filter = {
  where: {
    id
  },
  include: [{
    model: Profile
  }]
};

Product.findOne(filter).then((product) => {
  if (product) {
    return product.Profile.updateAttributes(updateProfile).then((result) => {
      return result;
    });
  } else {
    throw new Error("product not found");
  }
});

We get the Product with a filter object.

Then we get its child association with product.Profile to get the Profile model.

And we call the updateAttributes method on it.

In the filter object, we’ve to ad the include property with the child model we want to include.

Exclude Route from Express Middleware

To exclude a route from running an Express middleware, we can pass in the regex of the pattern of the route that we want to run.

For instance, we write:

app.use(//((?!foo).)*/, routerHandler);

We exclude the route with path /foo with our pattern.

Then routerHandler runs on everything except when the path is /foo .

Render Raw HTML

We can render raw HTML by writing:

app.get('/', (req, res) => {
  res.sendFile(__dirname + '/public/layout.html');
});

We use res.sendFile to render the HTML located in the absolute path.

Get File Type of File

To get the type of the file, we can use the mime module.

For instance, we can write:

const mime = require('mime');

const type = mime.getType('/path/to/file.txt');

We just call the getType method.

There’s also the mime-types method:

const mime = require('mime-types');

const type = mime.lookup(fileImageTmp);

Conclusion

We can use the body-parser to parse the request body.

Also, we can use a package to parse the mime type of a file.

Sequelize can update associations.

The cluster module can update child associations.

We can respond with PDFs with a cluster.

Categories
Node.js Tips

Node.js Tips — Promises, CSV to JSON, Watching Files

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Handle Errors Thrown by require() in Node.js

We can use try-catch to catch errors thrown by require .

For instance, we can write:

try {
  require('./app.js');
}
catch (e) {
  console.log(e);
}

We just call require in try then we can catch the error in catch .

How to use async/await with Streams

We can use async and await with streams by creating a promise with the stream with the Promise constructor.

For instance, we can write:

const fd = fs.createReadStream('/foo.txt');
const hash = crypto.createHash('sha1');
hash.setEncoding('hex');
fd.pipe(hash);

const readHash = new Promise((resolve, reject) => {
  hash.on('end', () => resolve(hash.read()));
  fd.on('error', reject);
});

Then we can use readHash in an async function by writing:

`(async function() {
  const sha1 = await` readHash`;
  console.log(sha1);
}());`

Promise Retry Design Patterns

We can retry a promise by delaying the promise.

For instance, we can write:

const retry = (fn, retries=3, err=null) => {
  if (!retries) {
    return Promise.reject(err);
  }
  return fn().catch(err => {
    return retry(fn, (retries - 1), err);
  });
}

fn is a function that returns a promise.

We retry as long as we haven’t exhausted the retries.

If the promise rejected by fn is rejected, then we call catch by passing in a callback to call retry with retries subtracted by 1.

We do that until retries reaches 0, then the retries are exhausted.

We Don’t Need .catch(err => console.error(err)) When Writing Promise Code

We don’t need to write .catch(err => console.error(err)) since promise rejections are already logged by Node.js.

We would know that an error happened even if we don’t have this line.

How to convert CSV to JSON in Node.js

We can use the csvtojson library to convert CSV to JSON.

To install it, we run:

npm install --save csvtojson

Then we can use it by writing:

const csv = require("csvtojson");

const readCsv = (csvFilePath) => {
  csv()
    .fromFile(csvFilePath)
    .then((jsonArrayObj) => {
       console.log(jsonArrayObj);
     })
}

We can read small files by passing in the csvFilePath to the fromFile method.

It returns a promise.

We get the converted result in the jsonArrayObj parameter.

We can also read the content from a stream and then get the JSON.

For instance, we can write:

const readFromStream = (readableStream) => {
  csv()
    .fromStream(readableStream)
    .subscribe((jsonObj) => {
       console.log(jsonObj)
    })
}

We call the fromStream method to get data from a stream.

Then we call subscribe with a callback to read the object.

Also, we can use fromFile with async and await since it returns a promise:

const csvToJson = async (filePath) => {
  const jsonArray = await csv().fromFile(filePath);
}

It can also be used on the command line.

For example, we can run it directly by running:

npm install -g csvtojson
csvtojson ./csvFile.csv

Edit Node App Files on the Server Without Restarting Node.js and See the Latest Changes

We can use a package like Node-Supervisor to watch for file changes and restart the Node app automatically.

To install it, we run:

npm install supervisor -g

Then we can run our program with Node-Supervisor by running:

supervisor app.js

Likewise, we can use Nodemon to do the same thing.

We can install it by running:

npm install -g nodemon

to install it globally.

Or we can install it by running:

npm install --save-dev nodemon

to install it as a project’s dev dependency.

Then we can run:

nodemon app.js

to run our app.

Chain and Share Prior Results with Promises

To share prior results with promises, we can chain them with then .

For instance, we can write:

makeRequest(url1, data1)
  .then((result1) => {
    return makeRequest(url2, data2);
  })
  .then((result2) => {
    return makeRequest(url3, data3);
  })
  .then((result3) => {
     // ...
  });

We get the resolved results from the previous promise in the then callbacks.

To make this shorter, we can use async and await to do the same thing:

const makeRequests = async () => {
  //...
  const r1 = await makeRequest(url1, data1);
  const r2 = await makeRequest(url2, data2);
  const r3 = await makeRequest(url3, data3);
  return someResult;
}

Conclusion

We can put streams in promises.

Also, we can watch for code file changes and restart our Node app with various programs.

The csvtojson library lets us convert CSV to JSON.

Promises can be retried by invoking it again.

Categories
Node.js Tips

Node.js Tips — Modules, Hashes, Favicons, Nested Routers

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Generate Random SHA1 Hash to Use as ID in Node.js

To generate a random SHA1 hash, we can use the crypto module.

For instance, we can write:

const crypto = require('crypto');
const currentDate = (new Date()).getTime().toString();
const random = Math.random().toString();
crypto.createHash('sha1').update(currentDate + random).digest('hex');

We use the getTime method of the Date instance to get the timestamp.

Then we create a random number.

Next, we create the hash by using the crypto module’s createHash and update methods to create the hash.

Then we convert that to hex with digest .

Set Custom Favicon in Express

We can serve a custom favicon by using the serve-favicon package.

For example, we can write:

const favicon = require('serve-favicon');
app.use(favicon(path.join(__dirname,'public','images','favicon.ico')));

The package is installed by running:

npm i serve-favicon

Then we can use the bundled middleware by passing in the path of the middleware.

Open Default Browser and Navigate to a Specific URL in a Node App

We can use the opn package to open the browser and go to a specific URL.

To install it, we run:

npm install opn

Then we can write:

const opn = require('opn');
opn('http://example.com');

to open the URL with the default browser.

We can also specify the browser by writing:

opn('http://example.com', { app: 'firefox' });

Then the URL will open in Firefox.

Express.js with Nested Router

We can nest routers Express.

For instance, we can write:

const express = require('express');
const app = express();

const userRouter = express.Router();
const itemRouter = express.Router({ mergeParams: true });

userRouter.use('/:userId/items', itemRouter);

userRouter.route('/')
  .get((req, res) `=>` {
    res.send('hello users');
  });

userRouter.route('/:userId')
  .get((req, res) `=>` {
    res.status(200)
  });

itemRouter.route('/')
  .get((req, res) `=>` {
    res.send('hello');
  });

itemRouter.route('/:itemId')
  .get((req, res) => {
    res.send(`${req.params.itemId} ${req.params.userId}`);
  });

app.use('/user', userRouter);

app.listen(3000);

We just pass the router items to the place we wish.

mergeParams is need on the itemRouter since we want to access parameters from the parent router.

Other than that, we just nest itemRouter in userRouter by writing:

userRouter.use('/:userId/items', itemRouter);

Convert a Binary NodeJS Buffer to JavaScript ArrayBuffer

We can convert a Node buffer to a JavaScript ArrayBuffer by writing:

const toArrayBuffer = (buffer) => {
  const arrayBuffer = new ArrayBuffer(buffer.length);
  const view = new Uint8Array(arrayBuffer);
  for (let i = 0; i < buffer.length; i++) {
    view[i] = buffer[i];
  }
  return arrayBuffer;
}

We just put each but of the buffer into the Uint8Array .

We can convert the other way by writing:

const toBuffer = (arrayBuffer) => {
  const buffer = Buffer.alloc(arrayBuffer.byteLength);
  const view = new Uint8Array(arrayBuffer);
  for (let i = 0; i < buffer.length; i++) {
    buffer[i] = view[i];
  }
  return buffer;
}

We create the Buffer with the alloc method.

Then we pass the arrayByffer to the Uint8Array constructor so we can loop through it.

Then we assign all the bits to the buffer array.

Create Directory When Writing To File In Node Apps

We can use the recursive option for the fs.mkdir method to create a directory before creating the file.

For instance, we can write:

fs.mkdir('/foo/bar/file', { recursive: true }, (err) => {
  if (err) {
    console.error(err);
  }
});

We create the file with path /foo/bar/file since we have the recursive option set to true .

Also, we can use the promise version by writing:

fs.promises.mkdir('/foo/bar/file', { recursive: true }).catch(console.error);

Load External JS File in with Access to Local Variables

We can load external JS files from another file if we create a module.

For instance, we can write:

module.js

module.exports = {
  foo(bar){
    //...
  }
}

app.js

const module = require('./module');
module.foo('bar');

We export the foo function by putting it in module.exports and import it with require .

Then we call it with module.foo('bar'); .

Conclusion

We can use the crypto module to generate a hash with a timestamp and a random string.

Also, we can convert between a Node buffer and an ArrayBuffer with loops.

The serve-favicon package lets us serve favicons in our Express apps.

We can create modules to export functions that can be run from other functions.

Express routers can be nested more than one level deep.