Categories
Node.js Tips

Node.js Tips — Config Files, Validating Strings, Multiple Streams,

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Validation Library for Node.js

To validate data in a Node app, we can use a library to make our lives easier.

For instance, we can use the validator package.

We can use it by writing”

const check = require('validator').check;
const sanitize = require('validator').sanitize;

check('test@email.com').len(6, 64).isEmail();
check('abc').isInt();

const int = sanitize('0123').toInt();
const bool = sanitize('true').toBoolean();

We check is the email is between the length of 6 and 64 with the len method.

Then we called the isEmail method to check if it’s an email.

check checks a string with something validator methods.

sanitize cleans up the string.

isInt checks for integers.

If it’s not valid, then an exception would be thrown.

toInt converts a string to an int.

toBoolean converts a string to a boolean.

Piping the Same Readable Stream into Multiple Writable Targets

We can pipe the same readable stream into multiple writable targets.

For instance, we can write:

const spawn = require('child_process').spawn;
const PassThrough = require('stream').PassThrough;

const a = spawn('ls');
const b = new PassThrough();
const c = new PassThrough();

a.stdout.pipe(b);
a.stdout.pipe(c);

let output = '';

b.on('data', (chunk) => {
  output += chunk.length;
});

b.on('end', () => {
  console.log(count);
  c.pipe(process.stdout);
});

We can pipe the output of the ls shell command that’s run with the spawn method with the PassThrough instances.

We get the stdout property, which is a readable stream, and call pipe on it to pipe it to the writable streams which we created with the PassThrough constructor.

Then we listen to the streams by adding listeners to the data and end methods.

.rc Configuration Files

Anyone can make .rc files to create their own configuration file.

We can make a .rc file that has JSON for example.

For instance, we can write:

const fs = require('fs');
fs.readFile('./.barrc', 'utf8', (err, data) => {
  if (err) {
    return console.error(err);
  }
  console.log(JSON.parse(data));
})

We call readFile to read the file, then we can parse the JSON that’s stored in it.

It can also hold YAML or any other standard text format.

Determine if a String is a MongoDB ObjectID

We can determine if a string is a MongoDB Object ID by using the ObjectId.isValid method.

For example, we can write:

const ObjectId = require('mongoose').Types.ObjectId;
ObjectId.isValid('4738');

It checks if a string is a 12 character long string or if it’s in the correct format.

Perform a Full-Text Search in MongoDB and Mongoose

We can perform a full-text search in MongoDB by creating an index.

Then we can use the find method to search for the field we want.

For example, we can write:

const schema = new Schema({
  name: String,
  email: String,
  address: {
    street: String,
  }
});
schema.index({name: 'text', 'address.street': 'text'});

We can also include all fields in our index by writing:

schema.index({'$**': 'text'});

Then we can call find by writing:

Person.find({ $text: { $search: search }})
  .skip(20)
  .limit(20)
  .exec((err, docs) => {
    //...
  });

Update a Line Rather than Create a New Line with console.log

We can clear the existing lines and then write the new line with the process module.

For instance, we can write:

process.stdout.write("Hello, World");
process.stdout.clearLine();
process.stdout.cursorTo(0);
process.stdout.write("n");

We called write with some text to display it on the screen.

Then we called clearLine to clear the line.

Then we called cursorTo with 0 to move the cursor back to the left side.

Then we write the end of line character on the screen.

Write Objects into File with Node.js

To write objects into a file in a Node app, we can use writeFileSync to write it.

We can write an array by writing:

fs.writeFileSync('./data.json', arr.join(',') , 'utf-8');

We called join to combine the entries into a string.

Then writeFileSync would write the string to data.json .

To write an object’s content into a file, we can use util.inspect to convert it into a formatted string.

Then we write that to a file.

For example, we can write:

const util = require('util');
fs.writeFileSync('./data.json', util.inspect(obj) , 'utf-8');

Conclusion

We can write an object into a file.

Validation strings are easy if we use a library.

We can pipe readable streams into multiple write streams.

Also, we can make our own .rc configuration files in whatever format we want.

Categories
Node.js Tips

Node.js Tips — Upsert Data, Ciphers, and Parsing Directories

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Difference Between response.send and response.write in Express

res.send is called only once and it sends data in the chunk of the buffer size to the client.

Then it comes back and sends more data to the client with the buffer size until the sending is done.

res.write is called multiple times followed by res.end . It creates a buffer with the size based on the whole piece of data.

Then it sends that over HTTP. Therefore, it’s faster in case of huge amount of data.

Create Document if not Exists and Update the Existing Document Otherwise with Mongoose

We can create document if it doesn’t exist and do update otherwise with the findOneAndUpdate method.

For instance, we can write:

const query = {};
const update = { expire: new Date() };
const options = {
  upsert: true
};


Model.findOneAndUpdate(query, update, options, (error, result) => {
  if (error) return;
});

We pass in an options object with the upsert property set to true .

update has the data that we want to add or update.

query has the query for getting the data.

The callback is called when the operation is finished.

result has the data for the written file.

error has the error object.

Convert a Directory Structure in the Filesystem to JSON with Node.js

We can use the directory-package to read the structure of the filesystem into JSON.

To install it, we run:

npm i directory-tree

Then we can use it by writing:

const dirTree = require("directory-tree");
const tree = dirTree("/foo/bar");

We use the directory-tree package’s dirTree function.

Then we pass in the path into the dirTree function.

We can restrict the file types that we want to enumerate by writing:

const dirTree = require("directory-tree");
const filteredTree = dirTree("/some/path", { extensions: /.txt/ });

We use the extension regex to do the filtering.

Listen on HTTP and HTTPS for a Single Express App

We can listen to HTTP and HTTPS at the same time by using the http and https packages and calling the createServer method on both.

For instance, we can write:

const express = require('express');
const https = require('https');
const http = require('http');
const fs = require('fs');
const app = express();

const options = {
  key: fs.readFileSync('/path/to/key.pem'),
  cert: fs.readFileSync('/path/to/cert.pem'),
  ca: fs.readFileSync('/path/to/ca.pem')
};

http.createServer(app).listen(80);
https.createServer(options, app).listen(443);

We read the certificate files and call createServer with them to listen to HTTPS requests.

And we also use http to listen to regular HTTP requests as usual.

Use Cases for process.nextTick in Node.js

We can use process.nextTick to put a callback into a queue.

It’s the same as using setTimeout to delay code execution.

For instance, we can write:

process.nextTick(() => {
  console.log('hello');
});

Encrypting Data that Needs to be Decrypted

We can encrypt data that needs to be decrypted with the crypto module.

It has the createCipher method to lets us create a cipher that can be deciphered.

For instance, we can write:

const crypto = require('crypto');

const algorithm = 'aes256';
const key = 'sercetkey';
const text = 'hello world';

const cipher = crypto.createCipher(algorithm, key);
const encrypted = cipher.update(text, 'utf8', 'hex') + cipher.final('hex');

const decipher = crypto.createDecipher(algorithm, key);
const decrypted = decipher.update(encrypted, 'hex', 'utf8') + decipher.final('utf8');

We called createCipher with the algorithm and key to create the cipher.

Then to encrypt the text , we call cipher.update and concatenate it with cipher.final('hex') .

cipher.final can only be called once so that the text can only be encrypted once.

When we wan to decipher the text, we call the createDecipher method with the same algorithm and key .

Then we can use decipher.update to decipher the encrypted text and concatenate that with decipher.find('hex') to decrypt the text.

decipher.final can only be called once so that the encrypted text can only be decrypted once.

Conclusion

We can create ciphers and decipher text with the crypto module.

res.send and res.write are different in Express. One is good for small data and one is good for bigger ones.

We can upsert data with Mongoose.

Directory structures can be read into JSON.

Categories
Node.js Tips

Node.js Tips — Promises, Socket.io, and Passing Data in Express

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Share Constants in NodeJS Modules

We can share constants in Node modules by exporting a frozen object.

For instance, we can write:

constants.js

module.exports = Object.freeze({
  FOO: 'foo',
  BAR: '123'
});

We call the Object.freeze method to freeze the object with the constants.

Then in another module, we can require by writing:

const constants = require('./constants');

Send Responses to All Clients Except Sender with Socket.io

There are multiple ways to send a response to all clients except the sender with Socket.io.

Yo send to the sender only, we can write:

socket.emit('message', "hello");

To send to all clients except the sender, we can write:

socket.broadcast.emit('message', "hello");

To send to all clients in a room except for the sender, we can write:

socket.broadcast.to('chatRoom').emit('message', 'hello');

To send to all clients including the sender, we write:

io.emit('message', "hello");

To send to all clients in a room including the sender, we can write:

io.in('chatRoom').emit('message', 'hello');

To send to the sender only in a room, we can write:

`socket.to('`chatRoom`').emit('message', 'hello');`

To send to all clients in a namespace, we can write:

io.of('namespace').emit('message', 'hello');

To send to a socket ID, we can write:

socket.broadcast.to(socketId).emit('message', 'hello');

Reasons to use Promise Libraries like Q or BlueBird

We can use promise libraries like Q or Bluebird to convert existing async functions to return promises.

For instance, we can write:

const Promise = require('bluebird');
const fs = Promise.promisifyAll(require('fs'));

fs.readFileAsync('foo.text').then((data) => {
   //...
});

We used Bluebird to convert everything in the fs module to promises.

So we can use methods like readFileAsync to read files asynchronously.

There are also methods exclusive to Bluebird that aren’t available with the ES6 Promise constructor.

Promise.promisify() converts Node callbacks to promises.

Promise.map() maps arrays to promises.

Promise.reduce() lets us map values to promises and invoke them sequentially and reduce the resolved values to a single value.

Promise.mapSeries() takes values, map them to promises, and run them in series.

Then we get the values of them at the end.

Promise.delay() lets us dealt a promise by a given number of milliseconds.

Node.js Global Variables

We can add a property to global to add a global variable.

For instance, we can write:

global._ = `require('underscore');`

Node.js Get File Extension

We can get the file extension of a file name with the extname method.

For instance, we can write:

const path = require('path');

path.extname('index.html');

Run Function in Script from Command Line

We can run a function in a script from the command line with node -e .

If we have a module, we can write:

hi.js

module.exports.hello = () => {
  console.log('hello');
};

For instance, we can write:

node -e 'require("./hi").hello()'

Difference Between res.send and res.json in Express.js

res.send and res.json are the same when arrays or objects are passed in.

However, if non-objects are passed in, then res.json also converts those values to something that can be returned.

res.json calls res.send in the end.

Passing Variables to the Next Middleware Using next() in Express.js

We add a property to the req variable to pass data to the next middleware.

For instance, we can write:

req.foo = 'bar';

Then we can access req.foo in the next middleware.

Also, we can add a property to the res.locals property to do the same thing.

Use Node.js Crypto to Create an HMAC-SHA1 Hash

We can use the Node crypto module to create an HMAC-SHA1 hash.

For instance, we can write:

const crypto = require('crypto')

const text = 'hello world';
const key = 'secret';

crypto.createHmac('sha1', key)
  .update(text)
  .digest('hex')

We call the createHmac method to create a hash from the key .

And then we hash the text with it with update and convert it to hex with update .

Conclusion

We can use the crypto module to hash text.

There are many ways to send data with socket.io.

We can get the extension of a file name with the path module.

Promise libraries are still useful.

global is the global object in Node.

Categories
Node.js Tips

Node.js Tips — Mongoose Find, Script, Shell Commands, and AWS Region

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Pass Variable from Jade Template File to a Script File

To pass a variable in a Jade template to a script, we can interpolate the script code.

For instance, we can write:

app.get('/', (req, res) => {
  res.render( 'index', { layout: false, name: 'bar' } );
});

index.jade

script.
  name="#{name}";

We use the script tag and interpolated the name variable in index.jad .

Mongoose Select a Specific Field with find

We can select a specific field with find by passing the field name as the argument.

For instance, we can write:

app.get('/name', (req, res, next) => {
  dbSchemas.Foo.find({}, 'name', (err, val) => {
    if(err) {
      return next(err);
    }
    res.send(val);
  });
});

We select a Foo document’s name field by passing the field name as the 2nd argument.

Difference Between async.waterfall and async.series

async.waterfall lets us pas results to the next function.

async.series pass all results to the final callback.

Write Asynchronous Functions for Node.js

We write callbacks that have the err for the error object and the result aS parameters.

For instance, we can write:

file.read('/foo.txt', (err, result) => {
  if (err) {
    return console.log(err);
  }
  doSomething(result);
});

The arguments before the callback is anything we want to use with read .

And the callback is a Node style callback with the err object as the first parameter.

result has the computed results in the 2nd parameter.

Configuring Region in Node.js AWS SDK

We can configure the AWS region with the AWS.config.update method.

For example, we can write:

const AWS = require('aws-sdk');
AWS.config.update({ region: 'us-east-1' });

to configure the region.

Pass Parameter to a Promise Function

To pass parameters to a promise function, we create a function that takes the parameter we want and returns the promise inside it.

For instance, we can write:

const login = (username, password) => {
  return new Promise((resolve, reject) => {
    //...
    if (success) {
      resolve("success");
    } else {
      reject(new Error("error"));
    }
  });
}

We return the promise by using the Promise constructor.

username and password are the parameters we passed in.

Then we can use it by writing:

login(username, password)
  .then(result => {
    console.log(result);
  })

We use then to get the value we passed into resolve .

Shell Command Execution with Node.js

To run shell commands in Node apps, we can use the child_process module.

For instance, we can write:

const spawn = require('child_process').spawn;
const child = spawn('ls', ['-l']);
let resp = "";

child.stdout.on('data', (buffer) => {
  resp += buffer.toString()
});

child.stdout.on('end', () => {
  console.log(resp)
});

We use the child_process ‘s spawn method.

Then we pass in the command as the first argument to spawn.

And the command line arguments as the 2nd argument.

Then we can watch for the results with the child.stdout.on method.

We listen to the 'data' event to get the output and concatenate the parts that are sent.

And we listen to 'end' to get the whole result when it ends.

Monitor Memory Usage of Node.js Apps

We can use the node-memwatch package to watch for memory usage of a Node app.

We can install it by running:

npm install memwatch

Then we can use it by writing:

const memwatch = require('memwatch');
memwatch.on('leak', (info) => {
  //...
});

We watch for memory leaks with the on method.

info has information about the leak.

It has something like:

{
  start: Fri, 29 Jun 2020 15:12:13 GMT,
  end: Fri, 29 Jun 2020 16:12:33 GMT,
  growth: 67984,
  reason: 'heap growth over 5 consecutive GCs (20s) - 11.67 mb/hr'
}

It’ll show us the growth in memory usage.

There also methods for watching heap usage and diffing memory usage in different times.

The built-in process module has the memoryUsage method to get the memory usage of our app.

For instance, we can write:

process.memoryUsage();

to get the memory usage.

Conclusion

We can watch for memory usage with the process module or 3rd party modules.

To pass data to a script tag in a Jade template, we can interpolate the code.

We can return a promise in a function if we want a promise with our own parameters.

Mongoose can be used to find data with a given field.

Categories
Node.js Tips

Node.js Tips — Express Errors, URLs, Parameters and Writing Files

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Render Error When Page Not Found

We can render an error message with res.render .

For instance, we can write:

app.use((req, res, next) => {
  res.render('404', { status: 404, url: req.url });
});

to render a 404 error.

'404' is the template name.

the object has the stuff we want to display on the 404 template.

We can also make a generic error handling middleware by writing;

app.use((err, req, res, next) => {
  res.render('500', {
    status: err.status || 500,
    error: err
  });
});

We have err as the first parameter.

We can get the error status code with err.status and we can pass the object off to the 500 page template.

Writing Files in Node.js

We can write files with Node.js with the writeFile , writeFileSync , or a write stream.

For instance, we can write:

const fs = require('fs');

fs.writeFile("/tmp/foo.txt", "hello world", (err) => {
  if (err) {
    return console.log(err);
  }
  console.log("The file was saved!");
});

Or we can writ:

const fs = require('fs');

fs.writeFileSync('/tmp/foo.txt', 'hello world');

They both take the file path and the content as the first 2 arguments.

writeFile also takes a callback that’s called with err if it exists.

To create a write stream, we can write:

const fs = require('fs');
const  stream = fs.createWriteStream("/tmp/foo.txt");
stream.once('open', (fd) => {
  stream.write("hellon");
  stream.write("worldn");
  stream.end();
});

We call createWriteStream to with the file path to write to.

Then we open the stream with stream.once .

Then we start writing with stream.write .

We’ve to close the stream with stream.end .

Get Query String Variables in Express

We can get query string variables with the req.query property.

It has the query string parsed into an object of the key-value pairs.

For instance, we can write:

const express = require('express');
const app = express();

app.get('/', (req, res) => {
  res.send(req.query.id);
});

app.listen(3000);

If we have a query string like ?id=10 , then req.query.id is 10.

Retrieve POST Query Parameters in an Express App

We can get POST query parameters with the body-parser package.

To use it, we’ve to install it by running:

npm install --save body-parser

Then we can write:

const bodyParser = require('body-parser');
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({
  extended: true
}));

We import the module so that we can use it.

Then we call bodyParser.json() which returns a middleware to use the middleware.

urlencoded parses URL encoded bodies.

extended set to true means that we can parse JSON bodies in the URL encoded string with it.

Adding CORS Headers to an OPTIONS Route

We can do all the CORDS stuff easily with our own middleware.

For instance, we can write:

const allowCrossDomain = (req, res, next) => {
  res.header('Access-Control-Allow-Origin', 'example.com');
  res.header('Access-Control-Allow-Methods', 'GET,PUT,POST,DELETE');
  res.header('Access-Control-Allow-Headers', 'Content-Type');
  next();
}

The Access-Control-Allow-Origin header lets the browser know what domains can access the API.

Access-Control-Allow-Methods tells the browser what kind of request it’s allowed to be called.

Access-Control-Allow-Headers tells the browser what kind of content it can send.

An easier way is to use the cors middleware.

To use it, we can run:

npm install cors --save

to install it.

Then we can write:

const cors = require('cors');
const express = require('express');
const app = express();
app.use(cors());
app.options('*', cors());

We use the cors middleware on the whole app with:

app.use(cors());

and we allow the OPTIONs route to be called with:

app.options('*', cors());

How to Get the Full URL in Express

We can use the req.protocol property to get the protocol, which is the http or https part.

Also, we can get the hostname with req.get('host') .

And we get the full URL with the req.originalUrl .

To parse combine the parts into the full URL, we can use the url module.

For instance, we can write:

const requrl = url.format({
  protocol: req.protocol,
  host: req.get('host'),
  pathname: req.originalUrl,
});

We can do that inside the route handler or anywhere else.

Conclusion

We can render an error page by passing in a template and get the status or error we want to display.

The url module can combine the URL parts together.

There are many ways to write files.

CORS can be done by setting a few response headers or using the cors middleware with Express.