Categories
Node.js Tips

Node.js Tips — Express Requests and Response

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Return JSON Using Node or Express

We can render a JSON response with the http package by setting the Content-Type header and the res.end method.

For instance, we can write:

const http = require('http');

const app = http.createServer((req, res) => {
  res.setHeader('Content-Type', 'application/json');
  res.end(JSON.stringify({
    a: 1
  }));
});

app.listen(3000);

We call createServer to create an HTTP server.

In the callback, we set the Content-Type response header to application/json to set the response data to JSON.

Then we call res.end with the stringified JSON to render the response.

With Express, we just have to write:

res.json({
  foo: "bar"
});

in our middleware.

res.json sets the headers and render the JSON.

Enable HTTPS with Express

We can enable HTTPS with Express by reading the certificate data and use http.createServer to create the server.

For instance, we can write;

const fs = require('fs');
const http = require('http');
const https = require('https');
const privateKey = fs.readFileSync('sslcert/server.key', 'utf8');
const certificate = fs.readFileSync('sslcert/server.crt', 'utf8');
const credentials = {
  key: privateKey,
  cert: certificate
};
const express = require('express');
const app = express();

const httpServer = http.createServer(app);
const httpsServer = https.createServer(credentials, app);
httpServer.listen(3000);
httpsServer.listen(3443);

We read the private key and certificate files from the server.

Then we pass them all in an object and that object is passed to https.createServer to create the server.

Then we call listen on the httpsServer to create the server.

We still have to create the HTTP server to listen to non-secure HTTP requests.

Fix ‘Error: request entity too large’ Error from Express

We can fix the ‘request entity too large’ error by increasing the allowed size of requests.

For instance, we can write:

const bodyParser = require('body-parser');
app.use(bodyParser.json({
  limit: '50mb'
}));
app.use(bodyParser.urlencoded({
  limit: '50mb',
  extended: true
}));

We set limit in the object that we passed in as the argument to set the max size of the request payload.

'50mb' is 50 megabytes.

Download a File from Node Server Using Express

We can send a file download response to the client with res.download .

For instance, we can write:

app.get('/download', (req, res) => {
  const file =  path.join(__dirname, '/upload-folder/movie.mp4');
  res.download(file);
});

The res.download requires the absolute path to the file, so we need to combine __dirname with the rest of the path.

res.download will set the Content-Disposition and other headers automatically and send the file to the client.

Server Static Files with Express

We can serve static files with Express using the express.static middleware.

For instance, we can write:

app.use(express.static(path.join(__dirname, '/public')));

We serve the folder we want to serve as the static folder with the absolute path.

express.static will expose the folder to the client.

Get the JSON POST Data in an Express Application

We can get the JSON data from a POST request by using the req.body property.

For instance, we can write:

const express = require('express');
const bodyParser = require('body-parser');
const app = express.createServer();

app.use(`bodyParser`.`json`());

app.post('/', (req, res) => {
  console.log(req.body);
  response.send(req.body);
});

app.listen(3000);

req.body has the JSON request payload.

The req.body property is available if we ran the bodyParser.json() middleware.

We did that with:

app.use(bodyParser.json());

Remove Documents Using Node.js Mongoose

We can remove a document with Mongoose by using the remove method on the model class.

For example, we can write:

Model.remove({
  _id: id
}, (err) => {
  if (err) {
    console.log(err);
  }
});

Get Uploaded Files with Express

To get uploaded files with Express, we can use the express-fileupload package.

For example, we can write:

const fileupload = require("express-fileupload");
app.use(fileupload());

app.post("/upload", (req, res) => {
  if (!req.files) {
    res.send("File not found");
    return;
  }
  const file = req.files.photo;
  res.send("File Uploaded");
});

We used the fileupload middleware from the express-fileupload package.

To use it, we write:

app.use(fileupload());

The req.files should be available when we upload one or more files.

We can get the files by using the req.files object.

photo is the field name of the file input.

We can replace that with the correct name.

Conclusion

There are a few ways to render JSON response to the client.

We can get file uploads with a middleware package.

Also, we can return a file download response with res.download .

We’ve to use body-parser to parse JSON request payloads.

Categories
Node.js Tips

Node.js Tips — Config Files, Validating Strings, Multiple Streams,

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Validation Library for Node.js

To validate data in a Node app, we can use a library to make our lives easier.

For instance, we can use the validator package.

We can use it by writing”

const check = require('validator').check;
const sanitize = require('validator').sanitize;

check('test@email.com').len(6, 64).isEmail();
check('abc').isInt();

const int = sanitize('0123').toInt();
const bool = sanitize('true').toBoolean();

We check is the email is between the length of 6 and 64 with the len method.

Then we called the isEmail method to check if it’s an email.

check checks a string with something validator methods.

sanitize cleans up the string.

isInt checks for integers.

If it’s not valid, then an exception would be thrown.

toInt converts a string to an int.

toBoolean converts a string to a boolean.

Piping the Same Readable Stream into Multiple Writable Targets

We can pipe the same readable stream into multiple writable targets.

For instance, we can write:

const spawn = require('child_process').spawn;
const PassThrough = require('stream').PassThrough;

const a = spawn('ls');
const b = new PassThrough();
const c = new PassThrough();

a.stdout.pipe(b);
a.stdout.pipe(c);

let output = '';

b.on('data', (chunk) => {
  output += chunk.length;
});

b.on('end', () => {
  console.log(count);
  c.pipe(process.stdout);
});

We can pipe the output of the ls shell command that’s run with the spawn method with the PassThrough instances.

We get the stdout property, which is a readable stream, and call pipe on it to pipe it to the writable streams which we created with the PassThrough constructor.

Then we listen to the streams by adding listeners to the data and end methods.

.rc Configuration Files

Anyone can make .rc files to create their own configuration file.

We can make a .rc file that has JSON for example.

For instance, we can write:

const fs = require('fs');
fs.readFile('./.barrc', 'utf8', (err, data) => {
  if (err) {
    return console.error(err);
  }
  console.log(JSON.parse(data));
})

We call readFile to read the file, then we can parse the JSON that’s stored in it.

It can also hold YAML or any other standard text format.

Determine if a String is a MongoDB ObjectID

We can determine if a string is a MongoDB Object ID by using the ObjectId.isValid method.

For example, we can write:

const ObjectId = require('mongoose').Types.ObjectId;
ObjectId.isValid('4738');

It checks if a string is a 12 character long string or if it’s in the correct format.

Perform a Full-Text Search in MongoDB and Mongoose

We can perform a full-text search in MongoDB by creating an index.

Then we can use the find method to search for the field we want.

For example, we can write:

const schema = new Schema({
  name: String,
  email: String,
  address: {
    street: String,
  }
});
schema.index({name: 'text', 'address.street': 'text'});

We can also include all fields in our index by writing:

schema.index({'$**': 'text'});

Then we can call find by writing:

Person.find({ $text: { $search: search }})
  .skip(20)
  .limit(20)
  .exec((err, docs) => {
    //...
  });

Update a Line Rather than Create a New Line with console.log

We can clear the existing lines and then write the new line with the process module.

For instance, we can write:

process.stdout.write("Hello, World");
process.stdout.clearLine();
process.stdout.cursorTo(0);
process.stdout.write("n");

We called write with some text to display it on the screen.

Then we called clearLine to clear the line.

Then we called cursorTo with 0 to move the cursor back to the left side.

Then we write the end of line character on the screen.

Write Objects into File with Node.js

To write objects into a file in a Node app, we can use writeFileSync to write it.

We can write an array by writing:

fs.writeFileSync('./data.json', arr.join(',') , 'utf-8');

We called join to combine the entries into a string.

Then writeFileSync would write the string to data.json .

To write an object’s content into a file, we can use util.inspect to convert it into a formatted string.

Then we write that to a file.

For example, we can write:

const util = require('util');
fs.writeFileSync('./data.json', util.inspect(obj) , 'utf-8');

Conclusion

We can write an object into a file.

Validation strings are easy if we use a library.

We can pipe readable streams into multiple write streams.

Also, we can make our own .rc configuration files in whatever format we want.

Categories
Node.js Tips

Node.js Tips — Upsert Data, Ciphers, and Parsing Directories

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Difference Between response.send and response.write in Express

res.send is called only once and it sends data in the chunk of the buffer size to the client.

Then it comes back and sends more data to the client with the buffer size until the sending is done.

res.write is called multiple times followed by res.end . It creates a buffer with the size based on the whole piece of data.

Then it sends that over HTTP. Therefore, it’s faster in case of huge amount of data.

Create Document if not Exists and Update the Existing Document Otherwise with Mongoose

We can create document if it doesn’t exist and do update otherwise with the findOneAndUpdate method.

For instance, we can write:

const query = {};
const update = { expire: new Date() };
const options = {
  upsert: true
};


Model.findOneAndUpdate(query, update, options, (error, result) => {
  if (error) return;
});

We pass in an options object with the upsert property set to true .

update has the data that we want to add or update.

query has the query for getting the data.

The callback is called when the operation is finished.

result has the data for the written file.

error has the error object.

Convert a Directory Structure in the Filesystem to JSON with Node.js

We can use the directory-package to read the structure of the filesystem into JSON.

To install it, we run:

npm i directory-tree

Then we can use it by writing:

const dirTree = require("directory-tree");
const tree = dirTree("/foo/bar");

We use the directory-tree package’s dirTree function.

Then we pass in the path into the dirTree function.

We can restrict the file types that we want to enumerate by writing:

const dirTree = require("directory-tree");
const filteredTree = dirTree("/some/path", { extensions: /.txt/ });

We use the extension regex to do the filtering.

Listen on HTTP and HTTPS for a Single Express App

We can listen to HTTP and HTTPS at the same time by using the http and https packages and calling the createServer method on both.

For instance, we can write:

const express = require('express');
const https = require('https');
const http = require('http');
const fs = require('fs');
const app = express();

const options = {
  key: fs.readFileSync('/path/to/key.pem'),
  cert: fs.readFileSync('/path/to/cert.pem'),
  ca: fs.readFileSync('/path/to/ca.pem')
};

http.createServer(app).listen(80);
https.createServer(options, app).listen(443);

We read the certificate files and call createServer with them to listen to HTTPS requests.

And we also use http to listen to regular HTTP requests as usual.

Use Cases for process.nextTick in Node.js

We can use process.nextTick to put a callback into a queue.

It’s the same as using setTimeout to delay code execution.

For instance, we can write:

process.nextTick(() => {
  console.log('hello');
});

Encrypting Data that Needs to be Decrypted

We can encrypt data that needs to be decrypted with the crypto module.

It has the createCipher method to lets us create a cipher that can be deciphered.

For instance, we can write:

const crypto = require('crypto');

const algorithm = 'aes256';
const key = 'sercetkey';
const text = 'hello world';

const cipher = crypto.createCipher(algorithm, key);
const encrypted = cipher.update(text, 'utf8', 'hex') + cipher.final('hex');

const decipher = crypto.createDecipher(algorithm, key);
const decrypted = decipher.update(encrypted, 'hex', 'utf8') + decipher.final('utf8');

We called createCipher with the algorithm and key to create the cipher.

Then to encrypt the text , we call cipher.update and concatenate it with cipher.final('hex') .

cipher.final can only be called once so that the text can only be encrypted once.

When we wan to decipher the text, we call the createDecipher method with the same algorithm and key .

Then we can use decipher.update to decipher the encrypted text and concatenate that with decipher.find('hex') to decrypt the text.

decipher.final can only be called once so that the encrypted text can only be decrypted once.

Conclusion

We can create ciphers and decipher text with the crypto module.

res.send and res.write are different in Express. One is good for small data and one is good for bigger ones.

We can upsert data with Mongoose.

Directory structures can be read into JSON.

Categories
Node.js Tips

Node.js Tips — Promises, Socket.io, and Passing Data in Express

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Share Constants in NodeJS Modules

We can share constants in Node modules by exporting a frozen object.

For instance, we can write:

constants.js

module.exports = Object.freeze({
  FOO: 'foo',
  BAR: '123'
});

We call the Object.freeze method to freeze the object with the constants.

Then in another module, we can require by writing:

const constants = require('./constants');

Send Responses to All Clients Except Sender with Socket.io

There are multiple ways to send a response to all clients except the sender with Socket.io.

Yo send to the sender only, we can write:

socket.emit('message', "hello");

To send to all clients except the sender, we can write:

socket.broadcast.emit('message', "hello");

To send to all clients in a room except for the sender, we can write:

socket.broadcast.to('chatRoom').emit('message', 'hello');

To send to all clients including the sender, we write:

io.emit('message', "hello");

To send to all clients in a room including the sender, we can write:

io.in('chatRoom').emit('message', 'hello');

To send to the sender only in a room, we can write:

`socket.to('`chatRoom`').emit('message', 'hello');`

To send to all clients in a namespace, we can write:

io.of('namespace').emit('message', 'hello');

To send to a socket ID, we can write:

socket.broadcast.to(socketId).emit('message', 'hello');

Reasons to use Promise Libraries like Q or BlueBird

We can use promise libraries like Q or Bluebird to convert existing async functions to return promises.

For instance, we can write:

const Promise = require('bluebird');
const fs = Promise.promisifyAll(require('fs'));

fs.readFileAsync('foo.text').then((data) => {
   //...
});

We used Bluebird to convert everything in the fs module to promises.

So we can use methods like readFileAsync to read files asynchronously.

There are also methods exclusive to Bluebird that aren’t available with the ES6 Promise constructor.

Promise.promisify() converts Node callbacks to promises.

Promise.map() maps arrays to promises.

Promise.reduce() lets us map values to promises and invoke them sequentially and reduce the resolved values to a single value.

Promise.mapSeries() takes values, map them to promises, and run them in series.

Then we get the values of them at the end.

Promise.delay() lets us dealt a promise by a given number of milliseconds.

Node.js Global Variables

We can add a property to global to add a global variable.

For instance, we can write:

global._ = `require('underscore');`

Node.js Get File Extension

We can get the file extension of a file name with the extname method.

For instance, we can write:

const path = require('path');

path.extname('index.html');

Run Function in Script from Command Line

We can run a function in a script from the command line with node -e .

If we have a module, we can write:

hi.js

module.exports.hello = () => {
  console.log('hello');
};

For instance, we can write:

node -e 'require("./hi").hello()'

Difference Between res.send and res.json in Express.js

res.send and res.json are the same when arrays or objects are passed in.

However, if non-objects are passed in, then res.json also converts those values to something that can be returned.

res.json calls res.send in the end.

Passing Variables to the Next Middleware Using next() in Express.js

We add a property to the req variable to pass data to the next middleware.

For instance, we can write:

req.foo = 'bar';

Then we can access req.foo in the next middleware.

Also, we can add a property to the res.locals property to do the same thing.

Use Node.js Crypto to Create an HMAC-SHA1 Hash

We can use the Node crypto module to create an HMAC-SHA1 hash.

For instance, we can write:

const crypto = require('crypto')

const text = 'hello world';
const key = 'secret';

crypto.createHmac('sha1', key)
  .update(text)
  .digest('hex')

We call the createHmac method to create a hash from the key .

And then we hash the text with it with update and convert it to hex with update .

Conclusion

We can use the crypto module to hash text.

There are many ways to send data with socket.io.

We can get the extension of a file name with the path module.

Promise libraries are still useful.

global is the global object in Node.

Categories
Node.js Tips

Node.js Tips — Mongoose Find, Script, Shell Commands, and AWS Region

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Pass Variable from Jade Template File to a Script File

To pass a variable in a Jade template to a script, we can interpolate the script code.

For instance, we can write:

app.get('/', (req, res) => {
  res.render( 'index', { layout: false, name: 'bar' } );
});

index.jade

script.
  name="#{name}";

We use the script tag and interpolated the name variable in index.jad .

Mongoose Select a Specific Field with find

We can select a specific field with find by passing the field name as the argument.

For instance, we can write:

app.get('/name', (req, res, next) => {
  dbSchemas.Foo.find({}, 'name', (err, val) => {
    if(err) {
      return next(err);
    }
    res.send(val);
  });
});

We select a Foo document’s name field by passing the field name as the 2nd argument.

Difference Between async.waterfall and async.series

async.waterfall lets us pas results to the next function.

async.series pass all results to the final callback.

Write Asynchronous Functions for Node.js

We write callbacks that have the err for the error object and the result aS parameters.

For instance, we can write:

file.read('/foo.txt', (err, result) => {
  if (err) {
    return console.log(err);
  }
  doSomething(result);
});

The arguments before the callback is anything we want to use with read .

And the callback is a Node style callback with the err object as the first parameter.

result has the computed results in the 2nd parameter.

Configuring Region in Node.js AWS SDK

We can configure the AWS region with the AWS.config.update method.

For example, we can write:

const AWS = require('aws-sdk');
AWS.config.update({ region: 'us-east-1' });

to configure the region.

Pass Parameter to a Promise Function

To pass parameters to a promise function, we create a function that takes the parameter we want and returns the promise inside it.

For instance, we can write:

const login = (username, password) => {
  return new Promise((resolve, reject) => {
    //...
    if (success) {
      resolve("success");
    } else {
      reject(new Error("error"));
    }
  });
}

We return the promise by using the Promise constructor.

username and password are the parameters we passed in.

Then we can use it by writing:

login(username, password)
  .then(result => {
    console.log(result);
  })

We use then to get the value we passed into resolve .

Shell Command Execution with Node.js

To run shell commands in Node apps, we can use the child_process module.

For instance, we can write:

const spawn = require('child_process').spawn;
const child = spawn('ls', ['-l']);
let resp = "";

child.stdout.on('data', (buffer) => {
  resp += buffer.toString()
});

child.stdout.on('end', () => {
  console.log(resp)
});

We use the child_process ‘s spawn method.

Then we pass in the command as the first argument to spawn.

And the command line arguments as the 2nd argument.

Then we can watch for the results with the child.stdout.on method.

We listen to the 'data' event to get the output and concatenate the parts that are sent.

And we listen to 'end' to get the whole result when it ends.

Monitor Memory Usage of Node.js Apps

We can use the node-memwatch package to watch for memory usage of a Node app.

We can install it by running:

npm install memwatch

Then we can use it by writing:

const memwatch = require('memwatch');
memwatch.on('leak', (info) => {
  //...
});

We watch for memory leaks with the on method.

info has information about the leak.

It has something like:

{
  start: Fri, 29 Jun 2020 15:12:13 GMT,
  end: Fri, 29 Jun 2020 16:12:33 GMT,
  growth: 67984,
  reason: 'heap growth over 5 consecutive GCs (20s) - 11.67 mb/hr'
}

It’ll show us the growth in memory usage.

There also methods for watching heap usage and diffing memory usage in different times.

The built-in process module has the memoryUsage method to get the memory usage of our app.

For instance, we can write:

process.memoryUsage();

to get the memory usage.

Conclusion

We can watch for memory usage with the process module or 3rd party modules.

To pass data to a script tag in a Jade template, we can interpolate the code.

We can return a promise in a function if we want a promise with our own parameters.

Mongoose can be used to find data with a given field.