Categories
Node.js Tips

Node.js Tips — Timestamp, Cookies, Send File Responses and Fetch

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Fix ‘ReferenceError: fetch is not defined’ Error

The Fetch API isn’t implemented in Node.

If we want to use it, we can use the node-fetch library.

To install it, we run:

npm i node-fetch --save

Then we can use it by writing:

const fetch = require("node-fetch");

There’s also the cross-fetch library.

We can install it by running:

npm install --save cross-fetch

Then we can write:

import fetch from 'cross-fetch';

fetch('https//example.com')
  .then(res => {
    if (res.status >= 400) {
      throw new Error("error");
    }
  })

Avoid Long Nesting of Asynchronous Functions in Node.js

We can reorganize our nested async functions by rewriting it as multiple functions.

For example, we can write:

http.createServer((req, res) => {
  getSomeData(client, (someData) => {
    getMoreData(client, function(moreData) => {
         //
      });
   });
});

to:

const moreDataParser = (moreData) => {
   // date parsing logic
};

const dataParser = (data) => {
  getMoreData(client, moreDataParser);
};

const callback = (req, res) => {
  getSomeData(client, dataParser);
};

http.createServer(callback);

We move the anonymous functions into their own named functions so we can call them one by one.

Use of the “./bin/www” in Express 4.x

./bin/www is the file with the entry point of the Express app.

In the start script, we have:

"scripts": {
  "start": "node ./bin/www",
}

to start the app.

Get and Set a Single Cookie with Node.js HTTP Server

If we’re using the http module to create a server, we can parse a cookie by splitting the string .

For instance, we can write:

const http = require('http');

const parseCookies = (cookieStr) => {
  const list = {};
  const cookies = cookieStr && cookieStr.split(';');
  for (const cookie of cookies) {
    const [key, value] = cookie.split('=');
    list[key.trim()] = decodeURI(value);
  });
  return list;
}

http.createServer((request, response) => {
  const cookies = parseCookies(request.headers.cookie);
  //...
}).listen(8888);

We get the cookie string with request.headers.cookie .

Then we pass that into the parseCookie function.

It splits the cookie string by the semicolon.

Then we loop through them to split the parts by the = sign.

The left side is the key and the right is the value.

To set a cookie, we can write:

const http = require('http');

`http.createServer((request, response) => {
  ` response.writeHead(200, {
    'Set-Cookie': 'foo=bar',
    'Content-Type': 'text/plain'
  });
  response.end('hellon');`
}).listen(8888);`

We use writeHead to set headers.

'Set-Cookie' sets the response cookie.

'Content-Type' sets the content type of the response.

Then we use response.end to return the response body.

Get Request Path with the Express req Object

We can use the req.originalUrl property to get the request path of the Express req object.

Copy Folder Recursively

We can copy a folder recursively with the node-fs-extra package.

Its copy method lets us do the copying recursively.

For instance, we can write:

fs.copy('/tmp/oldDir', '/tmp/newDir', (err) => {
  if (err) {
    console.error(err);
  } else {
    console.log("success");
  }
});

The argument is the old directory path.

The 2nd is the new directory path.

The last argument is the callback that’s run when the copy operation ends.

err has the error.

Return the Current Timestamp with Moment.js

We can return the current timestamp in Moment.js with a few methods.

We can write:

const timestamp = moment();

or:

const timestamp = moment().format();

or:

const timestamp = moment().unix();

or:

`const timestamp =` moment().valueOf();

to get the timestamp.

We can also convert it to a Date instance and call getTime :

const timestamp = moment().toDate().getTime();

Fix the ‘TypeError: path must be absolute or specify root to res.sendFile’ Error

This error can be fixed if we pass in an absolute path to res.sendFile .

To do that, we write:

res.sendFile('index.html', { root: __dirname });

or:

const path = require('path');
res.sendFile(path.join(__dirname, '/index.html'));

Either way, we get the absolute path which is required by res.sendFile .

Conclusion

We’ve to parse cookies manually if we use http to create our server.

Fetch API isn’t included with Node’s standard library.

res.sendFile only takes an absolute path.

We can rewrite nested callbacks by reorganizing them into their own functions.

Moment returns the current timestamp with a few functions.

Categories
Node.js Tips

Node.js Tips — Streams, Scraping, and Promisifying Functions

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Converting a Buffer into a ReadableStream in Node.js

We can convert a buffer to a readable stream with the ReadableStreamBuffer constructor.

For instance, we can write:

const { ReadableStreamBuffer } = require('stream-buffers');
const readableStreamBuffer = new ReadableStreamBuffer({
  frequency: 10,
  chunkSize: 2048
});

readableStreamBuffer.put(aBuffer);

where aBuffer is the buffer we want to convert.

frequency is the frequency in which the chunks are pumped out.

The size can be changed in the constructor with the chunkSize property.

Scrape Web Pages in Real-Time with Node.js

We can use the cheerio library to scrape web pages in real-time.

To install it, we run:

npm install cheerio

Then we can use it by writing:

const cheerio = require('cheerio');
const $ = cheerio.load('<h1 class="title">Hello world</h1>');

$('h1.title').text('Hello James');
$('h1').addClass('welcome');

$.html();

We can get HTML’s content and manipulate it as we do with jQuery.

We can combine this with an HTTP client like Axios to get the HTML.

Then we can use Cheerio to parse its content.

For instance, we can write:

const axios = require('axios');
const cheerio = require('cheerio');

axios.get('https://example.com')
.then(({ data }) => {
  const $ = cheerio.load(data);
  const text = $('h1').text();
  console.log(text);
})

We make a GET request with Axios to a website.

Then we use cheerio to parse the data with cheerio.load .

Then we get the content of h1 with the text method.

Any selector can be used to get data.

Generate an MD5 file Hash in JavaScript

We can generate an MD5 hash with the crypto-js package.

To install it, we can run:

npm install crypto-js

Then we can write:

import MD5 from "crypto-js/md5";
const md5Hash = MD5("hello world");

to generate the hash.

Server-Side Browser Detection with Node.js

We can get the user-agent header to get the user agent string from the request.

To parse the string, we can use the ua-parser-js package.

To install it, we run:

npm install ua-parser-js

In our Express app, we can create our own middleware to check the user agent:

const UAParser = require('ua-parser-js');

const checkBrowser = (req, res, next) => {
  const parser = new UAParser();
  const ua = req.headers['user-agent'];
  const browserName = parser.setUA(ua).getBrowser().name;
  const fullBrowserVersion = parser.setUA(ua).getBrowser().version;

  console.log(browserName);
  console.log(fullBrowserVersion);
  next();
}

app.all(/*/, checkBrowser);

We get the user-agent header with req.headers[‘user-agent’] .

Then we use the UAParser constructor to parse the user agent string.

We call getBrowser to get the browser data.

name has the browser name and version has the version.

Best Way to Store Database Config in an Express App

We can store the config in a configuration file.

For instance, we can write:

const fs = require('fs');
const configPath = './config.json';
const configFile = fs.readFileSync(configPath, 'utf-8')
const parsedConfig = JSON.parse(configFile);
exports.storageConfig = parsedConfig;

We called readFileSync to read config.json .

Then we parse the data from the JSON string.

And we export the config in the last line.

Promisify Node’s child_process.exec and child_process.execFile Functions

We can convert the child_process exec and execFile methods with Bluebird.

For instance, we can write:

const util = require('util');
const exec = util.promisify(require('child_process').exec);

const listFiles = async () => {
  try {
    const { stdout, stderr } = await exec('ls');
    console.log('stdout:', stdout);
    console.log('stderr:', stderr);
  } catch (e) {
    console.error(e);
  }
}

listFiles();

We used the util library’s promisify method to convert the exec method to a promise.

Then we can call the promisified exec method with the ls command.

And then we get the full output with stdout and stderr .

stdout has the results. stderr has the error output.

Conclusion

We can convert a buffer into a readable stream.

We can scrape web pages with an HTTP client and cheerio.

crypto-js has the MD5 method to create an MD5 hash.

Methods from the child_process can be converted to promises.

We can parse the user agent string on the server-side and parse the browser version data.

Categories
Node.js Tips

Node.js Tips — Redis, Query Strings, Concurrency of Promises, SQL Injections

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Best Way to Limit Concurrency when Using ES6’s Promise.all()

To limit concurrency when using Promise.all , we can use the es6-promise-pool package.

For instance, we can write:

const promiseProducer = function * () {
  for (let count = 1; count <= 5; count++) {
    yield delayValue(count, 1000)
  }
}

const promiseIterator = generatePromises();
const pool = new PromisePool(promiseIterator, 3);

pool.start()
  .then(function () {
    console.log('Complete')
  })

We get the values for the promises in the promiseProducer generator function.

Then we call the generator function so that we get an iterator.

We then pass that into the PromisePool constructor.

3 is the number of concurrent promises allowed.

Finally, we call pool.start to invoke the promises.

Remove All Files from Directory without Removing Directory in Node.js

We can remove all files in a directory without removing the directory by using the readdir method to read the directory’s contents.

Then we can loop through each file and call unlink on them.

For instance, we can write:

const fs = require('fs');
const path = require('path');

const directory = '/foo/bar';

fs.readdir(directory, (err, files) => {
  if (err){
    return console.log(err);
  }

  for (const file of files) {
    fs.unlink(path.join(directory, file), err => {
      if (err) {
        console.log(err);
      }
    });
  }
});

We call readdir to read the directory’s content. Then we use the for-of loop to loop through the items.

Then we call unlink to remove each file.

Redis and Node.js

We can access a Redis database by using the redis library.

To install it, we run:

npm install redis

Then in our Node app, we can use the Redis client by writing:

const redis = require("redis");
const client = redis.createClient();

client.on("error", (err) => {
  console.log(err);
});

client.set("key", "val", redis.print);
client.hset("hash key", "foo", "value 1", redis.print);
client.hset(["hash key", "bar", "value 2"], redis.print);
client.hkeys("hash key", (err, replies) => {
  replies.forEach((reply, i) => {
    console.log(i, reply);
  });
  client.quit();
});

We use the redis module and call createClient to create the client.

Then we attach a listener to listen to the 'error' event.

Then we call set to set the key and value.

We can use hset to set a hash key with their own value.

redis.print prints the values.

hkeys get the hash keys and print the index and results.

Parsing Query String in Node.js

We can parse the query string in Node.jhs with the url module.

For instance, if we’re using the http module to create our server, we can write:

const http = require('http');
const url = require('url');

const server = http.createServer((request, response) => {
  const queryData = url.parse(request.url, true).query;
  response.writeHead(200, { "Content-Type": "text/plain" });

  if (queryData.name) {
    response.end(queryData.name);
  } else {
    response.end("hello worldn");
  }
});

server.listen(8000);

We call url.parse to parse the query string that’s in the URL.

The request URL is stored in the request.url property.

The parsed result is assigned to the queryData constant.

Then we parse the name query parameter if it exists.

If make a request with query string ?name=joe , then queryData.name will be 'joe' .

Preventing SQL Injection in Node.js

To prevent SQL injection in Node apps, we should use a library that lets to make parameterized queries.

For instance, with the node-mysql-native library, we can write:

const userId = 5;
const query = connection.query('SELECT * FROM users WHERE id = ?', [userId], (err, results) => {
  //...
});

The ? indicates the parameter that we can pass in. The 2nd argument has the parameters.

The callback is called when the query is done.

results has the results.

Sanitization is done by the following rules:

  • numbers are untouched
  • booleans are converted to strings
  • date objects are converted to YYYY-mm-dd:ii:ss strings
  • buffers are converted to hex strings
  • strings are escaped
  • arrays are turned into lists
  • nested arrays are turned into grouped lists
  • objects are turned into key = 'val' pairs
  • undefined or null are converted to NULL
  • NaN and Infinity are left as is. They aren’t supported by MySQL and will error out if these are present.

Conclusion

We should use a library to make database queries to prevent SQL injection.

Also, we can use a library to limit the concurrency of parallel promises.

We can use Node’s Redis library to access Redis databases.

The url package can parse query strings.

Categories
Node.js Tips

Node.js Tips — Unzipping Files, Storing Passwords, and REPLs

ind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Storing passwords with Node.js and MongoDB

We can store passwords in a MongoDB document with the bcrypt library.

For instance. we can write:

const mongoose = require('mongoose');
const Schema = mongoose.Schema;
const bcrypt = require('bcrypt');
const SALT_WORK_FACTOR = 10;

const UserSchema = new Schema({
  username: {
    type: String,
    required: true,
    index: {
      unique: true
    }
  },
  password: { type: String, required: true }
});

module.exports = mongoose.model('User', UserSchema);

We create the UserSchema with the Schema constructor.

It has the username and password string fields.

Then we can use the pre method to hash and salt the password before it’s saved.

To do that, we write:

UserSchema.pre('save', function(next) {
  const user = this;
  if (!user.isModified('password')){
    return next();
  }

  bcrypt.genSalt(SALT_WORK_FACTOR, (err, salt) => {
    if (err) {
       return next(err);
    }

    bcrypt.hash(user.password, salt, (err, hash) => {
      if (err) {
        return next(err);
      }
      user.password = hash;
      next();
    });
  });
});

We override the plain text password with the hash and salted one/

To do that, we used the genSalt method to create the salt.

Then we create the hash from the password.

We passed our generated salt into the hashmethod.

Then we set the password property to the hash .

And finally, we call next to continue with saving the user data.

Fix Protocol “https:” not supported. Expected “http:” Error

We can fix this error, which occurs when we try to make HTTPS requests with http .

We’ve to use https.get instead of http.get to make a request to a URL that starts with https .

Mongoose Connect Error Callback

We can pick up the error in the callback we pass into connect .

For example, we can write:

mongoose.connect('mongodb://localhost/db', (err) => {
  if (err) throw err;
});

err has the error object that’s set when there’s an error.

Run Some Code and Then Go Into Node REPL

We can use the repl module to start the REPL.

For instance, we can write:

const repl = require("repl");
const r = repl.start("node> ");

We call start the REPL with the command prompt of our choice.

Get the Node.js Version at Runtime

We can get the Node version at runtime with the process.version property.

To get more version information, we can use the process.versions property.

Then we get something like:

{
  node: '12.16.3',
  v8: '7.8.279.23-node.35',
  uv: '1.34.2',
  zlib: '1.2.11',
  brotli: '1.0.7',
  ares: '1.16.0',
  modules: '72',
  nghttp2: '1.40.0',
  napi: '5',
  llhttp: '2.0.4',
  http_parser: '2.9.3',
  openssl: '1.1.1g',
  cldr: '36.0',
  icu: '65.1',
  tz: '2019c',
  unicode: '12.1'
}

Unzip (Decompress) a NodeJS Request’s Module gzip Response Body

We can decompress a gzipped file by using the zlib library.

For instance, we can write:

const http = require("http"),
const zlib = require("zlib");

const getGzipped = (url, callback) => {
  const buffer = [];
  http.get(url, (res) => {
    const gunzip = zlib.createGunzip();
    res.pipe(gunzip);

    gunzip.on('data', (data) => {
      buffer.push(data.toString())
    })
    .on("end", () => {
      callback(null, buffer.join(""));
    })
    .on("error", (e) => {
      callback(e);
    })
  })
  .on('error', (e) => {
    callback(e);
  });
}

getGzipped(url, (err, data) => {
   console.log(data);
});

We created the getGzipped function by making the GET request to get a zipped file with the http.get method.

Then call createGunzip to create the gunzip object that we can pipe the res stream object to.

Then we can listen to the data event to get the data.

We call push on the buffer object to gather the data.

Then when the end event is emitted, we call the callback to send the data to the callback.

When an error event is emitted from the gunzip or http.get methods, we’ll call the callback to send the error.

Then we can use the function to get the error or the unzipped file’s data.

Conclusion

We can get the version with the process object.

To unzip files obtained from the http.get method, we can unzip it with zlib .

We’ve to hash and salt the password manually if we want to store it securely.

We can use the repl module to access the REPL.

Categories
Node.js Tips

Node.js Tips — Run Commands, Response Headers, Read Files, and More

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Node.js Execute System Command Synchronously

We can run system commands synchronously with the child_process module.

For instance, we can write:

const execSync = require('child_process').execSync;
const code = execSync('ls');

We run execSync to run the command we want.

Get Rid of header X-Powered-By:Express

We can remove the X-Powered-By header by using the app.disable method.

For instance, we can write:

app.disable('x-powered-by');

to stop the x-powered-by header from being added to the response.

We can also write:

app.set('x-powered-by', false);

to do the same thing.

Add a Custom Script to my package.json File that runs a JavaScript File

We can add the script to the package.json file’s scripts section.

For instance, we can write:

"scripts": {
  "start": "node app.js",
},

Then we can run npm start to run it.

Generate Unique ID with Node.js

To make creating unique IDs easy, we can add the uuid package to help us create UUIDs.

To install it, we run:

npm install uuid

Then we can use it by writing:

const { v1: uuidv1 } = require('uuid');`
const id = uuidv1();`

to create a v1 UUID.

And we can write:

const { v4: uuidv4 } = require('uuid');`
const id = uuidv4();`

to create a v4 UUID.

We can also use the crypto module to create a unique ID. However, it won’t be a UUID.

For instance, we can write:

const crypto = require("crypto");
const id = crypto.randomBytes(16).toString("hex");

id would be a 32-bit character string.

Replace a String in a File with Node.js

We can replace the content with new content with readFile and writeFile .

For instance, we can write:

const fs = require('fs');
const someFile = './file.txt';

fs.readFile(someFile, 'utf8', (err, data) => {
  if (err) {
    return console.log(err);
  }

  const result = data.replace('old string', 'new string');
  fs.writeFile(someFile, result, 'utf8', (err) => {
    if (err) {
      return console.log(err);
    }
  });
});

someFile is the path of the file.

We call fs.readFile to read the file with the UTF-8 encoding.

In the callback, we get the data , which has the content.

We call replace to replace 'old string' in data with 'new string' .

Then we call writeFile to write the file with the new content.

We also need the utf8 encoding to indicate that it’s a text file.

err will have the error object if it’s encountered in both callbacks.

Fix the ‘process.env.NODE_ENV is undefined’ Error

We can set the value of NODE_ENV by running:

SET NODE_ENV=development

in Windows or:

export NODE_ENV=development

in Linux to set the NODE_ENV environment variable.

Then process.env.NODE_ENV is set to 'development' when we run our app after those commands are run.

Also, we can put a script in the scripts section to set the NODE_ENV before running the script.

For instance, we can write:

"scripts": {
  "start": "set NODE_ENV=development && node app.js"
}

in package.json .

Set Response Header on Express.js Assets

We can call res.set to set the header on Express assets.

For instance, we can write:

res.set('Content-Type', 'text/plain');

where the first argument is the header key and the 2nd is the header value.

We can also pas sin an object to set multiple headers:

res.set({
  'Content-Type': 'text/plain',
  'Content-Length': '123'
})

We set the Content-Type and Content-Length all at once.

The res.header method is an alias of res.set .

To add more headers to the existing response header, we can use the res.append method.

For instance, we can write:

res.append('Set-Cookie', 'foo=bar; Path=/; HttpOnly')

We appended a response header for the cookie.

Read a Text File into an Array in a Node App

To read a text file into an array in a Node app, we can use the readFile method.

For example, we can write:

const fs = require('fs');

fs.readFile('file.txt', (err, data) => {
  if (err) {
    return console.log(err);
  }
  const array = data.toString().split("n");
  for(const a of array) {
    console.log(a);
  }
});

To read it in a synchronous way, we can use readFileSync :

constfs = require('fs');

const array = fs.readFileSync('file.txt').toString().split("n");
for(const a of array) {
  console.log(a);
}

Conclusion

We can set environment variables before running a Node app.

Also, we can read files in various ways.

We can use the child_process module to run commands in a Node app.

Response headers can be set in various ways.