Categories
Node.js Tips

Node.js Tips — MongoDB, Express and Socket.io, and Postgres

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Get the _id of Inserted Document in Mongo Database in NodeJS

We can get the _id of the inserted document in the Mongo database from the callback that’s called by the insert method.

For instance, we can write:

collection.insert(obj, (err, docsInserted) =>{
  console.log(docsInserted);
});

We pass in the obj to the collection .

Then we get the docsInserted parameter’s value to get the whole object including the ID with the _id.

Notify Node App when Data Values Change in Redis

We can use the Redis client for Node to watch for data value changes.

For instance, we can write:

const redis = require("redis");
const client = redis.createClient();

client.subscribe("pubsub");
client.on("message", (channel, message) => {
  console.log(channel, message);
});

We just call createClient to create a Redis client instance.

Then we subscribe to to the pubsub action to listen to published changes.

Finally, we listen to the 'message' event to get the messages.

Delete Query with Sequelize.js

We can make a delete query with Sequelize by writing:

Model.destroy({
  where: {
    id: 123
  }
})

where Model is the model class of the data table that has the entry we want to remove.

Then we call destroy to delete the entry.

where has the where. id is the ID field.

It can be replaced with other criteria.

Make Express.js App Work on a Different Port

We don’t have to hard code the port that the Express app listens to.

Instead, we can read the value from an environment variable.

For instance, we can write:

app.js

const express = require("express");
const app = express();

app.set('port', process.env.PORT || 3000);

app.get('/', (req, res) => {
  res.send('hello world');
});

app.listen(app.get('port'));

Then we get the port from the environment variables instead of hard-coding the port.

process.env.PORT is the value of the PORT environment variable.

Now we can set the port and run our app at the same time by running:

PORT=8080 node app.js

Connect to a Postgres Database with the Node Postgres Module

To connect to a Postgres database with the Node Postgres module, we can use the pg.connect method.

For instance, we can write:

module.exports = {
  query(text, values, cb) {
    pg.connect((err, client, done) => {
      client.query(text, values, (err, result) => {
        done();
        cb(err, result);
      })
    });
  }
}

We make use of the pg.connect method to connect to the database.

Then we use the client object, which is the database client object, to make a query.

text has the parameterized query.

values has the values for the query.

The callback is called when the query is finished.

result has the results.

We call our cb callback so that the results can be obtained when we call the query method.

Using socket.io in Express 4 and express-generator’s /bin/www

We can use socket.io by adding the io object to the app.js file:

const express = require("express");
const socketIo = require("socket.io");
const app = express();
const io = socketIo();
app.io = io;

io.on("connection", ( socket ) => {
  console.log('connected');
});

module.exports = app;

We attach the io object as the property of app .

We listen to the 'connection' event with the on method.

Then we can use it by writing the following in bin/www :

//...
const server = http.createServer(app);

const io = app.io;
io.attach(server);

We import the app object before calling createServer and then call io.attach to attach the socket.io listeners to the Express app.

Then in our route file, we can write:

const app = require('express');

module.exports = (io) => {
  const router = app.Router();

  io.on('connection', (socket) => {
    //...
  });
  return router;
}

Then we change app.js to:

const express = require("express");
const socketIo = require("socket.io");
const app = express();
const io = socketIo();
app.io = io;
io.on("connection", ( socket ) => {
  console.log('connected');
});

const routes = require('./routes/index')(io);

//...

module.exports = app;

Conclusion

We can get the ID of the inserted MongoDB document from the insert callback.

We can incorporate socket.io into the Express app generated by the express-generator by attaching the io object to the app .

Also, we can use the Postgres database with Node apps by connecting to the database and using the client object returned after creating the connection.

Categories
Node.js Tips

Node.js Tips — Global Objects, Synchronous Requests, Shared Code for Tests

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Synchronous Requests in Node.js

We can make synchronous requests in Node apps with the sync-request package.

For instance, we can write:

const request = require('sync-request');

try {
  const res1 = request('GET', '/yrl1');
  const res2 = request('GET', '/url2');
  //...
}
catch (e) {
  //...
}

We import the sync-request package an use its request function to make requests synchronously.

However, as usual, synchronous code is blocking, so it shouldn’t be used in production apps.

However, it may be good for scripts.

Get Schema of a MongoDB Database Which Defined in Another Model with Mongoose

We can get the Mongoose schema with the model method.

For instance, we can write:

const PersonSchema = require('mongoose').model('person').schema

to get the Mongoose schema for the person collection.

Detect Node.js or Browser Environments

To check in a piece of code whether it’s running in Node or the browser, we can check if the window object exists.

If it does, then we know it’s running in the browser.

Otherwise, it’s running in Node.

For instance, we can write:

const app = window ? window : global;

to get the global variable of the corresponding platforms.

window is the global object for the browser.

global is the global object for Node.

Difference Between readFile and readFileSync

fs.readFile takes a callback and reads the file asynchronously.

The callback is called with the result whenever it’s done.

readFileSync reads the file synchronously and returns the content of it when it’s done.

We should use readFile in most apps since it doesn’t block other parts of an app from running.

However, readFileSync are good for scripts where we aren’t running anything else other than the code in the script sequentially.

How to Detect the Environment in an Express.js App

We can use the app.get method to get the environment variable in an Express app.

For instance, we can write:

app.get('env')

to get the environment variable.

Then we can write:

if (app.get('env') === 'development') {
  //...
}

to check if the app is running in the 'development' environment.

Parsing Form Data Request Payloads in Express

The body-parser package lets us parse request payloads in Express.

For instance, we can write:

const bodyParser = require('body-parser');
app.use(bodyParser.json());
app.use(bodyParser.urlencoded());

app.use(bodyParser.urlencoded({ extended: true }));

app.post('/post',(req, res) => {
  console.log(req.body)
})

We require the body-parser so that we can call the json method to make it parse JSON.

And we call urlencoded so that it can parse form data payloads.

extended means that we can parse JSON that’s nested in form-data requests.

Then the parsed request payload will be available in req.body .

Alternatively, we can use the express-formidable middleware to do the same thing.

We can install it by running:

npm install express-formidable

Then we can use it by writing:

const express = require('express');
const formidable = require('express-formidable');

const app = express();

app.use(formidable());

app.post('/upload', (req, res) => {
  res.send(JSON.stringify(req.fields));
});

We just pass in the formidable middleware.

Then we get the parsed request payload with the req.fields object.

Run Async Mocha Shared Test Code in Order

We can run async Mocha tests in order by using the before hook to run code that’s shared between multiple tests.

For instance, we can write:

let someCondition = false;
//...

describe('is dependent on someCondition', () => {
  const beforeTest = (done) => {
    if (someCondition) {
      done();
    }
    else {
      setTimeout(() => beforeTest(done), 1000);
    }
  }

  before((done) => {
    beforeTest(done);
  });

  it('does something', () => {
    // ...
  });

})

We check that someCondition is met before we run our tests with the beforeTest function.

Otherwise, we call beforeTest again after 1 second.

We do that until someCondition is true .

We pass in done to the beforeTest function so that we can call it when someConditional finally becomes true .

Once done is called, the test will be run.

Conclusion

We can run some checks before a test is run and wait for a condition to become true before running it.

We can use the sync-request package to run synchronous requests.

However, it’s blocking the rest of the app so it should probably only be used in scripts.

We can detect the platform that a piece of code is run in by checking which global object is available.

Categories
Node.js Tips

Node.js Tips — Modules, Request Bodies, Reading Lines, and More

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

How to Include Functions from Other Files

We can include functions from other files if we export the function from a module and then import them.

For example, we can write:

fns.js

module.exports = {
  foo() {
    //...
  }
};

module.exports indicates that we’re exporting the properties inside.

Then we can import it by writing:

const `fns` = require('./`fns`');
`fns.foo();`

We use the require keywords to import the fns.js module.

We don’t need the file extension.

Using Node.js require vs. ES6 import/export

require is the most versatile way to import modules and it’s supported by all versions of Node.

ES6 imports and exports are supported natively only by the most recent versions of Node.

Therefore, require is still the best bet until our apps are running on the latest version of Node.

Sending Command-Line Arguments to npm script

We can send command line arguments to an NPM script by using the -- characters.

For example, if we have the following in package.json :

"scripts": {
  "server": "node server.js"
}

Then we can run:

npm run server -- --port=1234

Retrieve POST Query Parameters in an Express App

We can retrieve POST query parameters in an Express app by installing the body-parser package.

To install it, we run:

npm install --save body-parser

For example, we can write:

const bodyParser = require('body-parser');
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({
  extended: true
}));

bodyParser.json() will parse JSON payloads.

And bodyParser.urlencoded will parse URL encoded payloads.

‘Can’t set headers after they are sent to the client’ Error

We should make sure that we don’t send a response more than once in our app.

For instance, we should write:

const foo = (request, response, next) => {
  request.params = {
    a: "b"
  };
  next();
};

const bar = function(request, response, next) => {
  response.setHeader("Content-Type", "text/html");
  response.write("hello world");
  response.end();
};

app.use(foo);
app.use(bar);

Anything that modifies the response should be placed last.

This way, we don’t send the response more than once.

Anything that calls next shouldn’t modify the response so that we won’t call any middleware that does send the response after it.

Convert an Existing Callback API to Promises

We can convert any async callback to a promise with the Promise constructor.

For instance, we can write:

const run = () => {
  return new Promise((resolve, reject) => {
    //...
    resolve();
  });
}

We return a promise in our run function so we can use it with async and await or then .

Also, we can use the util package’s promisify method to convert functions that take Node style callbacks to return a promise.

For instance, we can write:

const fs = require('fs');
const util = require('util');

const readFile = util.promisify(fs.readFile);

(async () => {
  const data = await `readFile('/foo.txt');
  //...
})();`

We convert the async readFile method to a promise by using the util.promisify method.

Then we get the data from reading the file with await .

Read a File One Line at a Time

To read a file one line at a time, we can use the readline module.

For example, we can write:

const fs = require('fs');
const readline = require('readline');

const readLineByLine = async () => {
  const fileStream = fs.createReadStream('foo.txt');

  const rl = readline.createInterface({
    input: fileStream,
    crlfDelay: Infinity
  });

  for await (const line of rl) {
    console.log(line);
  }
}

readLineByLine();

We call createInterface to get an iterator with the lines, then we use the for-await-of loop to loop through the lines.

Also, we can create a read stream by writing:

const readline = require('readline');

readline.createInterface({
  input: require('fs').createReadStream('file.in')
});

lineReader.on('line', (line) => {
  console.log(line);
});

We use the createInterface like the last example, but we call on to listen to the line event which has the line that’s read.

Conclusion

We can export and import files to include functions from other JavaScript files.

Reading a file one line at a time can be done with the readline library.

Also, we should make sure that we don’t send a response more than once in our Express app.

The body-parser package lets us parse request payloads in an Express app.

Categories
Node.js Tips

Node.js Tips — Templates, Queries, Dates, and Memory

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Get Data from fs.readFile

We can call the readFile method from the fs module to tread files.

For instance, we can write:

fs.readFile('./foo.txt', (err, data) => {
  if (err) {
    throw err;
  }
  console.log(data);
});

We pass in the path and then read it as a binary file.

data has the file content and err is the error object that’s set when an error is encountered.

There’s also the fs.readFileSync method:

const text = fs.readFileSync('./foo.txt', 'utf8');
console.log(text);

Get a URL Parameter in Express

To get a URL parameter in an Express app, we can use the req.params property to get the URL parameter.

For instance, we can write:

app.get('/tags/:tagId', (req, res) => {
  res.send(req.params.tagId);
});

Then if we make a GET request to /tags/2 , we get 2 as the value of req.params.tagId .

To get a value from a query parameter, we can use the req.query property:

app.get('/foo', (req, res) => {
  res.send(req.query.tagId);
});

Then if we make a request to /foo?tagId=2 , req.query.tagId would be 2.

Render Basic HTML View

We can render HTML by using include to include HTML file in our Jade view.

For instance, we can write in views/index.jade:

include plainHtml.html

Then in views/plainHtml.html , we can write normal HTML”

<!DOCTYPE html>
...

And we render index.jade with:

res.render(index);

NodeJS Support for import/export ES6 (ES2015) Modules

If our app is running on Node 13.2.0 or above, we can enable ES6 module support by adding the following to package.json :

{
  "type": "module"
}

With Node 13.1.0 or below, we can write:

node -r esm main.js

We turn on ES module support by using esm .

We just run npm i esm to add esm to our Node project.

Use underscore.js as a Template Engine

We can use the template method to pass in our HTML template string.

For instance, we can write:

const template  = _.template("<h1>hello <%= name %></h1>");

template is a function that takes an object with the placeholders.

Then name is the placeholder that we can pass in.

For instance, we can write:

const str = `template({ name: 'world' });`

Then str would be '<h1>hello world</h1>’ .

<%= %> means to print some value in the template.

<% %> executes some code.

<%- %> prints some values with the HTML escaped.

Start Script Missing Error When Running npm start

To solve this error, we can add the start script to package.json by putting it in the scripts section.

For instance, we can write:

"scripts": {
  "start": "node app.js"
}

The script with the key start will run when we run npm start .

Node.js Heap Out of Memory

We can increase the memory limit of Node apps by setting the max-old-space-size option.

For instance, we can write:

node --max-old-space-size=2048 app.js

to increase the memory limit to 2 GB.

We can also set it in the NODE_OPTIONS environment variable:

export NODE_OPTIONS=--max_old_space_size=2048

Callback After All Asynchronous forEach Callbacks are Completed

We can use the for-await-of loop to loop through promises.

For instance, we can write:

const get = a => fetch(a);

const getAll = async () => {
  const promises = [1, 2, 3].map(get);

  for await (const item of promises) {
    console.log(item);
  }
}

This will run each promise sequentially with a loop.

This is available since ES2018.

Format a UTC Date as a YYYY-MM-DD hh:mm:ss String

We can use the toISOString method to format a Date instance into a date string in the ‘YYYY-MM-DD hh:mm:ss’ format.

For instance, we can write”

const str = new Date()
  .toISOString()
  .`replace(/T/, ' ')
  .replace(/..+/, '');`

We replace the T with an empty string, and replace the text after the dot with an empty string.

Then we get '2020–06–04 16:15:55' as the result.

Conclusion

Underscore has a template method.

readFile can read files.

We can get query parameters in Express.

The memory limit of Node apps can be increased.

Jade templates can include HTML partial views.

Categories
Node.js Tips

Node.js Tips — Environment Variables, Downloads, DNS, and More

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Node.js Require All Files in a Folder

We can export all modules in a folder by creating a module that exports all the modules.

Then we can import them all at once.

For instance, we can write:

index.js

exports.foo = require("./routes/foo.js");
exports.bar= require("./routes/bar.js");

app.js

const routes = require("./routes");

We can also write:

const fs = require("fs");
const normalizedPath = require("path").join(__dirname, "routes");

fs.readdirSync(normalizedPath).forEach((file) => {
  require(`./routes/${file}`);
});

Stringify an Error

We can log an error by looping through the properties of an object.

For instance, we can write:

const error = new Error('error');
const propertyNames = Object.getOwnPropertyNames(error);

for (const prop of propertyNames) {
  const descriptor = Object.getOwnPropertyDescriptor(error, prop);
  console.log(prop, descriptor);
}

We use the Object.getOwnPropertyNames method to get the properties of the error object.

Then we get the descriptors of each property.

Difference Between “process.stdout.write” and “console.log” in Node.js

console.log calls process.stdout.write with formatted output.

The code for console.log is just:

console.log = function (d) {
  process.stdout.write(d + 'n');
};

They are pretty much equivalent except for the extra line break.

Set NODE_ENV to production/development in Mac OS or Windows

We can use the export command to set environment variables in Mac OS.

For instance, we can set the NODE_ENV by running:

export NODE_ENV=production

In Windows, we can use the SET command:

SET NODE_ENV=production

We can also set process.env.NODE_ENV directly:

process.env.NODE_ENV = 'production';

We can set the environment when we run the file by running:

NODE_ENV=production node app.js

It also works in the package.json scripts :

{
  ...
  "scripts": {
    "start": "NODE_ENV=production node ./app"
  }
  ...
}

Using require in Express Templates

require is included with CommonJS.

It’s not a part of browser-side JavaScript natively.

To include JavaScript files, we can use the script tag.

Also, we can use CommonJS module.

Or we can use async modules.

Browserify, Webpack, and Rollup can bundle modules into build artifacts usable by the browser.

We can also use ES6 modules natively in browsers with the type attribute set to module .

For instance, we can write:

script.js

export const hello = () => {
  return "Hello World";
}

We can then include script.js by writing:

<script type="module" src="script.js"></script>

Measure the Execution Time of JavaScript Code with Callbacks

We can use the console.time and console.timeEnd methods to measure the time of a script.

For instance, if we have:

for (let i = 1; i < 10; i++) {
  const user = {
    id: i,
    name: "name"
  };
  db.users.save(user, (err, saved) => {
    if(err || !saved) {
      console.log("error");
    } else {
      console.log("saved");
    }
  });
}

We can call the methods by writing:

console.time("save");

for (let i = 1; i < 10; i++) {
  const user = {
    id: i,
    name: "name"
  };
  db.users.save(user, (err, saved) => {
    if(err || !saved) {
      console.log("error");
    } else {
      console.log("saved");
    }
    console.timeEnd("save");
  });
}

We’ve to pass in the same string to time and timeEnd .

Download a File from NodeJS Server Using Express

We can download a file with the res.download method in an Express route.

For instance, we can write:

const path = reqyire('path');

app.get('/download', (req, res) => {
  const file = path.join(`${__dirname}, 'folder', 'img.jpg');
  res.download(file);
});

We just get the path and pass it to res.download .

URl Encode a String in Node.js

We can URI encode a string in Node.jhs with the querystring module.

For instance, we can write:

const querystring = require("querystring");
const result = querystring.stringify({ text: "hello world"});

Then we get 'text=hello%20world’ for result .

Get Local IP Address in Node.js

We can get a local IP address in a Node app with the dns module.

For instance, we can write:

const dns = require('dns');
const os = require('os');

dns.lookup(os.hostname(), (err, addr, fam) => {
  console.log(addr);
})

We call lookup with the hostname to get the IP address from the hostname.

Then addr has the IP address.

Conclusion

We can get the IP address with the dns module.

console has methods to let us measure the timing of code.

We can require files dynamically.

Environment variables can be set dynamically.