Category: Node.js Tips

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

setTimeout and Node.js

setTimeout waits for a minimum number of milliseconds before running.

It’s not a guarantee.

Passing 0, non-number of a negative number will cause it to wait a minimum number of milliseconds in Node.

For example, if we have:

setTimeout(() => {
  console.log('hello');
}, 100)

Then we wait for 100ms at least before running the callback.

Node.js Project Naming Conventions for Files and Folders

A typical Node project has the /bin folder for scripts, helpers, and binaries.

/lib for the app.

/config for the configuration.

/public for the public files.

And /test for the tests.

Including JavaScript Class Definition from Another File in Node.js

We can export the class and import it in another file.

For instance, we can write:

person.js

class Person {
  //...
}

module.exports = Person;

Then in another file, we write:

app.js

const Person = require('./person');
const person = new Person();

We use require to require the person module and then instantiate the class that we imported.

Also, we can do the same with ES modules.

For instance, we can write:

person.js

export default class Person {}

app.js

import Person from './person';
const person = new Person();

We import the person module and use it the same way.

The export has to be a default export so that we can import the class without braces.

Change Working Directory in the Current Shell Context when Running Node Script

To change the working directory in a Node script, we can use the process.chdir method.

For instance, we can write:

const process = require('process');
process.chdir('../');

to move one directory up.

Load Vanilla Javascript Libraries into Node.js

If we want to load JavaScript libraries that aren’t modules, we can use the vm module.

For instance, we can write:

const vm = require("vm");
const fs = require("fs");
module.exports = (path, context) => {
  context = context || {};
  const data = fs.readFileSync(path);
  vm.runInNewContext(data, context, path);
  return context;
}

We read in the script file with the function we’re exporting and run it with vm.runInNewContext .

data has the script file. path is the file path.

context is the context that the script is running in, which is the global object.

Then we can use execfile to run the file:

const execfile = require("execfile");
const context = execfile("example.js", { globalVar: 42 });
console.log(context.foo());
context.globalVar = 100;

example.js is a file with:

function foo() {
  return 'bar';
}

We should get 'bar' from the console log since foo is a global function.

globalVar is a global variable.

ES6 Variable Import Name in Node.js

We can import with import by name by writing:

const import = async () => {
  try {
    const module = await import('./path/module');
  }  catch (error) {
    console.error('import failed');
  }
}

We use the import function which returns a promise.

It can be used to import ES modules by the path string we passed in.

If it’s successful, it’ll resolve to the module.

Convert Blob to File in JavaScript

We can convert a blob to a file in JavaScript in a few ways.

We can pass in the blob to the File constructor.

For instance, we can write:

const file = new File([blob], "name");

where the blob is the blob instance and 'name' is the file name.

We can pass in more arguments:

const file = new File([blob], "name", { lastModified: 1534584790000 });

We set the lastModfiied date with the timestamp in the 3rd argument.

We can also set the lastModifiedDate property of the blob.

For instance, we can write:

const blobToFile = (blob, fileName) => {
  blob.lastModifiedDate = new Date();
  blob.name = fileName;
  return blob;
}

We set the lastModifiedDate on the blob , which is a Blob instance to set the last modified date,

blob.name sets the file name.

Conclusion

We set the modified date of a blob or file in various ways.

setTimeout doesn’t guarantee the delay time.

We can export classes with modules.

It takes some effort to load non-module JavaScript libraries in Node apps.

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Locate a Parent Folder with fs

We can use the 2 dots to go to the parent folder.

For instance, we can write:

fs.readFile(path.join(__dirname, '/../../foo.txt'));

We have /../ to go up one level.

2 would go up 2 levels.

The forward slash is also required.

How ‘use strict’ is Interpreted in Node.js Apps

Node interpret 'use strict' the same way as other JavaScript apps.

It enables checks for undeclared variables, disallows setting constant values like undefined and so on.

Illegal operations that are silent without strict mode converted to errors.

ES6 modules have strict mode enabled by default, so we don’t have to enable it explicitly.

HTTP GET Request in Express

To make a GET request in a Node app, we can use the requestify HTTP client to make the request.

For instance, we can write:

var requestify = require('requestify');

requestify.get('http://example.com/')
  .then((response) => {
    response.getBody();
  }
);

We can put that anywhere in our Express app to make our request.

For example, we can put in in our route handler or middleware to make the request when a request is made from the client.

Difference Between Synchronous and Asynchronous Programming

Synchronous programs are run line by line.

Async programs compute the results in an indeterminate amount of time.

We can get the results in callbacks or assign then with await if the async code is a promise.

For instance, we can write:

database.query("SELECT * FROM table", (result) => {
  console.log(result);
});

is asynchronous. It doesn’t block the main thread when this code is run.

The callback is called when the results are ready.

Synchronous code is run by writing:

const result = database.query("SELECT * FROM table");
console.log(result);

The query method returns the result and we can get it after it’s assigned.

Some tasks like file system operations run in a different process.

Send a Message to a Specific Client with socket.io

We can use the clients property and send method to send a message to the specific client.

For instance, we can write:

const io = io.listen(server);
io.clients[clientId].send();

We get the client by its ID and then call send on it.

Send a 404 Response with Express

To send a 404 response with Express, we can call res.status to send the status code.

If we want to add a message, we can call send after it.

For instance, we can write:

res.status(404).send('not found');

Also, we can use the sendtatus method:

res.sendStatus(404);

Create Folder or Use Existing One if Exists

To make us create a folder or use one if it exists, we can use the mkdirp package.

To install it, we run:

npm install mkdirp

Then we can write:

cpnst mkdirp = require('mkdirp');
mkdirp('/foo', (err) => {
  //...
});

err has the error if it’s encountered.

'/foo' is the directory path.

The callback is called after the directory is created or that it already exists.

Since Node 10, we can also do the same thing without a library.

For instance, we can write:

fs.mkdirSync('/foo', { recursive: true })

to use or create a folder synchronously with the recursive option.

Also, we can write:

await fs.promises.mkdir('/foo', { recursive: true })

to do the same thing asynchronously.

'/foo' is the path.

How to Use NODE_ENV in Express

We can set the NODE_ENV environment variable with the export command.

For instance, we can write:

export NODE_ENV=production

Then in our Express app, we can write:

app.get('env')

to get the environment name.

Update a Specific Node Package

To update a specific Node package, we can use the npm update command.

For instance, we can run:

npm update express

Or we can run:

npm install express@4.17.1

to update Express to the specific version.

We can also update to the latest version by running:

npm install express@latest

Conclusion

We can update packages with npm update .

We can use async code to run long-running operations.

Also, we can send messages to a specific client.

There are HTTP clients made for Node apps.

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Return JSON Using Node or Express

We can render a JSON response with the http package by setting the Content-Type header and the res.end method.

For instance, we can write:

const http = require('http');

const app = http.createServer((req, res) => {
  res.setHeader('Content-Type', 'application/json');
  res.end(JSON.stringify({
    a: 1
  }));
});

app.listen(3000);

We call createServer to create an HTTP server.

In the callback, we set the Content-Type response header to application/json to set the response data to JSON.

Then we call res.end with the stringified JSON to render the response.

With Express, we just have to write:

res.json({
  foo: "bar"
});

in our middleware.

res.json sets the headers and render the JSON.

Enable HTTPS with Express

We can enable HTTPS with Express by reading the certificate data and use http.createServer to create the server.

For instance, we can write;

const fs = require('fs');
const http = require('http');
const https = require('https');
const privateKey = fs.readFileSync('sslcert/server.key', 'utf8');
const certificate = fs.readFileSync('sslcert/server.crt', 'utf8');
const credentials = {
  key: privateKey,
  cert: certificate
};
const express = require('express');
const app = express();

const httpServer = http.createServer(app);
const httpsServer = https.createServer(credentials, app);
httpServer.listen(3000);
httpsServer.listen(3443);

We read the private key and certificate files from the server.

Then we pass them all in an object and that object is passed to https.createServer to create the server.

Then we call listen on the httpsServer to create the server.

We still have to create the HTTP server to listen to non-secure HTTP requests.

Fix ‘Error: request entity too large’ Error from Express

We can fix the ‘request entity too large’ error by increasing the allowed size of requests.

For instance, we can write:

const bodyParser = require('body-parser');
app.use(bodyParser.json({
  limit: '50mb'
}));
app.use(bodyParser.urlencoded({
  limit: '50mb',
  extended: true
}));

We set limit in the object that we passed in as the argument to set the max size of the request payload.

'50mb' is 50 megabytes.

Download a File from Node Server Using Express

We can send a file download response to the client with res.download .

For instance, we can write:

app.get('/download', (req, res) => {
  const file =  path.join(__dirname, '/upload-folder/movie.mp4');
  res.download(file);
});

The res.download requires the absolute path to the file, so we need to combine __dirname with the rest of the path.

res.download will set the Content-Disposition and other headers automatically and send the file to the client.

Server Static Files with Express

We can serve static files with Express using the express.static middleware.

For instance, we can write:

app.use(express.static(path.join(__dirname, '/public')));

We serve the folder we want to serve as the static folder with the absolute path.

express.static will expose the folder to the client.

Get the JSON POST Data in an Express Application

We can get the JSON data from a POST request by using the req.body property.

For instance, we can write:

const express = require('express');
const bodyParser = require('body-parser');
const app = express.createServer();

app.use(`bodyParser`.`json`());

app.post('/', (req, res) => {
  console.log(req.body);
  response.send(req.body);
});

app.listen(3000);

req.body has the JSON request payload.

The req.body property is available if we ran the bodyParser.json() middleware.

We did that with:

app.use(bodyParser.json());

Remove Documents Using Node.js Mongoose

We can remove a document with Mongoose by using the remove method on the model class.

For example, we can write:

Model.remove({
  _id: id
}, (err) => {
  if (err) {
    console.log(err);
  }
});

Get Uploaded Files with Express

To get uploaded files with Express, we can use the express-fileupload package.

For example, we can write:

const fileupload = require("express-fileupload");
app.use(fileupload());

app.post("/upload", (req, res) => {
  if (!req.files) {
    res.send("File not found");
    return;
  }
  const file = req.files.photo;
  res.send("File Uploaded");
});

We used the fileupload middleware from the express-fileupload package.

To use it, we write:

app.use(fileupload());

The req.files should be available when we upload one or more files.

We can get the files by using the req.files object.

photo is the field name of the file input.

We can replace that with the correct name.

Conclusion

There are a few ways to render JSON response to the client.

We can get file uploads with a middleware package.

Also, we can return a file download response with res.download .

We’ve to use body-parser to parse JSON request payloads.

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Validation Library for Node.js

To validate data in a Node app, we can use a library to make our lives easier.

For instance, we can use the validator package.

We can use it by writing”

const check = require('validator').check;
const sanitize = require('validator').sanitize;

check('test@email.com').len(6, 64).isEmail();
check('abc').isInt();

const int = sanitize('0123').toInt();
const bool = sanitize('true').toBoolean();

We check is the email is between the length of 6 and 64 with the len method.

Then we called the isEmail method to check if it’s an email.

check checks a string with something validator methods.

sanitize cleans up the string.

isInt checks for integers.

If it’s not valid, then an exception would be thrown.

toInt converts a string to an int.

toBoolean converts a string to a boolean.

Piping the Same Readable Stream into Multiple Writable Targets

We can pipe the same readable stream into multiple writable targets.

For instance, we can write:

const spawn = require('child_process').spawn;
const PassThrough = require('stream').PassThrough;

const a = spawn('ls');
const b = new PassThrough();
const c = new PassThrough();

a.stdout.pipe(b);
a.stdout.pipe(c);

let output = '';

b.on('data', (chunk) => {
  output += chunk.length;
});

b.on('end', () => {
  console.log(count);
  c.pipe(process.stdout);
});

We can pipe the output of the ls shell command that’s run with the spawn method with the PassThrough instances.

We get the stdout property, which is a readable stream, and call pipe on it to pipe it to the writable streams which we created with the PassThrough constructor.

Then we listen to the streams by adding listeners to the data and end methods.

.rc Configuration Files

Anyone can make .rc files to create their own configuration file.

We can make a .rc file that has JSON for example.

For instance, we can write:

const fs = require('fs');
fs.readFile('./.barrc', 'utf8', (err, data) => {
  if (err) {
    return console.error(err);
  }
  console.log(JSON.parse(data));
})

We call readFile to read the file, then we can parse the JSON that’s stored in it.

It can also hold YAML or any other standard text format.

Determine if a String is a MongoDB ObjectID

We can determine if a string is a MongoDB Object ID by using the ObjectId.isValid method.

For example, we can write:

const ObjectId = require('mongoose').Types.ObjectId;
ObjectId.isValid('4738');

It checks if a string is a 12 character long string or if it’s in the correct format.

Perform a Full-Text Search in MongoDB and Mongoose

We can perform a full-text search in MongoDB by creating an index.

Then we can use the find method to search for the field we want.

For example, we can write:

const schema = new Schema({
  name: String,
  email: String,
  address: {
    street: String,
  }
});
schema.index({name: 'text', 'address.street': 'text'});

We can also include all fields in our index by writing:

schema.index({'$**': 'text'});

Then we can call find by writing:

Person.find({ $text: { $search: search }})
  .skip(20)
  .limit(20)
  .exec((err, docs) => {
    //...
  });

Update a Line Rather than Create a New Line with console.log

We can clear the existing lines and then write the new line with the process module.

For instance, we can write:

process.stdout.write("Hello, World");
process.stdout.clearLine();
process.stdout.cursorTo(0);
process.stdout.write("n");

We called write with some text to display it on the screen.

Then we called clearLine to clear the line.

Then we called cursorTo with 0 to move the cursor back to the left side.

Then we write the end of line character on the screen.

Write Objects into File with Node.js

To write objects into a file in a Node app, we can use writeFileSync to write it.

We can write an array by writing:

fs.writeFileSync('./data.json', arr.join(',') , 'utf-8');

We called join to combine the entries into a string.

Then writeFileSync would write the string to data.json .

To write an object’s content into a file, we can use util.inspect to convert it into a formatted string.

Then we write that to a file.

For example, we can write:

const util = require('util');
fs.writeFileSync('./data.json', util.inspect(obj) , 'utf-8');

Conclusion

We can write an object into a file.

Validation strings are easy if we use a library.

We can pipe readable streams into multiple write streams.

Also, we can make our own .rc configuration files in whatever format we want.

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Difference Between response.send and response.write in Express

res.send is called only once and it sends data in the chunk of the buffer size to the client.

Then it comes back and sends more data to the client with the buffer size until the sending is done.

res.write is called multiple times followed by res.end . It creates a buffer with the size based on the whole piece of data.

Then it sends that over HTTP. Therefore, it’s faster in case of huge amount of data.

Create Document if not Exists and Update the Existing Document Otherwise with Mongoose

We can create document if it doesn’t exist and do update otherwise with the findOneAndUpdate method.

For instance, we can write:

const query = {};
const update = { expire: new Date() };
const options = {
  upsert: true
};


Model.findOneAndUpdate(query, update, options, (error, result) => {
  if (error) return;
});

We pass in an options object with the upsert property set to true .

update has the data that we want to add or update.

query has the query for getting the data.

The callback is called when the operation is finished.

result has the data for the written file.

error has the error object.

Convert a Directory Structure in the Filesystem to JSON with Node.js

We can use the directory-package to read the structure of the filesystem into JSON.

To install it, we run:

npm i directory-tree

Then we can use it by writing:

const dirTree = require("directory-tree");
const tree = dirTree("/foo/bar");

We use the directory-tree package’s dirTree function.

Then we pass in the path into the dirTree function.

We can restrict the file types that we want to enumerate by writing:

const dirTree = require("directory-tree");
const filteredTree = dirTree("/some/path", { extensions: /.txt/ });

We use the extension regex to do the filtering.

Listen on HTTP and HTTPS for a Single Express App

We can listen to HTTP and HTTPS at the same time by using the http and https packages and calling the createServer method on both.

For instance, we can write:

const express = require('express');
const https = require('https');
const http = require('http');
const fs = require('fs');
const app = express();

const options = {
  key: fs.readFileSync('/path/to/key.pem'),
  cert: fs.readFileSync('/path/to/cert.pem'),
  ca: fs.readFileSync('/path/to/ca.pem')
};

http.createServer(app).listen(80);
https.createServer(options, app).listen(443);

We read the certificate files and call createServer with them to listen to HTTPS requests.

And we also use http to listen to regular HTTP requests as usual.

Use Cases for process.nextTick in Node.js

We can use process.nextTick to put a callback into a queue.

It’s the same as using setTimeout to delay code execution.

For instance, we can write:

process.nextTick(() => {
  console.log('hello');
});

Encrypting Data that Needs to be Decrypted

We can encrypt data that needs to be decrypted with the crypto module.

It has the createCipher method to lets us create a cipher that can be deciphered.

For instance, we can write:

const crypto = require('crypto');

const algorithm = 'aes256';
const key = 'sercetkey';
const text = 'hello world';

const cipher = crypto.createCipher(algorithm, key);
const encrypted = cipher.update(text, 'utf8', 'hex') + cipher.final('hex');

const decipher = crypto.createDecipher(algorithm, key);
const decrypted = decipher.update(encrypted, 'hex', 'utf8') + decipher.final('utf8');

We called createCipher with the algorithm and key to create the cipher.

Then to encrypt the text , we call cipher.update and concatenate it with cipher.final('hex') .

cipher.final can only be called once so that the text can only be encrypted once.

When we wan to decipher the text, we call the createDecipher method with the same algorithm and key .

Then we can use decipher.update to decipher the encrypted text and concatenate that with decipher.find('hex') to decrypt the text.

decipher.final can only be called once so that the encrypted text can only be decrypted once.

Conclusion

We can create ciphers and decipher text with the crypto module.

res.send and res.write are different in Express. One is good for small data and one is good for bigger ones.

We can upsert data with Mongoose.

Directory structures can be read into JSON.