Categories
Node.js Tips

Node.js Tips — HTTP Proxies, Mongoose Schemas, Read Files, and More

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Create a Simple HTTP Proxy in Node.js

To create a simple HTTP proxy with Node.js, we can use the node-http-proxy library.

We can install it by running:

npm install http-proxy --save

Then we can write:

const http = require('http');
const httpProxy = require('http-proxy');
const proxy = httpProxy.createProxyServer({});

http.createServer((req, res) => {
  proxy.web(req, res, { target: 'http://www.example.com' });
}).listen(3000);

We call createProxyServer to create the proxy server.

Then we call proxy.web to redirect our request to http://example.com .

fs.createReadStream vs fs.readFile in Node.js

We should use fs.createReadStream to read a file one chunk at a time.

This means that we can read big files without loading all of the content into memory.

fs.readFile is good for reading the entire file. This means it’s good for reading small files.

Read stream can accept data in a way that doesn’t overwhelm the host system.

Get the Domain Originating the Request in Express.js

We can get the domain of the originating request with Express.

All we have to do is to use req.get('host') to get the HOST header.

This works for a non-cross-origin request.

For a cross-origin request, we can use req.get('origin') instead.

Also, we can use req.headers.host and req.headers.origin to do the same thing.

If we want to get a client’s IP address, we can use req.socket.remoteAddress .

Create Mongoose Schema with an Array of Object IDs

To create a Mongoose schema with an array of object IDs, we can just pass in a constructor or schema to an array.

For instance, we can write:

const userSchema = mongoose.Schema({
  lists: [listSchema],
  friends: [{ type : ObjectId, ref: 'User' }]
});

exports.User = mongoose.model('User', userSchema);

We pass in listSchema to use an existing schema for a schema with an array.

listSchema is an existing Mongoose schema.

We can also pass in an object as we did with friends .

We make friends reference itself.

Listen to All Emitted Events in Node.js

To listen to all emitted events, we can use a wildcard to do it.

We can do that with the eventemitter2 package.

It supports namespacing and wildcards.

For instance, we can write:

const EventEmitter2 = require('eventemitter2');
const emitter = new EventEmitter2({
  wildcard: false,
  delimiter: '.',
  newListener: false,
  removeListener: false,
  maxListeners: 10,
  verboseMemoryLeak: false,
  ignoreErrors: false
});

emitter.on('bar.*', (val) => {
  console.log(this.event, valu);
});

We listen to all events with names starting with bar.

Also, we can listen to multiple events and events with identifiers denoted with symbols.

Accessing Express.js Local Variables in Client-Side JavaScript

We can access Express local variables in client-side JavaScript.

All we have to do is to pass in an object as the 2nd argument of res.render .

Then we can access the variable by the key names.

For instance, we can write:

res.render('index', {
  title: 'page title',
  urls: JSON.stringify(urls),
});

Then we can access them by writing:

const urls = !{urls};

in our Jad template

We’ve to stringify urls since the variables are added with string interpolation.

How to Check if Headers have Already Sent with Express

We can check if headers are already sent with the res.headerSent property.

For instance, we can write:

if (res.headersSent) {
  //...
}

Convert Timestamp to Human Date

We can convert a timestamp to a human date by using the Date constructor.

For example, we can write:

const date = new Date(1591399037536);

Then we can call toDateString , toTimeString , or toLocaleDateString to format the string the way we like.

Measuring Timing of Code in a Node App

To measure the timing of Node apps, we can use the performance.now() method to do that.

For example, we can write:

const {
  performance
} = require('perf_hooks');

console.log(performance.now());

We just import the performance object from perf_hooks and then call the now method on it.

Conclusion

We can use the performance.now() method to measure performance.

To create an HTTP proxy, we can use a 3rd party package to do it.

Mongoose schemas can have arrays in it.

We can get the request’s hostname from the headers in Express.

Read streams are good for reading big files, readFile is good for reading small files.

Categories
Node.js Tips

Node.js Tips — Start Scripts, Object IDs, and Logging

As with many types of apps, there are difficult issues to solve when we write Node apps. In this article, we’ll look at some solutions to common problems when writing Node apps.

npm start vs node app.js

npm start can be run when we put a start script in the package.json .

In package.json , we’ve to add a script like:

{
  ...
  `"scripts": {
    "start": "node index.js"
  }
  ...
}`

Then we can run npm start , which runs node index.js .

npm start can run any script.

On the other hand, node app.js just runs app.js with node .

It’s always the same.

Convert a String to ObjectId in Node.js MongoDB Native Driver

We can convert a string to an object ID with the Node MongoDB driver by using the ObjectId function from the mongodb package.

For instance, we can write:

const {ObjectId} = require('mongodb');

const updateStuff = (id, doc) => {
  if (!ObjectId.isValid(s)) {
    return Promise.reject(new Error('invalid id'));
  }
  return collection.findOneAndUpdate(
    { _id: ObjectId(id) },
    { $set: doc },
    { returnOriginal: false }
  );
};

We check if the string is a valid object ID with the ObjectId.isValid method. If it is, then we call findOneAndUpdate to do the update. We’ve to convert the id string to an object ID with the ObjectId function.

Create a Sleep or Delay Function in Node.js that is Blocking

If we want a synchronous sleep or delay function in Node, we can create a loop that runs until a given the time span has elapsed.

For instance, we can write:

const sleep = (time) => {
  const stop = new Date().getTime();
  while(new Date().getTime() < stop + time) {

  }
}

We just create a while loop with an empty body that runs until the timestamp is stop + time .

This is a synchronous sleep function that blocks the rest of the app from running. Therefore, we probably shouldn’t use it in most apps.

Global Modules

We can require a module and set it to a global variable.

For instance, we can write:

global.util = require('util');

We can set the util module as a global variable, but we probably shouldn’t do that since it doesn’t bring any advantages over importing modules. And we don’t know what’s in the global object if we do that a lot.

How to Generate Timestamp UNIX epoch format Node.js

To generate timestamp as a UNIX epoch, we can use the Date constructor.

For instance, we can write:

const timestamp = Math.floor(new Date() / 1000);

new Date() returns the UNIX epoch in milliseconds.

We can divide it by 1000 to get the epoch in seconds.

And we use Math.floor to round it down to the nearest integer.

Asynchronous Module Loading with Node.js

We can make require asynchronous by creating our own module that can load asynchronously.

For instance, we can write:

bar.js

const fs = require('fs');
module.exports = (callback) => {
  fs.readFile('/foo/bar', (err, data) => {
    callback(err, data);
  });
};

We call readFile to read the content of /foo/bar and calls the callback which has the data read from the file.

We call callback inside it so that we can use it outside of the readFile callback.

Then we can use bar.js by writing:

require('./bar')((err, bar) => {
  // ...
});

We require bar.js with require .

Then we call the required function with our own callback to get the data from bar.js via the bar parameter.

Difference Between console.log and sys.puts in Node.js

sys.puts can’t print the content of objects. It just prints the string passed into it in the logs. It’s also deprecated so it shouldn’t be used. console.log can print the content of arrays and objects. It’s not deprecated so it’s safe to use in our apps.

Conclusion

npm start runs the script named start from package.json . We can check if a string is a valid object ID before converting it to such. sys.puts shouldn’t be used since the sys module is deprecated. We can generate a timestamp with the Date constructor. To read modules asynchronously, we can use readFile to read the content. We shouldn’t use synchronous log functions even if we can write one.

Categories
Node.js Tips

Node.js Tips — Non-Module Libraries, Blobs, setTimeout, and More

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

setTimeout and Node.js

setTimeout waits for a minimum number of milliseconds before running.

It’s not a guarantee.

Passing 0, non-number of a negative number will cause it to wait a minimum number of milliseconds in Node.

For example, if we have:

setTimeout(() => {
  console.log('hello');
}, 100)

Then we wait for 100ms at least before running the callback.

Node.js Project Naming Conventions for Files and Folders

A typical Node project has the /bin folder for scripts, helpers, and binaries.

/lib for the app.

/config for the configuration.

/public for the public files.

And /test for the tests.

Including JavaScript Class Definition from Another File in Node.js

We can export the class and import it in another file.

For instance, we can write:

person.js

class Person {
  //...
}

module.exports = Person;

Then in another file, we write:

app.js

const Person = require('./person');
const person = new Person();

We use require to require the person module and then instantiate the class that we imported.

Also, we can do the same with ES modules.

For instance, we can write:

person.js

export default class Person {}

app.js

import Person from './person';
const person = new Person();

We import the person module and use it the same way.

The export has to be a default export so that we can import the class without braces.

Change Working Directory in the Current Shell Context when Running Node Script

To change the working directory in a Node script, we can use the process.chdir method.

For instance, we can write:

const process = require('process');
process.chdir('../');

to move one directory up.

Load Vanilla Javascript Libraries into Node.js

If we want to load JavaScript libraries that aren’t modules, we can use the vm module.

For instance, we can write:

const vm = require("vm");
const fs = require("fs");
module.exports = (path, context) => {
  context = context || {};
  const data = fs.readFileSync(path);
  vm.runInNewContext(data, context, path);
  return context;
}

We read in the script file with the function we’re exporting and run it with vm.runInNewContext .

data has the script file. path is the file path.

context is the context that the script is running in, which is the global object.

Then we can use execfile to run the file:

const execfile = require("execfile");
const context = execfile("example.js", { globalVar: 42 });
console.log(context.foo());
context.globalVar = 100;

example.js is a file with:

function foo() {
  return 'bar';
}

We should get 'bar' from the console log since foo is a global function.

globalVar is a global variable.

ES6 Variable Import Name in Node.js

We can import with import by name by writing:

const import = async () => {
  try {
    const module = await import('./path/module');
  }  catch (error) {
    console.error('import failed');
  }
}

We use the import function which returns a promise.

It can be used to import ES modules by the path string we passed in.

If it’s successful, it’ll resolve to the module.

Convert Blob to File in JavaScript

We can convert a blob to a file in JavaScript in a few ways.

We can pass in the blob to the File constructor.

For instance, we can write:

const file = new File([blob], "name");

where the blob is the blob instance and 'name' is the file name.

We can pass in more arguments:

const file = new File([blob], "name", { lastModified: 1534584790000 });

We set the lastModfiied date with the timestamp in the 3rd argument.

We can also set the lastModifiedDate property of the blob.

For instance, we can write:

const blobToFile = (blob, fileName) => {
  blob.lastModifiedDate = new Date();
  blob.name = fileName;
  return blob;
}

We set the lastModifiedDate on the blob , which is a Blob instance to set the last modified date,

blob.name sets the file name.

Conclusion

We set the modified date of a blob or file in various ways.

setTimeout doesn’t guarantee the delay time.

We can export classes with modules.

It takes some effort to load non-module JavaScript libraries in Node apps.

Categories
Node.js Tips

Node.js Tips — Making HTTP Requests, Express, Packages, and File Operations

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Locate a Parent Folder with fs

We can use the 2 dots to go to the parent folder.

For instance, we can write:

fs.readFile(path.join(__dirname, '/../../foo.txt'));

We have /../ to go up one level.

2 would go up 2 levels.

The forward slash is also required.

How ‘use strict’ is Interpreted in Node.js Apps

Node interpret 'use strict' the same way as other JavaScript apps.

It enables checks for undeclared variables, disallows setting constant values like undefined and so on.

Illegal operations that are silent without strict mode converted to errors.

ES6 modules have strict mode enabled by default, so we don’t have to enable it explicitly.

HTTP GET Request in Express

To make a GET request in a Node app, we can use the requestify HTTP client to make the request.

For instance, we can write:

var requestify = require('requestify');

requestify.get('http://example.com/')
  .then((response) => {
    response.getBody();
  }
);

We can put that anywhere in our Express app to make our request.

For example, we can put in in our route handler or middleware to make the request when a request is made from the client.

Difference Between Synchronous and Asynchronous Programming

Synchronous programs are run line by line.

Async programs compute the results in an indeterminate amount of time.

We can get the results in callbacks or assign then with await if the async code is a promise.

For instance, we can write:

database.query("SELECT * FROM table", (result) => {
  console.log(result);
});

is asynchronous. It doesn’t block the main thread when this code is run.

The callback is called when the results are ready.

Synchronous code is run by writing:

const result = database.query("SELECT * FROM table");
console.log(result);

The query method returns the result and we can get it after it’s assigned.

Some tasks like file system operations run in a different process.

Send a Message to a Specific Client with socket.io

We can use the clients property and send method to send a message to the specific client.

For instance, we can write:

const io = io.listen(server);
io.clients[clientId].send();

We get the client by its ID and then call send on it.

Send a 404 Response with Express

To send a 404 response with Express, we can call res.status to send the status code.

If we want to add a message, we can call send after it.

For instance, we can write:

res.status(404).send('not found');

Also, we can use the sendtatus method:

res.sendStatus(404);

Create Folder or Use Existing One if Exists

To make us create a folder or use one if it exists, we can use the mkdirp package.

To install it, we run:

npm install mkdirp

Then we can write:

cpnst mkdirp = require('mkdirp');
mkdirp('/foo', (err) => {
  //...
});

err has the error if it’s encountered.

'/foo' is the directory path.

The callback is called after the directory is created or that it already exists.

Since Node 10, we can also do the same thing without a library.

For instance, we can write:

fs.mkdirSync('/foo', { recursive: true })

to use or create a folder synchronously with the recursive option.

Also, we can write:

await fs.promises.mkdir('/foo', { recursive: true })

to do the same thing asynchronously.

'/foo' is the path.

How to Use NODE_ENV in Express

We can set the NODE_ENV environment variable with the export command.

For instance, we can write:

export NODE_ENV=production

Then in our Express app, we can write:

app.get('env')

to get the environment name.

Update a Specific Node Package

To update a specific Node package, we can use the npm update command.

For instance, we can run:

npm update express

Or we can run:

npm install express@4.17.1

to update Express to the specific version.

We can also update to the latest version by running:

npm install express@latest

Conclusion

We can update packages with npm update .

We can use async code to run long-running operations.

Also, we can send messages to a specific client.

There are HTTP clients made for Node apps.

Categories
Node.js Tips

Node.js Tips — Express Requests and Response

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Return JSON Using Node or Express

We can render a JSON response with the http package by setting the Content-Type header and the res.end method.

For instance, we can write:

const http = require('http');

const app = http.createServer((req, res) => {
  res.setHeader('Content-Type', 'application/json');
  res.end(JSON.stringify({
    a: 1
  }));
});

app.listen(3000);

We call createServer to create an HTTP server.

In the callback, we set the Content-Type response header to application/json to set the response data to JSON.

Then we call res.end with the stringified JSON to render the response.

With Express, we just have to write:

res.json({
  foo: "bar"
});

in our middleware.

res.json sets the headers and render the JSON.

Enable HTTPS with Express

We can enable HTTPS with Express by reading the certificate data and use http.createServer to create the server.

For instance, we can write;

const fs = require('fs');
const http = require('http');
const https = require('https');
const privateKey = fs.readFileSync('sslcert/server.key', 'utf8');
const certificate = fs.readFileSync('sslcert/server.crt', 'utf8');
const credentials = {
  key: privateKey,
  cert: certificate
};
const express = require('express');
const app = express();

const httpServer = http.createServer(app);
const httpsServer = https.createServer(credentials, app);
httpServer.listen(3000);
httpsServer.listen(3443);

We read the private key and certificate files from the server.

Then we pass them all in an object and that object is passed to https.createServer to create the server.

Then we call listen on the httpsServer to create the server.

We still have to create the HTTP server to listen to non-secure HTTP requests.

Fix ‘Error: request entity too large’ Error from Express

We can fix the ‘request entity too large’ error by increasing the allowed size of requests.

For instance, we can write:

const bodyParser = require('body-parser');
app.use(bodyParser.json({
  limit: '50mb'
}));
app.use(bodyParser.urlencoded({
  limit: '50mb',
  extended: true
}));

We set limit in the object that we passed in as the argument to set the max size of the request payload.

'50mb' is 50 megabytes.

Download a File from Node Server Using Express

We can send a file download response to the client with res.download .

For instance, we can write:

app.get('/download', (req, res) => {
  const file =  path.join(__dirname, '/upload-folder/movie.mp4');
  res.download(file);
});

The res.download requires the absolute path to the file, so we need to combine __dirname with the rest of the path.

res.download will set the Content-Disposition and other headers automatically and send the file to the client.

Server Static Files with Express

We can serve static files with Express using the express.static middleware.

For instance, we can write:

app.use(express.static(path.join(__dirname, '/public')));

We serve the folder we want to serve as the static folder with the absolute path.

express.static will expose the folder to the client.

Get the JSON POST Data in an Express Application

We can get the JSON data from a POST request by using the req.body property.

For instance, we can write:

const express = require('express');
const bodyParser = require('body-parser');
const app = express.createServer();

app.use(`bodyParser`.`json`());

app.post('/', (req, res) => {
  console.log(req.body);
  response.send(req.body);
});

app.listen(3000);

req.body has the JSON request payload.

The req.body property is available if we ran the bodyParser.json() middleware.

We did that with:

app.use(bodyParser.json());

Remove Documents Using Node.js Mongoose

We can remove a document with Mongoose by using the remove method on the model class.

For example, we can write:

Model.remove({
  _id: id
}, (err) => {
  if (err) {
    console.log(err);
  }
});

Get Uploaded Files with Express

To get uploaded files with Express, we can use the express-fileupload package.

For example, we can write:

const fileupload = require("express-fileupload");
app.use(fileupload());

app.post("/upload", (req, res) => {
  if (!req.files) {
    res.send("File not found");
    return;
  }
  const file = req.files.photo;
  res.send("File Uploaded");
});

We used the fileupload middleware from the express-fileupload package.

To use it, we write:

app.use(fileupload());

The req.files should be available when we upload one or more files.

We can get the files by using the req.files object.

photo is the field name of the file input.

We can replace that with the correct name.

Conclusion

There are a few ways to render JSON response to the client.

We can get file uploads with a middleware package.

Also, we can return a file download response with res.download .

We’ve to use body-parser to parse JSON request payloads.