Categories
Node.js Tips

Node.js Tips — Related Documents, Unzipping Files, and Parsing HTML

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Creating Rooms in Socket.io

To create rooms in socket.io, we just have to join a room by name.

It doesn’t have to be created.

For instance, we can write:

socket.on('create', (room) => {
  socket.join(room);
});

We listen to the create event with the room name.

We get the room parameter with the room we want to join.

Then we can emit the event so join is run:

socket.emit('create', 'room');

'create' is the event name.

'room' is the room name.

Use populate with Multiple Schemas in Mongoose

We can chain the populate calls to select multiple related documents.

For instance, we can write:

Person.find()
  .populate('address')
  .populate('phone')
  .exec((err, results) => {
    //...
  });

to get the Person with the address and phone models.

Then we get all the related documents in the results .

Also, we can call populate with a space-delimited string to specify multiple schemas without call populate multiple times.

For instance, we can write:

Person.find()
  .populate('address phone')
  .exec((err, results) => {
    //...
  });

This is available since version 3.6 of Mongoose.

Download and Unzip a Zip File in Memory in a Node App

We can download and unzip a zip file in memory with the adm-zip package.

To install it, we run:

npm install adm-zip

Then we can download and unzip a file as follows:

const fileUrl = 'https://github.com/notepad-plus-plus/notepad-plus-plus/releases/download/v7.8.6/npp.7.8.6.bin.zip'

const AdmZip = require('adm-zip');
const http = require('http');

http.get(fileUrl, (res) => {
  let data = [];
  let dataLen = 0;

res.on('data', (chunk) => {
  data.push(chunk);
  dataLen += chunk.length;

})
.on('end', () => {
  const buf = Buffer.alloc(dataLen);

  for (let i = 0, pos = 0; i < data.length,; i++) {
    data[i].copy(buf, pos);
    pos += data[i].length;
  }

  const zip = new AdmZip(buf);
  const zipEntries = zip.getEntries();

  for (const zipEntry of zipEntries) {
    if (zipEntry.entryName.match(/readme/))
      console.log(zip.readAsText(zipEntry));
    }
  };
});

We use the http module’s get method to make a GET request to get the zip file.

Then we put the response data chunks into the data array.

Once we did that, we create a new buffer with Buffer.alloc .

Then we put the chunks in data into the buffer according to the position.

Once we copied everything into the buffer, we create a new AdmZip instance with the buffer.

Then we can get the entries from it.

We looped through the entries in the for-of loop.

Use a DOM Parser with Node.js

There are several DOM parser libraries made for Node apps.

One is the jsdom library.

We can use it by writing:

const jsdom = require("jsdom");
const dom = new jsdom.JSDOM(`<p>Hello world</p>`);
dom.window.document.querySelector("p").textContent;

We pass in an HTML string into the JSDOM constructor.

Then we can use the dom.window.document.querySelector method to get an element from the parsed DOM tree.

There’s also the htmlparser2 library.

It’s faster and more flexible than jsdom but the API is more complex.

To use it, we can write:

const htmlparser = require("htmlparser2");
const parser = new htmlparser.Parser({
  onopentag(name, attrib){
    console.log(name, attrib);
  }
}, { decodeEntities: true });
parser.write(`<p>Hello world</p>`);
parser.end();

We use the htmlparser.Parser constructor, which takes an object with the onopentag handler that’s run when the HTML string is parsed.

name has the tag name and attrib has the attributes.

decodeEntities means that we decode the entities within the document.

cheerio is another DOM parser library. It has an API similar to jQuery, so it should be familiar to many people.

We can use it by writing:

const cheerio = require('cheerio');
const $ = cheerio.load(`<p>Hello world</p>`);
$('p').text('bye');
$.html();

cheerio.load loads the HTML string into a DOM tree.

Then we can use the returned $ function to select elements and manipulate it.

Conclusion

We don’t have to create a room explicitly to join it.

We can zip files with a library.

There are many libraries to let us parse HTML strings into a DOM tree.

We can use Mongoose’spopulate to get related documents.

Categories
Node.js Tips

Node.js Tips — Body Parser Errors, Static Files, and Update Collections

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Register Mongoose Models

We can register Mongoose Models by exporting them and then requiring them in our Express app’s entry point.

For instance, we can write the following in models/User.js:

const mongoose = require('mongoose');
const Schema = mongoose.Schema;

const userSchema = new Schema({
  name: String
});

mongoose.model('User', userSchema);

Then in our app.js file, we can write:

mongoose.connect("mongodb://localhost/news");
require("./models/User");

const users = require('./routes/users');

const app = express();

Requiring the ./models/User.js will make it run the code to register the model.

Catch Express body-parser Error

We can catch Express body-parser errors with our own middleware.

For instance, we can write:

app.use( (error, req, res, next) => {
  if (error instanceof SyntaxError) {
    sendError(res, 'syntax error');
  } else {
    next();
  }
});

We can check if error is an instance of SyntaxError and do something with it.

Otherwise, we call next to continue to the next middleware.

Route All Requests to index.html in an Express App

We can route all requests to index.html in an Express app by using rthe sendFile method in one route handler.

For instance, we can write:

const express = require('express')
const server = express();
const path = require('path');

server.use('/public', express.static(path.join(__dirname, '/assets')));

server.get('/foo', (req, res) => {
  res.send('ok')
})

server.get('/*', (req, res) => {
  res.sendFile(path.join(__dirname, '/index.html'));
})

const port = 8000;
server.listen(port)

We redirect all requests to index.html with a catch-all route at the end.

This way, we can still have some routes that do something else.

The foo route responds with ok .

express.static serves static content.

Basic Web Server with Node.js and Express for Serving HTML File and Assets

We can use the express.static middleware to serve static assets.

For instance, we can write:

const express = require('express');
const app = express();
const http = require('http');
const path= require('path');
const httpServer = http.Server(app);

app.use(express.static(path.join(__dirname, '/assets'));

app.get('/', (req, res) => {
  res.sendfile(path.join(__dirname, '/index.html'));
});
app.listen(3000);

We create a server with the express.static middleware to serve static files from the assets folder.

Then we create a GET route to render index.html .

We can also use connect to do the same thing:

const util = require('util');
const connect = require('connect');
const port = 1234;

connect.createServer(connect.static(__dirname)).listen(port);

We call createServer to create a web server.

Then we use connect.static to serve the current app folder as a static folder.

Gzip Static Content with Express

We can use the compression middleware to serve static content with Gzip.

For instance, we can write:

const express = require('express');
const compress = require('compression');
const app = express();
app.use(compress());

Find the Size of a File in Node.js

We can use the statSync method to get the size pf a file.

For instance, we can write:

const fs = require("fs");
const stats = fs.statSync("foo.txt");
const fileSizeInBytes = stats.size

We call statSync to return an object with some data about the file.

The size in bytes is in the size property.

Automatically Add Header to Every “render” Response

We can use the res.header method to return a header for every response.

For instance, we can write:

app.get('/*', (req, res, next) => {
  res.header('X-XSS-Protection', 0);
  next();
});

to create a middleware to set the X-XSS-Protection header to 0.

Update Child Collection with Mongoose

We get the updated collection after the update in the callback.

For instance, we can write:

Lists.findById(listId, (err, list) => {
  if (err) {
    ...
  } else {
    list.items.push(task)
    list.save((err, list) => {
      ...
    });
  }
});

We called findById to get the collection.

Then we call push on the items child collection.

Finally, we call list.save to save the data and get the latest list in the list array of the save callback.

Also, we can use the $push operator to do the push outside the callback:

Lists.findByIdAndUpdate(listId, {$push: { items: task }}, (err, list) => {
  //...
});

We call findByIdAndUpdate with the listId to get the list.

Then we set items with our task object and use $push to push that.

Then we get the latest data from the list parameter in the callback.

Conclusion

We can register Mongoose models by running the code to register the schema.

There’s more than one way to add an item to a child collection.

We can serve static files with Express or Connect.

We can catch body-parser errors.

Categories
Node.js Tips

Node.js Tips — Auto-Scrolling, Delays, Hashing, and Get the Latest Mongo Document

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Using Q delay in Node.js

The Q promise library has a delay method that can delay the execution of code asynchronously.

For instance, we can use it by wiring:

const Q = require('q');

const asyncDelay = (ms) => {
  return Q.delay(ms);
}

asyncDelay(1000)
.then(() => {
  //...
});

We have the asyncDelay function which returns the Q.delay(ms) promise.

It returns a promise that delays the execution of something by the given number of milliseconds.

We can call it and then call then and run the code we want after asyncDelay is done in the then callback.

Also, we can pass in a 2nd argument which can be called a resolved value.

For instance, we can write:

const Q = require('q');

Q.delay(1000, "Success")
  .then((value) => {
      console.log(value);
  });

Then value would be 'Success' since that’s what we passed in as the 2nd argument of delay .

Convert an Image to a base64-Encoded Data URL

To convert an image to a base64-encoded data URL, we can use the readFileSync method to read the file’s data.

Then we pass the content into the Buffer constructor.

And then we can call toString to convert it to a base64 string.

For instance, we can write:

const fs = require('fs');

const file = 'img.jpg';
const bitmap = fs.readFileSync(file);
const dataUrl = new Buffer(bitmap).toString('base64');

We pass the file path into readFileSync .

Then we pass the returned content into the Buffer constructor.

Then we call toString with 'base64' to convert it to a base64 string.

Return the Updated Document with MongoDB findOneAndUpdate

We should set the returnOriginal property to false in the options object to return the updated document in the callback.

For instance, we can write:

db.collection('user_setting').findOneAndUpdate({
  user_id: data.userId
}, {
  $set: data
}, {
  returnOriginal: false
}, (err, res) => {
  if (err) {
    return console.log(err);
  } else {
    console.log(res);
  }
});

The first argument is the query to find the object to update.

The 2nd argument is the data to update the object with.

The 3rd is the options for updating.

We have the returnOriginal option set to false in the 3rd argument.

The last argument is the callback.

res should have the updated results.

Use jQuery Installed with NPM in Express App

To use jQuery installed with NPM in an Express app, we can serve the jQuery’s dist folder with express.static .

For instance, we can write:

const path = require('path');

app.use('/jquery', express.static(path.join(__dirname,  '/node_modules/jquery/dist/')));

Then we can use it in our HTML file by writing:

<script src="/jquery/jquery.js"></script>

Scroll Down Until We Can’t Scroll with Puppeteer

We can check if we’re at the end of the page and use the scrollBy method to scroll.

For instance, we can write:

const distance = 100;
const delay = 100;
while (document.scrollingElement.scrollTop + window.innerHeight < document.scrollingElement.scrollHeight) {
  document.scrollingElement.scrollBy(0, distance);
  await new Promise(resolve => { setTimeout(resolve, delay); });
}

We check if we’re can scroll with:

document.scrollingElement.scrollTop + window.innerHeight < document.scrollingElement.scrollHeight

When scrollTop and innerHeight is less than scrollHeight , we can scroll.

Find OS Username in Node.js

We can get the current user’s username with the os module.

We write:

require("os").userInfo().username

to get the username

In Windows 10, it returns the first name of the owner account that has been used.

There’s also the username module.

We can install it by running:

npm install username

Then we can use it by writing:

const username = require('username');

(async () => {
  console.log(await username());
})();

Hash JavaScript Objects

To hash JavaScript objects, we can use the object-hash module.

For instance, we can write:

const hash = require('object-hash');

const obj = {a: 1, b: 2};
const hashed = hash(obj);

We just call the hash function from the package to return a hashed string.

Conclusion

We can hash JavaScript objects with the object-hash module.

Also, we can get the username with os or a 3rd party module.

The Q promise library has a delay method to delay execution of functions.

We can get the updated object with findOneAndUpdate .

We can scroll until we can’t scroll with Puppeteer by checking the heights.

Categories
Node.js Tips

Node.js Tips — Socket.io, MongoDB and Nested Objects, and Express Issues

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Updating a Nested Array with MongoDB

We can update nested arrays with the native MongoDB driver.

To do that, we can use the update method.

We can write:

Model.update({
    "questions.answers._id": "123"
  }, {
    "$push": {
      "questions.answers.$.answeredBy": "success"
    }
  },
  (err, numAffected) => {
    // ...
  }
);

We called update with a few arguments.

The first is the query for the nested array entry to update.

The 2nd is the $push appends a new entry to the questions.answers array with the answeredBy value.

The $ sign is the positional operator, and it’s a placeholder for a single value.

Then the callback has the result after the push operation.

Express Static Relative to Parent Directory

We can use path.join to get the absolute path of the directory that we want to serve as the static folder.

For example, we can write:

const path = require('path');
const express = require('express');
const app = express();
app.use(express.static(path.join(__dirname, '../public')));

The ../ symbols move one directory level up and we get the public directory from there.

Fix ‘TypeError: Router.use() requires middleware function but got an Object’ Error in Express Apps

To fix this error, we’ve to make sure the routeing module is exported.

For instance, we can write:

users.js

const express = require('express');
const router = express.Router();

router.get('/', (req, res, next) => {
  //Do whatever...
});

module.exports = router;

We have:

module.exports = router;

to export the router .

Then we can use it in another file by writing:

const users = require('./users');

//...
app.use('/users', users);

Socket.io Error Hooking into Express.js

We can incorporate socket.io in our Express app by call the socket.io function.

For instance, we can write:

const express = require('express');
const app = express();
const server = app.listen(port);
const io = require('socket.io')(server);

We just require the socket.io module and call the required function with our server .

server is an http server instance.

Wait for Multiple Async Calls with Node.js

We can use Underscore’s or Lodash’s after method to wait for multiple async calls to be done before calling another function.

For instance, we can write:

const finished = _.after(2, baz);

foo(data, (err) => {
  //...
  finished();
});

bar(data, (err) => {
  //...
  finished();
})

const baz = () => {
  //...
}

The after method returns a finish function that we can call when the async callbacks are called.

Since we want to wait for 2 functions to finish, we pass 2 into after as the first argument.

We call finished in each callback to indicate that it’s finished.

Use Aliases with Node.js’s require Function

We can create aliases for required members by using the destructuring syntax.

For example, we can write:

const { foo: a, bar: b } = require('module');

We import a and b with require .

Then we alias a as foo and b as bar with the destructuring assignment syntax.

Then they’re accessed as foo and bar .

How to Send JavaScript Object with Socket.io

We can send JavaScript objects with socket.io in a few ways.

We can use the send method:

socket.send({ foo: 'b' });

This would convert the object automatically to JSON.

Also, we can use socket.json.send :

socket.json.send({ foo: 'b' });

And we can also use emit :

socket.emit('someEvent', { foo: 'b' });

emit emits an event with name someEvent and an object in the 2nd argument.

Get Error Status Code from HTTP GET

To get the error code for a GET request made with the http module, we can get it from the statusCode property of res .

For instance, we can write:

const https = require('https');

https.get('https://example.com/error', (res) => {
  console.log(res.statusCode);

  res.on('data', (d) => {
    process.stdout.write(d);
  });
})
.on('error', function(e) {
  console.error(e);
});

The status code is in the res.statusCode property to get the status code.

Conclusion

We can use update to update a nested array with MongoDB.

The status code of a request made with http is in the statusCode property.

We can send objects with socket.io.

Static folders should be served with absolute paths with Express.

We can use after to wait for multiple async calls to complete before calling another function.

Categories
Node.js Tips

Node.js Tips — POST Requests, Express Route Params, S3 Data, and More

Like any kind of apps, there are difficult issues to solve when we write Node apps. In this article, we’ll look at some solutions to common problems when writing Node apps.

Get the Browser Language in Express

To get the browser language in an Express app, we can use the req.headers['accept-language'] property to get the accept-language header.

Then request.acceptsLanguages method also gets the accepted language header.

For instance, we can write:

const express = require('express');
app.get('/translation', (req, res) => {
  const lang = req.acceptsLanguages('fr', 'es', 'en');
  if (lang) {
    console.log(lang);
  }
  else {
    console.log('None of [fr, es, en] is accepted');
  }
  //...
});

acceptsLanguages makes a list of languages that the app can accept.

If one of the values are in the Accepts-Language header, then it returns true ,

Otherwise, it returns false .

There’s also the accepts module that does the same thing.

To install it, we run:

npm install accepts

Then we can use it by writing:

const accepts = require('accepts');
const http = require('http');

const app = (req, res) => {
  const accept = accepts(req);
  switch (accept.type(['json', 'html'])) {
    case 'json':
      res.setHeader('Content-Type', 'application/json');
      res.write(JSON.stringify({ hello: 'world' ));
      break;
    case 'html':
      res.setHeader('Content-Type', 'text/html');
      res.write('<b>hello, world</b>');
      break;
    default:
      res.setHeader('Content-Type', 'text/plain');
      res.write('hello, world');
      break;
  }
  res.end()
}

http.createServer(app).listen(3000)

We use the accept module to check the Accept header. We can check the kind of language that it requests in the Accept header. It has the languages method to get the browser language from the Accepts-Language header.

How to Construct HTTP POST Request with Form Params

We can construct an HTTP POST request with form parameters with the querystring module.

To install it, we run:

npm i querystring

Then we can write:

const querystring = require('querystring');

axios.post(
  '/login',
  querystring.stringify({
    username: 'abcd',
    password: '12345',
  }),
  {
    headers: {
      "Content-Type": "application/x-www-form-urlencoded"
    }
  }
)
.then((response) => {
  console.log(response);
});

We called the querystring.stringify method to construct a query string with the username and password keys. Also, we set the headers with the Content-Type header set to “application/x-www-form-urlencoded” . This way, we send the query string as form data.

How to Configure Dynamic Routes with Express.js

We can configure dynamic routes with Express by allowing parameters.

For instance, we can write:

app.get('/article/:name', (req, res) => {
  res.render(req.params.name);
});

We can also restrict the allowed values of id by writing:

app.get('/article/:name(article|article2|article3)', (req, res) => {
  const name = req.params.name;
  res.render(name);
});

We restrict the name parameter to be one of article1 , article2 ,or article3 .

In both examples, we get the parameter value by using the req.params.name property.

req.params has the value of name .

Catch Express body-parser Error

We can create our own middleware to catch errors raised by body-parser .

For instance, we can write:

app.use((error, req, res, next) => {
  if (error instanceof SyntaxError) {
    sendError(res, 'error');
  } else {
    next();
  }
});

We check if a SyntaxError is raised by using error instanceof SyntaxError. If it is, then we call sendError to process the error. Otherwise, we call next to call the next middleware in the chain.

Return Hostname with Node.js

We can use the os module’s hostname method to return the hostname.

For instance, we can write:

const os = require("os");
const hostname = os.hostname();

to do that.

Listing Objects from AWS S3 in a Node App

We can get items stored in S3 with the aws-sdk package.

For instance, we can write:

const AWS = require('aws-sdk');
AWS.config.update({
  accessKeyId: 'key',
  secretAccessKey: 'secret',
  region: 'region'
});

const s3 = new AWS.S3();

const params = {
  Bucket: 'bucket',
  Delimiter: '/',
  Prefix: '/path/prefix'
}

s3.listObjects(params, (err, data) => {
  if (err){
    return console.log(err);
  }
  console.log(data);
});

We authenticate with AWS. Then we get the bucket with the Bucket property.

Delimiter is the path separator.

Prefix is the prefix for the path and bucket data.

Then we call listObjects with the path.

The callback’s data parameter has the bucket data.

Conclusion

If it’s not created when we join it, then it’ll be created. We can get data from S3. Express routes can be made dynamic with parameters. We can send form data as a query with a POST request with Axios