Categories
Node.js Tips

Node.js Tips — Shell Commands, View Globals, Query with Sequelize

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Run an exe File in a Node.js App

We can run a .exe file in a Node app with the exec method.

For instance, we can write:

const exec = require('child\_process').execFile;

const run = () => {
  exec('program.exe', (err, data) => {
    console.log(err)
    console.log(data.toString());
  });
}

We created a run function to run program.exec with exec .

Once it’s run, the callback is called.

data has the result of running the program.

Escape a String for a Shell Command in Node

We can let the spawn function do the escaping for us.

We pass in the command as the first argument.

And the arguments for the command as the 2nd argument.

For instance, we can write:

const { spawn } = require('child\_process');

const ls = spawn('ls', \['-lh', '/tmp'\]);

ls.stdout.on('data', (data) => {
  console.log(data);
});

ls.stderr.on('data', (data) => {
  console.log(data);
});

ls.on('close', (code) => {
  console.log(code);
});

We get the spawn function from the child_process module.

Then we run the ls commands with some options.

The arguments are in the array.

Then we listen to the stdout and stderr for output.

stderr has the error output.

stdout has the standard output.

We listen to the close event to get the exit code of our command.

code has the exit code.

The arguments are escaped automatically.

We also shouldn’t let anyone pass in anything to those arguments even though they’re escaped.

Close Client Connection with socket.io

We can call socket.disconnect() on the client to disconnect from the server.

Or we can write:

socket.on('end', () => {
  socket.disconnect(0);
});

to listen to the end event emitted from the client on the server

And on the client, we write:

const io = io();
io.emit('end');

to emit the end event.

Express.js View Globals Variables

We can set our global variables for the views by putting them as properties of the app.locals property.

For instance, we can write:

app.locals.title = "app";
app.locals.version = 5;
app.locals.foo = () => {
  return "bar";
};

We add the title , version , and foo properties to app.locals .

They’re all merged together.

Then we can use them right in our Jade template by writing:

\=title
\=version
\=foo()

Then we get:

app
5
bar

displayed on the screen.

If we use connect-flash to provide messages to users, we can use app.set to set the message.

For instance, we can write:

app.use((req, res, next) => {
  app.set('message', req.flash('hello'));
  next();
});

Then we can access message by writing:

\=settings.message

in our template.

Wait Until Element is Displayed in the Node Version of Selenium WebDriver

With the Node version of Selenium WebDriver, we can call driver.wait to do the waiting.

For example, we can write:

driver.wait(() => {
  return driver.isElementPresent(webdriver.By.name("username"));
}, timeout);

We wait for an element with the name attribute username to be present until the timeout in milliseconds is reached.

Run Where Statement with Date with Sequelize

We can run a where statement with a Date object with Sequelize.

To do that, we just use the built-in operator constants as the key of where.

For instance, we can write:

const { Op } = require('sequelize');
const moment = require('moment');

model.findAll({
  where: {
    effective\_date: {
      \[Op.gte\]: moment().subtract(20, 'days').toDate()
    }
  }
})

We use the findAll method for the model class with an object that has the where property to run a where statement.

Then we can put the Op.gte key in the object with the column name as the key.

Op.gte means greater than or equal to.

We compute the date that we want to get the start range from with the moment’s subtract method and then toDate to convert it to the Date instance.

Then we look at records with effective_date 20 days before today or later.

There’s also Op.gt for greater than Op.lt for less than, Op.lte for less than or equal to and Op.ne for not equal.

Conclusion

We can run an .exe file with exec .

Sequelize lets us query with date ranges.

We can add global variables for templates with Express.

We can use wait to wait for something in Selenium.

Shell commands are escaped automatically with spawn .

Categories
Node.js Tips

Node.js Tips — Scheduled Tasks, Jest Tests, and Headers for Tests

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Looping Through Dynamic Test Cases with Jest

We can use the tests.each method to loop through each tets.

For instance, we can write:

test.each([[4, 5, 9], [1, 2, 3], [2, 3, 5]])(
  'add(%i, %i)',
  (a, b, expected) => {
    expect(a + b).toBe(expected);
  },
);

The first argument is the items we want to pass into each test.

The nested array entries are the parameters for the callback.

The 2nd argument is the name of each test.

The 3rd is a callback to let the arguments and run our expectations.

Setting Default Headers with Supertest

We can create a header we can use with our test.

Then we can pass it into the set method.

For instance, we can write:

const request = require("supertest");

const baseUrl = 'http://localhost';
request = request(baseUrl);
const commonHeaders = { authorization: "abc" };

describe("testing", () => {
  it.should('present authorization header to server', (done) => {
    request.get('/foo')
      .set(commonHeaders)
      .set({ foo: "bar" })
      .expect(200, done)
  })

})

commonHeaders has the shared headers between multiple tests.

We just pass that into the set method.

Then we can pass other headers into the set method that are specific to the test.

Read a File Synchronously in Node.js

To read a file synchronously with Node, we can use the readFileSync method.

For instance, we can write:

const fs = require('fs');
const content = fs.readFileSync('file');
console.log(content);

We just pass in the path to the file and the content will be returned.

Base64 Encode a JavaScript Object

We can base64 encode a JavaScript object by converting it to a string with JSON.stringigy .

Then we can call Buffer.from to convert it to a buffer and then call toString to convert it to a base64 string.

For instance, we can write:

const str = JSON.stringify(obj);
const objJsonB64 = Buffer.from(str).toString("base64");

Buffer.from converts the stringified JSON object to a byte stream.

Then we call toString with the 'base64' argument to convert it to a base64 string.

Mongoose Populate Embedded

We can call populate on embedded documents with Mongoose if we define a schema with the related schemas.

For instance, if we have:

const UserSchema = new Schema({
  name: String,
  friends: [{ type: ObjectId, ref: 'User' }]
});

Then we have a self-referencing schema since friends is referencing another User document.

Now we can call populate with friends by writing:

User.
  findOne({ name: 'james' }).
  populate({
    path: 'friends',
    populate: { path: 'friends' }
  });

We call populate the friends to get friends.

And we can use the populate property to get friends of friends.

Running a Function Everyday

We can run a function in a scheduled manner by using the node-schedule package.

For instance, we can write:

import schedule from 'node-schedule'

schedule.scheduleJob('0 0 0 * * *', () => {
  //...
})

We run the schedule.scheduleJob to run the callback with the scheduled specified by the string.

It takes a string in that specifies the schedule in the same way as a cron job.

The first number is the second.

The 2nd is the minute.

The 3rd is the hour.

The 4th is the day of the month.

The 5th is the month.

The 6th is the day of the week.

We specify the hour, minute, and second to specify when it’ll run in the day.

Then the rest are asterisks to that it’ll run every day.

We can also specify the date instead of a cron format string.

For instance, we can write:

const schedule = require('node-schedule');
const date = new Date(2020, 0, 0, 0, 0, 0);

const j = schedule.scheduleJob(date, () => {
  console.log('hello');
});

Then the callback only runs on the given date.

We can also set recurrence rules.

For instance, we can write:

const schedule = require('node-schedule');

const rule = new schedule.RecurrenceRule();
rule.minute = 30;

const j = schedule.scheduleJob(rule, () => {
  console.log('half hour job');
});

We use the schedule.RecurrenceRule constructor to create a rule.

We set the minute in the example above.

But we can also set the second , hour , date , month , year , and dayOfWeek .

Conclusion

node-schedule is a useful scheduler package.

We can set headers for tests with supertest.

Jest can define and run tests dynamically.

MongoDB can reference related documents.

We can convert JavaScript objects into base64 strings.

Categories
Node.js Tips

Node.js Tips — Mail, Mocha Tests, S3 and JSON

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

‘UnhandledPromiseRejectionWarning: This error originated either by throwing inside of an async function without a catch block’ Error in Express Apps

We’ve to handle all promise errors manually in our Express apps.

For instance, we should write:

router.get("/fetch-email", authCheck, async (req, res, next) => {
  try {
    const emailFetch = await gmaiLHelper.getEmails(req.user.id, '/messages', req.user.accessToken)
    emailFetch = emailFetch.data
    res.send(emailFetch);
  } catch (err) {
    next(err);
  }
})

In our async route handler function, we have a catch block to catch errors.

Then we call next to call our error handling middleware.

The error handler middleware should be added after the routes so that next will call it.

Sort Sequelize.js Query by Date

We can sort a Sequelie query by date by adding an order property to the object we pass into findAll .

For instance, we can write:

Post.findAll({ limit: 50, order: '"updatedAt" DESC' })

Then we sort the updateAt column in descending order.

Sending an Email to Multiple Recipients with nodemailer

We can send an email to multiple recipients with nodemailer .

To do that, we can set the to property to an array.

For instance, we can write:

const mailList = [
  'foo@bar.com',
  'baz@bar.com',
  'qux@bar.com',
];

const msg = {
  from: "from@bar.com",
  subject: "Hello",
  text: "Hello",
  cc: "cc@bar.com",
  to: mailList
}

smtpTransport.sendMail(msg, (err) => {
  if (err) {
    return console.log(err);
  }
});

We just set to to our mailList array and send the message with sendMail .

Also, we can pass in a comma-separated string to do the same thing.

For instance, we can write:

const mailList = [
    'foo@bar.com',
    'baz@bar.com',
    'qux@bar.com',
  ]
  .join(',');
const msg = {
  from: "from@bar.com",
  subject: "Hello",
  text: "Hello",
  cc: "cc@bar.com",
  to: mailList
}
smtpTransport.sendMail(msg, (err) => {
  if (err) {
    return console.log(err);
  }
});

We called join to join the email address by commas.

Then the rest is the same.

Socket.io custom client ID

We can generate a custom ID for a client with socket.io.

We create our own generateId method to do so.

For instance, we can write:

const app = require('express')();
const http = require('http').Server(app);
const io = require('socket.io')(http);

io.engine.generateId = (req) => {
  return 1
}

io.on('connection', (socket) => {
  console.log(socket.id);
})

We add the generateId method, then we can access the returned result with socket.id .

We can replace 1 with our own auto-generated ID.

Expect Mocha to Fail a Test

We can expect Mocha to fail a test with assert.fail .

For instance, we can write:

it("should fail", () => {
  assert.fail("actual", "expected", "Error message");
});

We pass in the actual result, expected result, and error message if something isn’t expected.

Wait for All Images to load with Puppeteer Then Take Screenshot

We can call goto with the waitUntil option.

For instance, we can write:

await page.goto('https://www.example.com/', {"waitUntil" : "networkidle0" });

'networkidle0' means navigation is considered finished when there are no more than 0 network connections for at least 500 ms.

There’s also 'networkidle2' which means no more than 2 network connections for at least 500 ms.

Then we can call page.screenshot to take the screenshot after this line.

Parse Stream to Object with S3 Download

If we have a download from S3, we can parse it to an object with data.Body.toString() .

For example, we can write:

const options = {
  BucketName: 'mybucket',
  ObjectName: 'path/to/my.json',
  ResponseContentType: 'application/json'
};

s3.GetObject(options, (err, data) => {
  const fileContents = data.Body.toString();
  const json = JSON.parse(fileContents);
  console.log(json);
});

We pass in the bucket, pathname, and response data type.

We use s3.GetObject to do the download with the given options.

Since we have JSON, we can convert it to a string with data.Body.toString() .

Then we can use JSON.parse to parse it to an object.

Conclusion

We can convert an item downloaded from S3 into a string.

We’ve to handle errors in Express route handlers ourselves.

Also, we can send email to multiple recipients with nodemailer.

We can sort by date with Sequelize queries.

With socket.io, we can generate our own IDs.

Categories
Node.js Tips

Node.js Tips — Related Documents, Unzipping Files, and Parsing HTML

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Creating Rooms in Socket.io

To create rooms in socket.io, we just have to join a room by name.

It doesn’t have to be created.

For instance, we can write:

socket.on('create', (room) => {
  socket.join(room);
});

We listen to the create event with the room name.

We get the room parameter with the room we want to join.

Then we can emit the event so join is run:

socket.emit('create', 'room');

'create' is the event name.

'room' is the room name.

Use populate with Multiple Schemas in Mongoose

We can chain the populate calls to select multiple related documents.

For instance, we can write:

Person.find()
  .populate('address')
  .populate('phone')
  .exec((err, results) => {
    //...
  });

to get the Person with the address and phone models.

Then we get all the related documents in the results .

Also, we can call populate with a space-delimited string to specify multiple schemas without call populate multiple times.

For instance, we can write:

Person.find()
  .populate('address phone')
  .exec((err, results) => {
    //...
  });

This is available since version 3.6 of Mongoose.

Download and Unzip a Zip File in Memory in a Node App

We can download and unzip a zip file in memory with the adm-zip package.

To install it, we run:

npm install adm-zip

Then we can download and unzip a file as follows:

const fileUrl = 'https://github.com/notepad-plus-plus/notepad-plus-plus/releases/download/v7.8.6/npp.7.8.6.bin.zip'

const AdmZip = require('adm-zip');
const http = require('http');

http.get(fileUrl, (res) => {
  let data = [];
  let dataLen = 0;

res.on('data', (chunk) => {
  data.push(chunk);
  dataLen += chunk.length;

})
.on('end', () => {
  const buf = Buffer.alloc(dataLen);

  for (let i = 0, pos = 0; i < data.length,; i++) {
    data[i].copy(buf, pos);
    pos += data[i].length;
  }

  const zip = new AdmZip(buf);
  const zipEntries = zip.getEntries();

  for (const zipEntry of zipEntries) {
    if (zipEntry.entryName.match(/readme/))
      console.log(zip.readAsText(zipEntry));
    }
  };
});

We use the http module’s get method to make a GET request to get the zip file.

Then we put the response data chunks into the data array.

Once we did that, we create a new buffer with Buffer.alloc .

Then we put the chunks in data into the buffer according to the position.

Once we copied everything into the buffer, we create a new AdmZip instance with the buffer.

Then we can get the entries from it.

We looped through the entries in the for-of loop.

Use a DOM Parser with Node.js

There are several DOM parser libraries made for Node apps.

One is the jsdom library.

We can use it by writing:

const jsdom = require("jsdom");
const dom = new jsdom.JSDOM(`<p>Hello world</p>`);
dom.window.document.querySelector("p").textContent;

We pass in an HTML string into the JSDOM constructor.

Then we can use the dom.window.document.querySelector method to get an element from the parsed DOM tree.

There’s also the htmlparser2 library.

It’s faster and more flexible than jsdom but the API is more complex.

To use it, we can write:

const htmlparser = require("htmlparser2");
const parser = new htmlparser.Parser({
  onopentag(name, attrib){
    console.log(name, attrib);
  }
}, { decodeEntities: true });
parser.write(`<p>Hello world</p>`);
parser.end();

We use the htmlparser.Parser constructor, which takes an object with the onopentag handler that’s run when the HTML string is parsed.

name has the tag name and attrib has the attributes.

decodeEntities means that we decode the entities within the document.

cheerio is another DOM parser library. It has an API similar to jQuery, so it should be familiar to many people.

We can use it by writing:

const cheerio = require('cheerio');
const $ = cheerio.load(`<p>Hello world</p>`);
$('p').text('bye');
$.html();

cheerio.load loads the HTML string into a DOM tree.

Then we can use the returned $ function to select elements and manipulate it.

Conclusion

We don’t have to create a room explicitly to join it.

We can zip files with a library.

There are many libraries to let us parse HTML strings into a DOM tree.

We can use Mongoose’spopulate to get related documents.

Categories
Node.js Tips

Node.js Tips — Body Parser Errors, Static Files, and Update Collections

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Register Mongoose Models

We can register Mongoose Models by exporting them and then requiring them in our Express app’s entry point.

For instance, we can write the following in models/User.js:

const mongoose = require('mongoose');
const Schema = mongoose.Schema;

const userSchema = new Schema({
  name: String
});

mongoose.model('User', userSchema);

Then in our app.js file, we can write:

mongoose.connect("mongodb://localhost/news");
require("./models/User");

const users = require('./routes/users');

const app = express();

Requiring the ./models/User.js will make it run the code to register the model.

Catch Express body-parser Error

We can catch Express body-parser errors with our own middleware.

For instance, we can write:

app.use( (error, req, res, next) => {
  if (error instanceof SyntaxError) {
    sendError(res, 'syntax error');
  } else {
    next();
  }
});

We can check if error is an instance of SyntaxError and do something with it.

Otherwise, we call next to continue to the next middleware.

Route All Requests to index.html in an Express App

We can route all requests to index.html in an Express app by using rthe sendFile method in one route handler.

For instance, we can write:

const express = require('express')
const server = express();
const path = require('path');

server.use('/public', express.static(path.join(__dirname, '/assets')));

server.get('/foo', (req, res) => {
  res.send('ok')
})

server.get('/*', (req, res) => {
  res.sendFile(path.join(__dirname, '/index.html'));
})

const port = 8000;
server.listen(port)

We redirect all requests to index.html with a catch-all route at the end.

This way, we can still have some routes that do something else.

The foo route responds with ok .

express.static serves static content.

Basic Web Server with Node.js and Express for Serving HTML File and Assets

We can use the express.static middleware to serve static assets.

For instance, we can write:

const express = require('express');
const app = express();
const http = require('http');
const path= require('path');
const httpServer = http.Server(app);

app.use(express.static(path.join(__dirname, '/assets'));

app.get('/', (req, res) => {
  res.sendfile(path.join(__dirname, '/index.html'));
});
app.listen(3000);

We create a server with the express.static middleware to serve static files from the assets folder.

Then we create a GET route to render index.html .

We can also use connect to do the same thing:

const util = require('util');
const connect = require('connect');
const port = 1234;

connect.createServer(connect.static(__dirname)).listen(port);

We call createServer to create a web server.

Then we use connect.static to serve the current app folder as a static folder.

Gzip Static Content with Express

We can use the compression middleware to serve static content with Gzip.

For instance, we can write:

const express = require('express');
const compress = require('compression');
const app = express();
app.use(compress());

Find the Size of a File in Node.js

We can use the statSync method to get the size pf a file.

For instance, we can write:

const fs = require("fs");
const stats = fs.statSync("foo.txt");
const fileSizeInBytes = stats.size

We call statSync to return an object with some data about the file.

The size in bytes is in the size property.

Automatically Add Header to Every “render” Response

We can use the res.header method to return a header for every response.

For instance, we can write:

app.get('/*', (req, res, next) => {
  res.header('X-XSS-Protection', 0);
  next();
});

to create a middleware to set the X-XSS-Protection header to 0.

Update Child Collection with Mongoose

We get the updated collection after the update in the callback.

For instance, we can write:

Lists.findById(listId, (err, list) => {
  if (err) {
    ...
  } else {
    list.items.push(task)
    list.save((err, list) => {
      ...
    });
  }
});

We called findById to get the collection.

Then we call push on the items child collection.

Finally, we call list.save to save the data and get the latest list in the list array of the save callback.

Also, we can use the $push operator to do the push outside the callback:

Lists.findByIdAndUpdate(listId, {$push: { items: task }}, (err, list) => {
  //...
});

We call findByIdAndUpdate with the listId to get the list.

Then we set items with our task object and use $push to push that.

Then we get the latest data from the list parameter in the callback.

Conclusion

We can register Mongoose models by running the code to register the schema.

There’s more than one way to add an item to a child collection.

We can serve static files with Express or Connect.

We can catch body-parser errors.