Categories
Node.js Tips

Node.js Tips — CORS, Excel Files, and CSVs

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Reading Excel File Using Node.js

There are a few packages to let us read Excel files in a Node app.

One package we can use is the node-xlsx package.

To use it, we write:

const xlsx = require('node-xlsx');
const path = require('path');
const obj = xlsx.parse(path.join(__dirname, '/myFile.xlsx'));

We can pass in a path to parse the Excel file from the path.

Also, we can write:

const xlsx = require('node-xlsx');
const path = require('path');
const obj = xlsx.parse(fs.readFileSync(path.join(__dirname, '/myFile.xlsx'));

to parse a buffer, which readFileSync returns.

ExcelJS is another package that we can use to parse workbooks.

For instance, we can write:

const ExcelJS = require('exceljs');
const path = require('path');
const workbook = new Excel.Workbook();
const filename = path.join(__dirname, '/myFile.xlsx');
workbook.xlsx.readFile(filename);
  .then(() => {
    // use workbook
  });

to read from a workbook.

Also, we can write:

const workbook = new Excel.Workbook();
stream.pipe(workbook.xlsx.createInputStream());

to read from a stream.

Pass Variables to JavaScript in Express

We can pass variables to Express templates by passing in a 2nd argument to res.render .

For instance, we can write:

app.get('/foo.js', (req, res) => {
  res.set('Content-Type', 'application/javascript');
  res.render('foo', { foo: { bar: 'baz' } });
});

Then in the foo template, we can write:

<script>
  const foo = <%- JSON.stringify(foo) %>;
</script>

We’re using an EJS template, so we interpolate variables with <%- %> .

Also, we’ve to call JSON.stringify to convert it to a string so we can interpolate it.

Write to a CSV in Node.js

We can use csv-stringify package to create a CSV string from a nested array.

Then we can use fs.writeFile to write the CSV string to a file.

For instance, we can write:

import stringify from 'csv-stringify';
import fs from 'fs';

let data = [];
let columns = {
  id: 'id',
  name: 'name'
};

for (let i = 0; i < 100; i++) {
  data.push([i, `name ${i}`]);
}

stringify(data, { header: true, columns }, (err, output) => {
  if (err) throw err;
  fs.writeFile('file.csv', output, (err) => {
    if (err) throw err;
    console.log('csv saved.');
  });
});

We create a nested array with a columns array for the columns.

The key in columns are the field name. The value in columns are the heading names.

Then we populate our data into the data array.

Next, we call stringify to turn the heading and data into one CSV string.

The first argument is the data.

Then 2nd is an object with some options.

header set to true means we show some header.

columns is for setting the column fields.

The columns are populated by the position of each item in each row.

For each row in data , the first entry is the id and the 2nd is the name .

The callback is called when the CSV conversion is done.

output has the CSV string.

Then we can fs.writeFile to write the string to a file.

Reload Express.js Routes Changes without Manually Restarting Server

We can make an Express app restart automatically when we change the code by using Nodemon.

We can run our app with nodemon instead of node once it’s installed.

To install it, we run:

npm install -g nodemon

Then we run:

nodemon app.js

to use it.

Fix ‘Request header field Authorization is not allowed by Access-Control-Allow-Headers in preflight response’ Error with Express Apps

We can add the cors middleware to allow cross-origin requests.

Then this error should go away.

We can write:

const express = require('express');
const cors = require('cors');
const app = express();
app.use(cors());
app.options('*', cors());

We use the cors middleware to add the Access-Control-Allow-Origin and Access-Control-Allow-Methods headers.

Also, we added the required OPTION routes with the cors middleware.

We allow all routes to accept OPTION requests with '*' .

Conclusion

There are several packages to read Excel files.

We can pass variables to templates with Express.

Also, we can write CSV easily to a file with a library.

To allow our Express app to listen for cross-origin requests, we need the cors middleware.

Categories
Node.js Tips

Node.js Tips — Closing Connections, Templates, and Dates

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Custom Validation Using 2 Fields with Mongoose

To do custom validation using 2 fields with Mongoose, we can create a custom schema and a validation function and pass that as the value of a field in the object we pass into the Schema constructor.

For instance, we can write:

const dateRangeSchema = {
  startDate: { type: Date },
  endDate: { type: Date }
};

const checkDates = (value) => {
  return value.endDate < value.startDate;
}

const schema = new Schema({
  dateRange: { type: dateRangeSchema, validate: checkDates }
});

We created the dateRangeSchema with the startDate and endDate fields.

Also, we created thew checkDates function to return the date range.

And then we pass both into the object to specify the data type for a field.

We can also pass in a callback to validate by writing:

schema.pre('validate', function(next) {
  if (this.startDate > this.endDate) {
    next(new Error('start date is after the end date'));
  } else {
    next();
  }
});

We get the startDate and endDate values from the model and call next with an error if the start date is bigger than the end date.

Otherwise, we call next with nothing to proceed as normal.

Format a Date Coming from MongoDB

We can format a date coming from MongoDB with the toDateString method.

For instance, we can write:

Schema
  .virtual('date')
  .get(function() {
    return this._id.generationTime.toDateString();
  });

We can also use the moment.js library to make our lives easier.

For instance, we can write:

Schema
  .virtual('date')
  .get(function() {
    return moment(this._id.generationTime).format("YYYY-MM-DD HH:mm");
  });

We called the format method to format the moment object created from the moment function into the date format we want.

It’s returned as a string.

Variable in Class Name in Jade/Pug Template

We can interpolate a string into a Jade/Pug template by writing:

div(class="language-#{session.language}")

Listen to All Interfaces Instead of Localhost Only in an Express App

We listen to all interfaces in an Express app by passing in '0.0.0.0' as the 2nd argument of listen.

For instance, we can write:

const express = require('express');
const app = express();
app.listen(3000, '0.0.0.0');

Accessing EJS Variable in Javascript Logic

If we have a route that renders an EJS template, we can render the variable that’s passed in from the render method.

For instance, we can write:

app.get("/post/:title, (req, res) => {
  //...
  res.render("post", { title, description });
}

to create our route,

Then in our EJS template, we can write:

<% if (title) { %>
     <h2>Post</h2>
     <script>
        const postTitle = <%= title %>
     </script>
<% } %>

We get the title from the object in the 2nd argument.

Then we interpolate the title into the template.

Now we can assign that to a variable.

End an Express.js POST Response

We call res.end to end the response to end the response.

For instance, we can write:

res.end('hello');

Or we can render a template and then call res.end :

res.render('some.template');
res.end();

Shut Down an Express Server Gracefully When Its Process Is Killed

We can listen to the SIGTERM and SIGINT signals and then run our code in the handler for those signals and close the server in the callback.

For instance, we can write:

const express = require('express');

const app = express();

app.get('/', (req, res) => res.send('hello'});

const server = app.listen(3000);

setInterval(() => server.getConnections((err, connections) =>  {
  console.log(`${connections} connections currently open`)
}), 1000);

process.on('SIGTERM', shutDown);
process.on('SIGINT', shutDown);

let connections = [];

server.on('connection', connection => {
  connections.push(connection);
  connection.on('close', () => {
    connections = connections.filter(curr => curr !== connection)
  });
});

const shutDown = () => {
  server.close(() => {
    process.exit(0);
  });

  setTimeout(() => {
    process.exit(1);
  }, 10000);

  connections.forEach(curr => curr.end());
  setTimeout(() => connections.forEach(curr => curr.destroy()), 5000);
}

We listen to the SIGTERM and SIGINT signals with the process.on method.

We pass in the shutDown function, where we call server.close to close the connections.

We also exit with code 1 after 10 seconds if the connections aren’t closed successfully.

The connections are stored in the connections array and call end to close each connection.

We also call destroy to destroy the connections after 5 seconds.

The connections cames from the connection event handler where we pushed new connection objects into the connections array.

We also add the close handler with the filter method.

Conclusion

We can validate multiple fields concurrently with Mongoose.

Also, we can shut down connections gracefully.

We can also format the date coming from MongoDB.

Categories
Node.js Tips

Node.js Tips — Mongoose Updates, HTML Emails, and Stubbing Dependencies

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Using Mongoose Promises with async/await

Mongoose methods return promises.

So we can use them with async and await .

For instance, we can write:

const orderEmployees = async(companyID) => {
  try {
    const employees = await User.find({ company: companyID }).exec();
    console.log(employees);
    return employees;
  } catch (err) {
    return 'error occured';
  }
}

We can User.find with a query.

That returns a promise with the resolved result.

We can also catch errors with catch .

Pass Variable to HTML Template in nodemailer

Nodemailer accepts HTML for content.

To make creating HTML emails easier, we can use a templating engine like Handlebars.

For instance, we can write:

const nodemailer = require('nodemailer');
const smtpTransport = require('nodemailer-smtp-transport');
const handlebars = require('handlebars');
const path= require('path');
const fs = require('fs');

const readHTMLFile = (path, callback) => {
  fs.readFile(path, {
    encoding: 'utf-8'
  }, (err, html) => {
    if (err) {
      throw err;
      callback(err);
    } else {
      callback(null, html);
    }
  });
};

smtpTransport = nodemailer.createTransport(smtpTransport({
  host: mailConfig.host,
  secure: mailConfig.secure,
  port: mailConfig.port,
  auth: {
    user: mailConfig.auth.user,
    pass: mailConfig.auth.pass
  }
}));

readHTMLFile(path.join(__dirname, '/template.html'), (err, html) => {
  const template = handlebars.compile(html);
  const replacements = {
    username: "joe"
  };
  const htmlToSend = template(replacements);
  const mailOptions = {
    from: 'from@email.com',
    to: 'to@email.com',
    subject: 'test',
    html: htmlToSend
  };
  smtpTransport.sendMail(mailOptions, (error, response) => {
    if (error) {
      console.log(error);
      callback(error);
    }
  });
});

We import the Handlebars library and call compiler with the template.

The template is read from the readHTMLFile function which uses fs to read the file.

Then it’s passed to the callback.

We get the template with the html parameter.

Once we compile the template, we call template with the object with the properties to interpolate the properties into the template to get the final email.

Then we call smptTransport.sendMail to send the email.

We set the html property with the HTML content.

htmlToSend is an HTML string.

Formatting ISODate from MongoDB

We can format the date object from MongoDB with the toTimeString method.

We just call ut by writing:

const date = new Date("2019-07-14T01:00:00+01:00")
consr str = date.toTimeString();

MongoDB also has the ISODate function to make working with ISO date strings easy.

For example, we can write:

ISODate("2019-07-14T01:00:00+01:00").toLocaleTimeString()

to parse the ISO date string and convert it to a locale-sensitive time string.

We can also get the hours and minutes:

ISODate("2019-07-14T01:00:00+01:00").getHours()
ISODate("2019-07-14T01:00:00+01:00").getMinutes()

Stub Node.js Built-in fs During Testing

We want to mock the fs module so that tests won’t do real file operations.

To do that, we use rewire to stub out required modules.

For instance, if our real code is:

reader.js

const fs = require('fs');
const findFile = (path, callback) => {
  fs.readdir(path, (err, files) => {
     //...
  })
}

We can use rewire to stub out readdir in our test code by writing:

const rewire = require('rewire')
const `reader` = rewire('./`reader`')

const fsStub = {
  readdir(path, callback){
    callback(null, [])
  }
}

`reader`.__set__('fs', fsStub)

`reader`()

Now we can use reader without actually calling readdir since we called __set__ to set the fs module to our stub instead of the read fs module.

Return Updated Collection with Mongoose

We can return an updated collection in Mongoose, then we can get it from findOneAndUpdate with the new option set to true .

For instance, we can write:

Model
  .findOneAndUpdate({
    user_id: data.userId
  }, {
    $set: {
      session_id: id
    }
  }, {
    "new": true
  })
  .exec()
  .then(data => {
    return {
      sid: data.session_id
    }
  })

We call findOneAndUpdate and then we use $set to update a field,

We pass in an object with the new property set to true to return the latest data in the then callback.

Also, we can write:

Lists.findByIdAndUpdate(listId, {
  $push: {
    items: taskId
  }
}, (err, list) => {
  ...
});

We get the latest list with list .

$push appends an item to the items list in the document.

Conclusion

We can get the latest list after the update.

We can stub dependencies with rewire .

We can use HTML templates with Nodemailer.

Mongoose methods return promises.

Categories
Node.js Tips

Node.js Tips — Shell Commands, View Globals, Query with Sequelize

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Run an exe File in a Node.js App

We can run a .exe file in a Node app with the exec method.

For instance, we can write:

const exec = require('child\_process').execFile;

const run = () => {
  exec('program.exe', (err, data) => {
    console.log(err)
    console.log(data.toString());
  });
}

We created a run function to run program.exec with exec .

Once it’s run, the callback is called.

data has the result of running the program.

Escape a String for a Shell Command in Node

We can let the spawn function do the escaping for us.

We pass in the command as the first argument.

And the arguments for the command as the 2nd argument.

For instance, we can write:

const { spawn } = require('child\_process');

const ls = spawn('ls', \['-lh', '/tmp'\]);

ls.stdout.on('data', (data) => {
  console.log(data);
});

ls.stderr.on('data', (data) => {
  console.log(data);
});

ls.on('close', (code) => {
  console.log(code);
});

We get the spawn function from the child_process module.

Then we run the ls commands with some options.

The arguments are in the array.

Then we listen to the stdout and stderr for output.

stderr has the error output.

stdout has the standard output.

We listen to the close event to get the exit code of our command.

code has the exit code.

The arguments are escaped automatically.

We also shouldn’t let anyone pass in anything to those arguments even though they’re escaped.

Close Client Connection with socket.io

We can call socket.disconnect() on the client to disconnect from the server.

Or we can write:

socket.on('end', () => {
  socket.disconnect(0);
});

to listen to the end event emitted from the client on the server

And on the client, we write:

const io = io();
io.emit('end');

to emit the end event.

Express.js View Globals Variables

We can set our global variables for the views by putting them as properties of the app.locals property.

For instance, we can write:

app.locals.title = "app";
app.locals.version = 5;
app.locals.foo = () => {
  return "bar";
};

We add the title , version , and foo properties to app.locals .

They’re all merged together.

Then we can use them right in our Jade template by writing:

\=title
\=version
\=foo()

Then we get:

app
5
bar

displayed on the screen.

If we use connect-flash to provide messages to users, we can use app.set to set the message.

For instance, we can write:

app.use((req, res, next) => {
  app.set('message', req.flash('hello'));
  next();
});

Then we can access message by writing:

\=settings.message

in our template.

Wait Until Element is Displayed in the Node Version of Selenium WebDriver

With the Node version of Selenium WebDriver, we can call driver.wait to do the waiting.

For example, we can write:

driver.wait(() => {
  return driver.isElementPresent(webdriver.By.name("username"));
}, timeout);

We wait for an element with the name attribute username to be present until the timeout in milliseconds is reached.

Run Where Statement with Date with Sequelize

We can run a where statement with a Date object with Sequelize.

To do that, we just use the built-in operator constants as the key of where.

For instance, we can write:

const { Op } = require('sequelize');
const moment = require('moment');

model.findAll({
  where: {
    effective\_date: {
      \[Op.gte\]: moment().subtract(20, 'days').toDate()
    }
  }
})

We use the findAll method for the model class with an object that has the where property to run a where statement.

Then we can put the Op.gte key in the object with the column name as the key.

Op.gte means greater than or equal to.

We compute the date that we want to get the start range from with the moment’s subtract method and then toDate to convert it to the Date instance.

Then we look at records with effective_date 20 days before today or later.

There’s also Op.gt for greater than Op.lt for less than, Op.lte for less than or equal to and Op.ne for not equal.

Conclusion

We can run an .exe file with exec .

Sequelize lets us query with date ranges.

We can add global variables for templates with Express.

We can use wait to wait for something in Selenium.

Shell commands are escaped automatically with spawn .

Categories
Node.js Tips

Node.js Tips — Scheduled Tasks, Jest Tests, and Headers for Tests

Like any kind of apps, there are difficult issues to solve when we write Node apps.

In this article, we’ll look at some solutions to common problems when writing Node apps.

Looping Through Dynamic Test Cases with Jest

We can use the tests.each method to loop through each tets.

For instance, we can write:

test.each([[4, 5, 9], [1, 2, 3], [2, 3, 5]])(
  'add(%i, %i)',
  (a, b, expected) => {
    expect(a + b).toBe(expected);
  },
);

The first argument is the items we want to pass into each test.

The nested array entries are the parameters for the callback.

The 2nd argument is the name of each test.

The 3rd is a callback to let the arguments and run our expectations.

Setting Default Headers with Supertest

We can create a header we can use with our test.

Then we can pass it into the set method.

For instance, we can write:

const request = require("supertest");

const baseUrl = 'http://localhost';
request = request(baseUrl);
const commonHeaders = { authorization: "abc" };

describe("testing", () => {
  it.should('present authorization header to server', (done) => {
    request.get('/foo')
      .set(commonHeaders)
      .set({ foo: "bar" })
      .expect(200, done)
  })

})

commonHeaders has the shared headers between multiple tests.

We just pass that into the set method.

Then we can pass other headers into the set method that are specific to the test.

Read a File Synchronously in Node.js

To read a file synchronously with Node, we can use the readFileSync method.

For instance, we can write:

const fs = require('fs');
const content = fs.readFileSync('file');
console.log(content);

We just pass in the path to the file and the content will be returned.

Base64 Encode a JavaScript Object

We can base64 encode a JavaScript object by converting it to a string with JSON.stringigy .

Then we can call Buffer.from to convert it to a buffer and then call toString to convert it to a base64 string.

For instance, we can write:

const str = JSON.stringify(obj);
const objJsonB64 = Buffer.from(str).toString("base64");

Buffer.from converts the stringified JSON object to a byte stream.

Then we call toString with the 'base64' argument to convert it to a base64 string.

Mongoose Populate Embedded

We can call populate on embedded documents with Mongoose if we define a schema with the related schemas.

For instance, if we have:

const UserSchema = new Schema({
  name: String,
  friends: [{ type: ObjectId, ref: 'User' }]
});

Then we have a self-referencing schema since friends is referencing another User document.

Now we can call populate with friends by writing:

User.
  findOne({ name: 'james' }).
  populate({
    path: 'friends',
    populate: { path: 'friends' }
  });

We call populate the friends to get friends.

And we can use the populate property to get friends of friends.

Running a Function Everyday

We can run a function in a scheduled manner by using the node-schedule package.

For instance, we can write:

import schedule from 'node-schedule'

schedule.scheduleJob('0 0 0 * * *', () => {
  //...
})

We run the schedule.scheduleJob to run the callback with the scheduled specified by the string.

It takes a string in that specifies the schedule in the same way as a cron job.

The first number is the second.

The 2nd is the minute.

The 3rd is the hour.

The 4th is the day of the month.

The 5th is the month.

The 6th is the day of the week.

We specify the hour, minute, and second to specify when it’ll run in the day.

Then the rest are asterisks to that it’ll run every day.

We can also specify the date instead of a cron format string.

For instance, we can write:

const schedule = require('node-schedule');
const date = new Date(2020, 0, 0, 0, 0, 0);

const j = schedule.scheduleJob(date, () => {
  console.log('hello');
});

Then the callback only runs on the given date.

We can also set recurrence rules.

For instance, we can write:

const schedule = require('node-schedule');

const rule = new schedule.RecurrenceRule();
rule.minute = 30;

const j = schedule.scheduleJob(rule, () => {
  console.log('half hour job');
});

We use the schedule.RecurrenceRule constructor to create a rule.

We set the minute in the example above.

But we can also set the second , hour , date , month , year , and dayOfWeek .

Conclusion

node-schedule is a useful scheduler package.

We can set headers for tests with supertest.

Jest can define and run tests dynamically.

MongoDB can reference related documents.

We can convert JavaScript objects into base64 strings.