Categories
JavaScript Nodejs

Node.js FS Module — Truncating and Removing Files

Manipulating files and directories are basic operations for any program. Since Node.js is a server side platform and can interact with the computer that it’s running on directly, being able to manipulate files is a basic feature. Fortunately, Node.js has a fs module built into its library. It has many functions that can help with manipulating files and folders. File and directory operation that are supported include basic ones like manipulating and opening files in directories. Likewise, it can do the same for files. It can do this both synchronously and asynchronously. It has an asynchronous API that have functions that support promises. Also it can show statistics for a file. Almost all the file operations that we can think of can be done with the built in fs module. In this article, we will truncate files with the truncate family of functions and remove files and symbolic links with the unlink family of functions.

Truncate Files with the fs.truncate Family of Functions

We can truncate files with the Node.js truncate family of functions. Truncating a file is to shrink the file to a specified size. To truncate a file asynchronously we can use the truncate function. The function takes 3 arguments. The first argument is the path object, which can be a string, a Buffer object or an URL object. When a file descriptor is passed into the first argument instead of the path, it will automatically call ftruncate to truncate the file with the given file descriptor. Passing in a file descriptor is deprecated and may thrown an error in the future. The second argument is the length of the file in bytes that you want to truncate it to. The default value is 0. Any extra data bigger than the size specified is lost when the size is smaller than the original size. The third argument is a callback function that is run when the truncate operation ends. It takes an err parameter which is null when the truncate operation succeeds and has an object with the error information otherwise.

To truncate a file with the truncate function, we can write the following code:

const fs = require("fs");  
const truncateFile = "./files/truncateFile.txt";

fs.truncate(truncateFile, 1, err => {  
  if (err) throw err;  
  console.log("File truncated");  
});

If we run the code above, there should be a single byte of content left in the file you’re truncating.

The synchronous version of the truncate function is the truncateSync function. The function takes 2 arguments. The first argument is the path object, which can be a string, a Buffer object or an URL object. When a file descriptor is passed into the first argument instead of the path, it will automatically call ftruncateSync to truncate the file with the given file descriptor. Passing in a file descriptor is deprecated and may thrown an error in the future .The second argument is the length of the file in bytes that you want to truncate it to. The default value is 0. Any extra data bigger than the size specified is lost when the size is smaller than the original size. It returns undefined .

We can use the truncateSync function like in the following code:

const fs = require("fs");  
const truncateFile = "./files/truncateFile.txt";

try {  
  fs.truncateSync(truncateFile, 1);  
  console.log("File truncated");  
} catch (error) {  
  console.error(error);  
}

If we run the code above, there should be a single byte of content left in the file you’re truncating.

There’s also a promise version of the truncate function. The function takes 2 arguments. The first argument is the path object, which can be a string, a Buffer object or an URL object.The second argument is the length of the file in bytes that you want to truncate it to. The default value is 0. Any extra data bigger than the size specified is lost when the size is smaller than the original size. It returns a promise that is resolved with no arguments when the operation is successful.

To truncate a file with the promise version of the truncate function, we can write the following code:

const fsPromises = require("fs").promises;  
const truncateFile = "./files/truncateFile.txt";

(async () => {  
  try {  
    await fsPromises.truncate(truncateFile, 1);  
    console.log("File truncated");  
  } catch (error) {  
    console.error(error);  
  }  
})();

If we run the code above, again there should be a single byte of content left in the file you’re truncating.

Remove Files and Symbolic Links with the fs.unlink Family of Functions

We can remove a file or a symbolic link with the unlink function. The function takes 2 arguments. The first argument is the path object, which can be a string, a Buffer object or an URL object.The second argument is a callback function that takes an err object, which is null when the file or symbolic link removal operation succeeds, and has the error data if the operation failed. The unlink function doesn’t work on directories in any state. To remove directories, we should use the rmdir function.

To use the unlink function to remove a file, we can something like the code below:

const fs = require("fs");  
const fileToDelete = "./files/deleteFile.txt";

fs.unlink(fileToDelete, err => {  
  if (err) {  
    throw err;  
  }  
  console.log("Removal complete!");  
});

If we run the code above, the file that’s to be deleted should be gone.

The synchronous version of the unlink function is the unlinkSync function. The function takes one argument. The only argument is the path object, which can be a string, a Buffer object or an URL object. It returns undefined .

We can use it like in the following code:

const fs = require("fs");  
const fileToDelete = "./files/deleteFile.txt";

try {  
  fs.unlinkSync(fileToDelete);  
  console.log("Removal complete!");  
} catch (error) {  
  console.error(error);  
}

There’s also a promise version of the unlink function. The function takes one argument. The only argument is the path object, which can be a string, a Buffer object or an URL object. It returns a promise that’s resolved with no argument when the operation is successful.

If we run the code above, the file that’s to be deleted should be gone.

We can use it like in the following code:

const fsPromises = require("fs").promises;  
const fileToDelete = "./files/deleteFile.txt";

(async () => {  
  try {  
    await fsPromises.unlink(fileToDelete);  
    console.log("Removal complete!");  
  } catch (error) {  
    console.error(error);  
  }  
})();

If we run the code above, the file that’s to be deleted should be gone.

The promise version of the unlink function is a much better choice than the unlinkSync function when you want to do multiple things sequentially that includes a call to the unlink function since it doesn’t tie up the whole program waiting for the file or symbolic link deletion operation to complete before continuing to program other parts of the program.

In this article, we truncate files with the truncate family of functions and remove files and symbolic links with the unlink family of functions. The truncate family of functions let us specify the number of bytes to keep while truncating the rest of the file. The unlink family of functions remove files and symbolic links. If we want to do these operations sequentially with other operations, the promise versions of these functions. Even though the API is still experimental, it is much better than the synchronous versions of these functions since it allows for sequential and asynchronous operations with promises. Also, it helps avoid callback hell where we nest promises in too many levels.

Categories
Express JavaScript Nodejs

Easy Debugging with the errorhandler Express Middleware

The errorhandler Express middleware can be used to handle errors during app development.

In this article, we’ll look at how to use it to debug our code during development.

Why do we Need This Middleware?

We need this for debugging purposes. It shows the full error stack trace and internals of any object that’s passed to this module will be sent back to the client if an error occurs.

It’ll display as many details as possible about errors. Object details will be fully displayed. The data can be displayed as HTML, JSON, and plain text.

When the object is a standard Error object, the string provided by the stack property will be returned in HTML or text responses.

When an object is a non-error object, the result of util.inspect will be returned in HTML or text format.

Error involving JSON responses will be an object with enumerable properties from the object in the response.

Installing the Package

We can install it by running:

npm install errorhandler

Options

The errorhandler middleware takes a few options to control what to return.

log

The log property lets us provide a function to be called with the error and a string representation of the error.

It can be used to write the error to any desired location or set to false to only send the error back in the response.

The function has the signature (err, str, req, res) where err is the Error object, str is a string representation of the error, req is the request object and res is the response object.

This function is run after the response has been written.

The default value for this option is true unless process.env.NODE_ENV has the value 'test' .

Other possible values include true , which logs errors using console.error(str) , or false , which only sends the error back in the response.

Examples

We can use it as follows:

const express = require('express');  
const express = require('express');  
const bodyParser = require('body-parser');  
const errorHandler = require('errorhandler');  
const app = express();
app.use(bodyParser.json());
app.get('/', (req, res, next) => {  
  throw new Error('error');  
  res.send();  
})
app.use(errorHandler());
app.listen(3000);

Since our route throws an error, we should get objects with the error string and object logged.

We should get something like::

Connect  
500 Error: error  
   at app.get (/home/runner/index.js:9:9)  
   at Layer.handle [as handle_request] (/home/runner/node_modules/express/lib/router/layer.js:95:5)  
   at next (/home/runner/node_modules/express/lib/router/route.js:137:13)  
   at Route.dispatch (/home/runner/node_modules/express/lib/router/route.js:112:3)  
   at Layer.handle [as handle_request] (/home/runner/node_modules/express/lib/router/layer.js:95:5)  
   at /home/runner/node_modules/express/lib/router/index.js:281:22  
   at Function.process_params (/home/runner/node_modules/express/lib/router/index.js:335:12)  
   at next (/home/runner/node_modules/express/lib/router/index.js:275:10)  
   at jsonParser (/home/runner/node_modules/body-parser/lib/types/json.js:110:7)  
   at Layer.handle [as handle_request] (/home/runner/node_modules/express/lib/router/layer.js:95:5)

Displayed on our screen.

We can add our own custom logging function as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const errorHandler = require('errorhandler');  
const app = express();  
const errorNotification = (err, str, req) => {  
  console.log(str);  
}
app.use(bodyParser.json());
app.get('/', (req, res, next) => {  
  throw new Error('error');  
  res.send();  
})
app.use(errorHandler({ log: errorNotification }));
app.listen(3000);

In the code above, we added our own errorNotification function. All we did was logging the error string with our errorNotification handler function.

Then we should get:

Error: error  
    at app.get (/home/runner/index.js:12:9)  
    at Layer.handle [as handle_request] (/home/runner/node_modules/express/lib/router/layer.js:95:5)  
    at next (/home/runner/node_modules/express/lib/router/route.js:137:13)  
    at Route.dispatch (/home/runner/node_modules/express/lib/router/route.js:112:3)  
    at Layer.handle [as handle_request] (/home/runner/node_modules/express/lib/router/layer.js:95:5)  
    at /home/runner/node_modules/express/lib/router/index.js:281:22  
    at Function.process_params (/home/runner/node_modules/express/lib/router/index.js:335:12)  
    at next (/home/runner/node_modules/express/lib/router/index.js:275:10)  
    at jsonParser (/home/runner/node_modules/body-parser/lib/types/json.js:110:7)  
    at Layer.handle [as handle_request] (/home/runner/node_modules/express/lib/router/layer.js:95:5)

From the console.log output in addition to what we see on the page.

Conclusion

With the errorhandler middleware, we can debug our Express apps more easily because we can see more information about the error like the stack trace and the object with more error information.

Also, we can see the request and response object when the error occurred.

The error data can be displayed in HTML, JSON or plain text.

We can add it by including the middleware with use , but we have to be careful not run use this in production since it shows internal data about the error.

Categories
Express JavaScript

Guide to the Express Response Object — Files and JSON

The Express response object lets us send a response to the client.

Various kinds of responses like strings, JSON, and files can be sent. Also, we can send various headers and status code to the client in addition to the body.

In this article, we’ll look at various properties of the response object, including files and JSON responses.

Methods

res.download

res.download sends a file response to the server.

For example, if we have:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))
app.get('/', (req, res) => {  
  res.download('./public/foo.txt');  
})
app.listen(3000, () => console.log('server started'));

Then when a request is made to this route, then we’ll get a file downloaded.

We can save a file with a different than the file on the server by writing:

const express = require('express');  
const bodyParser = require('body-parser');
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
app.get('/', (req, res) => {  
  res.download('./files/foo.txt', 'foobar.txt');  
});
app.listen(3000, () => console.log('server started'));

The file foo.txt from the server will be saved as foobar.txt by the client.

The download method also takes an error handler in case the download response fails.

For example, we can handle errors with the built-in error handler as follows:

const express = require('express');  
const bodyParser = require('body-parser');
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
app.get('/', (req, res, next) => {  
  res.download('./files/foo.txt', 'foobar.txt', err => next(err));  
});
app.listen(3000, () => console.log('server started'));

We called next with the error object passed in to pass it onto the error handler.

res.end([data] [, encoding])

res.end ends the response process. The method comes from response.end() of the http module.

We can use it to end the response without sending any data. To respond with data, we can use res.send() or other methods.

For example, we can use it as follows:

const express = require('express');  
const bodyParser = require('body-parser');
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
app.get('/', (req, res) => {  
  res.status(404).end();  
});
app.listen(3000, () => console.log('server started'));

Then we sent a 404 response back to the client with no data.

res.format(object)

The res.format method takes the Accept HTTP header from the request object and use res.accepts to select a handler for the request.

For example, we can use it as follows:

const express = require('express');  
const bodyParser = require('body-parser');
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
app.get('/', (req, res) => {  
  res.format({  
    'text/plain': () => {  
      res.send('hi');  
    }, 'text/html': () => {  
      res.send('<p>hi</p>');  
    }, 'application/json': () => {  
      res.json({ message: 'hi' });  
    }, 'default': () => {  
      res.status(406).end();  
    }  
  })  
});
app.listen(3000, () => console.log('server started'));

If we send a GET request to / with Accept header’s value set to text/plain , we get hi . If we send text/html as the Accept header’s value, we get <p>hi</p> , and so on.

default specifies the default response if none of the response types are matched.

It’ll work even if we abbreviate the MIME type names to something shorter:

const express = require('express');  
const bodyParser = require('body-parser');
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
app.get('/', (req, res) => {  
  res.format({  
    text: () => {  
      res.send('hi');  
    }, html: () => {  
      res.send('<p>hi</p>');  
    }, json: () => {  
      res.json({ message: 'hi' });  
    }, default: () => {  
      res.status(406).end();  
    }  
  })  
});
app.listen(3000, () => console.log('server started'));

Then we get the same thing as before.

res.get(field)

The get method returns the HTTP response header specified by field . The match isn’t case sensitive.

For example, we can use it as follows:

const express = require('express');  
const bodyParser = require('body-parser');
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
app.get('/', (req, res) => {    
  res.send(res.get('Content-Type'));  
});
app.listen(3000, () => console.log('server started'));

res.json([body])

res.json lets us send a JSON response to the client. The parameter can be any JSON type, including object, array, string, Boolean, number, or null.

For example, if we have:

const express = require('express')  
const app = express()
app.use(express.json())  
app.use(express.urlencoded({ extended: true }))
app.get('/', (req, res) => {  
  res.json({ message: 'hi' });  
})
app.listen(3000, () => console.log('server started'));

Then we get:

{"message":"hi"}

as the response.

res.jsonp([body])

We can use this to send a JSON response with JSONP support. It returns a callback with the JSON body passed in.

For example, we can use it as follows:

const express = require('express');  
const bodyParser = require('body-parser');
const app = express();
app.use(bodyParser.json());  
app.use(bodyParser.urlencoded({ extended: true }));
app.get('/', (req, res) => {  
  res.jsonp({ user: 'foo' });  
});
app.listen(3000, () => console.log('server started'));

If we send a GET request with ?callback=cb as the query string, then we get:

// typeof cb === 'function' && cb({"user":"foo"});

as the response. If we leave out the callback search parameter, then we get a normal JSON response:

{  
    "user": "foo"  
}

Conclusion

We can send various kinds of responses easily with the response object.

To send files, we can use download .

To send different things according to the Accept request header, we can use the format method.

Finally, to send JSON, we can use the json or jsonp methods depending on if we need to call a callback function on the client or not.

Categories
Express JavaScript Nodejs

Parse Cookies in Requests in Express Apps with Cookie-Parser

By default, Express 4.x or later doesn’t come with any middleware to parse cookies.

To do this, we can add the cookie-parser middleware.

In this article, we’ll look at how to use it to parse cookies sent with requests.

Adding the Package

We can install the package by running:

npm install cookie-parser

Then we can write:

const express = require('express')  
const cookieParser = require('cookie-parser')  
  
const app = express()  
app.use(cookieParser())

to add the middleware to our app.

Methods

cookieParser(secret, options)

The cookie-parser middleware’s cookieParser function takes a secret string or array of strings as the first argument and an options object as the second argument.

The secret is optional. If it’s not specified it won’t parse signed cookies. If a string is provided, it’ll be used as the secret. If an array of strings is provided, then each secret will be tried for decoding the signed cookie

options is an object that is passed into the cookie.parse as the second option.

cookieParser.JSONCookie(str)

The JSONCookie parses a JSON cookie. This returns the pared JSON value if it’s a JSON cookie. It’ll return the passed value otherwise.

cookieParser.JSONCookies(cookies)

The method will iterate over the keys and call JSONCookie on each value and replace the original value with the parsed value. This returns the same object that’s passed in.

cookieParser.signedCookie(str, secret)

signedCookie parses a cookie as a signed cookie. It’ll return the parsed unsigned value if it was a signed cookie and the signature is valid. If the value wasn’t signed, then the original value is returned. If the value is signed but the cookie can’t be validated with the secret, then false is returned.

cookieParser.signedCookies(cookies, secret)

This method will iterate over the keys and check if any value is a signed cookie. If it’s signed and the signature is valid, then the key will be deleted from the object and added to the new object that’s returned.

The secret can be an array or string. If it’s a string, then it’ll check against the string secret. Otherwise, each cookie will be checked with each string in the array.

The JSONCookie, JSONCookies, signedCookie and signedCookies methods will be automatically invoked depending on the type of cookie sent from the client. Parsing Unsigned Cookies

A simple example is parsing unsigned cookies as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const cookieParser = require('cookie-parser');  
const app = express();
app.use(bodyParser.json());  
app.use(cookieParser());
app.get('/', (req, res) => {  
  res.send(req.cookies);  
});
app.listen(3000);

Then when we send a GET request with the Cookie header’s value set to foo=bar , then we get:

{  
    "foo": "bar"  
}

as the response since req.cookies has the parsed cookie.

Parsing Signed Cookies

We can send a signed cookie and parse it as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const cookieParser = require('cookie-parser');  
const app = express();  
const secret = 'secret';
app.use(bodyParser.json());  
app.use(cookieParser(secret));
app.get('/cookie', (req, res) => {  
  res  
    .cookie('foo', 'bar', { signed: true })  
    .send();  
});
app.get('/', (req, res) => {    
  res.send(req.signedCookies);  
});
app.listen(3000);

The code has the /cookie endpoint to send the signed cookie to the client.

{ signed: true }

will make Express sign the cookie.

Then from the client, when we make a request to / , we get the signed cookie via req.signedCookies and so we get:

{  
    "foo": "bar"  
}

as the response from / .

Conclusion

We can use the cookie-parser middleware to parse the cookies.

It can parse both signed and unsigned cookies. To parse a signed cookie, we just have to sign our cookie with a secret and then cookie-parser can decode the cookie if we pass in the secret to the cookieParser middleware function.

The JSONCookie, JSONCookies, signedCookie and signedCookies methods will be automatically invoked depending on the type of cookie sent from the client.

Categories
Express JavaScript

Guide to the Express Request Object — Methods

The request object lets us get the information about requests made from the client in middlewares and route handlers.

In this article, we’ll look at the properties of Express’s request object methods in detail, including getting headers and query strings.

Methods

The request object has various methods.

req.accepts(types)

The accepts method checks the Accept request header for the value that’s sent with the header. It takes a string with the value of the data type that we want to check if it’s sent with the Accept request header.

It returns the best match, or if nothing matches, false is returned.

If false is returned, then the 406 response code should be returned since the data type isn’t accepted by our route.

For example, we can use it as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();
app.get('/', (req, res) => {  
  res.send(req.accepts('html'));  
})
app.listen(3000);

Then when we send the Accept header with a value that has html in it, then it’ll return html.

Otherwise, false will be returned.

Other examples include:

req.accepts('text/html')  
req.accepts(['json', 'text'])  
req.accepts('application/json')  
req.accepts('image/png')  
req.accepts('png')  
req.accepts(['html', 'json'])

req.acceptsCharsets(charset [, …]), req.acceptsEncodings(encoding [, …]), req.acceptsLanguages(lang [, …])

The method returns the first accepted charset of the specified character set. The charset is from the request’s Accept-Charset HTTP header field.

For example, we can use it as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();
app.get('/', (req, res) => {  
  res.send(req.acceptsCharsets('utf8'));  
})
app.listen(3000);

Then we get back the character set in the Accept-Charset HTTP header if it’s set.

acceptsEncodings returns the encoding based on the Accepts-Encoding HTTP header; and acceptsLanguages returns the encoding based on the Accepts-Language HTTP header.

req.get(field)

We can get the header field value with the get method. It takes the key of the header and it’s case-insensitive.

For example, we can use it as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();
app.get('/', (req, res) => {  
  res.send(req.get('content-type'));  
})
app.listen(3000);

Then when we make a request to the / route with the Content-Type header’s value set to text/cmd, then we get back text/cmd.

req.is(type)

The is method returns the Content-Type header’s value if it matches the type passed in. If the request has no body, the null is returned. Otherwise, false is returned.

For example, we can use it as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();
app.get('/', (req, res) => {  
  res.send(req.is('html'));  
})
app.listen(3000);

Then we get null since it’s a GET request, which has no request body.

On the other hand, if we have:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();
app.post('/', (req, res) => {  
  res.send(req.is('html'));  
})
app.listen(3000);

When we make a POST request to the/ route with the Content-Type header set to text/html, then we get back html .

If the Content-Type header is set to something without html, then we get back false .

req.param(name [, defaultValue])

req.param gets the value from the query string’s search parameter with the given key.

For example, we can use it as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();
app.post('/', (req, res) => {  
  res.send(req.param('name', 'Joe'));  
})
app.listen(3000);

The code above get the search parameter value with the name key. The default value is set to 'Joe'.

Then when we make a request to /?name=jane, then we get jane. Otherwise, if the name search parameter isn’t specified, then we get Joe.

req.param is deprecated, so we should use req.params, req.body or req.query, as applicable instead.

Lookup is of query string, URL parameter and body values are performed in the following order:

  • req.params
  • req.body
  • req.query

req.range(size[, options])

The range method parses the Range header. The size parameter has the maximum size of the resource. The options parameter is an object which can have the following property:

  • combine — a boolean that specifies if overlapping and adjacent ranges should be combined.

For example, we can use it as follows:

const express = require('express');  
const bodyParser = require('body-parser');  
const app = express();app.get('/', (req, res) => {  
  const range = req.range(20000);  
  if (range && range.type === 'bytes') {  
    res.json(range);  
  }  
  else {  
    res.send();  
  }  
})app.listen(3000);

Then when we make a GET request to / with the Range header set to bytes=200–1000, 2000–6576, 19000- , we get:

[  
    {  
        "start": 200,  
        "end": 1000  
    },  
    {  
        "start": 2000,  
        "end": 6576  
    },  
    {  
        "start": 19000,  
        "end": 19999  
    }  
]

since we specified the size parameter to be 20000, so the last end value is 19999.

If we didn’t specify bytes in the value, we get nothing.

Conclusion

The request object has methods to parse query strings and get various headers like the Range, Accepts and Content-Type headers.

We can get the Range header with range, Accepts with accepts and Content-Type with is.

The get method can get any request header field’s value.