Manipulating files is a basic operation for any program. Since Node.js is a server-side platform and can interact with the computer that it’s running on directly, being able to manipulate files is a basic feature. Fortunately, Node.js has a fs module built into its library. It has many functions that can help with manipulating files and folders.
File and folder operations that are supported include basic ones like manipulating and opening files in directories. Likewise, it can do the same for files. It can do this both synchronously and asynchronously. It has an asynchronous API that has functions that support promises. Also, it can show statistics for a file.
Almost all the file operations that we can think of can be done with the built-in fs module. In this article, we will introduce the fs module and how to construct paths that can be used to open files and open files with it.
We will also experiment with the fs promises API to do equivalent operations that exist in the regular fs module if they exist in the fs promises API. The fs promise API functions return promises so this let us run asynchronous operations sequentially much more easily.
To use the fs module, we just have to require it like in the following line of code:
const fs = require('fs');
Asynchronous Operations
The asynchronous versions of the file functions take a callback that has an error and the result of the operation as the argument. If the operation is done successfully, then null or undefined will be passed in for the error argument, which should the first parameter of the callback function passed in for each function. Asynchronous operations aren’t executed sequentially. For example, if we want to delete a file called file, we can write:
const fs = require('fs');
fs.unlink('/file', (err) => {
if (err) throw err;
console.log('successfully deleted file');
});
We have the err parameter which has the error data if it exists, which will have it if an error occurred. Asynchronous operations aren’t done sequentially, to do multiple operations sequentially, we can either convert it to a promise or nest operations in the callback function of the earlier operation. For example, if we want to rename and then check for the existence of a file a file, we can shouldn’t write the following:
fs.rename('/file1', '/file2', (err) => {
if (err) throw err;
console.log('renamed complete');
});
fs.stat('/file1', (err, stats) => {
if (err) throw err;
console.log(stats);
});
because they aren’t guaranteed to be run sequentially. Therefore, we should instead write:
fs.rename('./file1', './file2', (err) => {
if (err) throw err;
console.log('renamed complete');
fs.stat('./file2', (err, stats) => {
if (err) throw err;
console.log(stats);
})
;});
However, this gets messy if we want to do lots of operations sequentially. We have too much nesting of callback functions if we have multiple file operations. Too much nesting of callback functions is called callback hell and it makes reading and debugging the code very hard and confusing. Therefore, we should use something like promises do run asynchronous operations. For example, we should rewrite the example above like the following code:
const fsPromises = require("fs").promises;
(async () => {
try {
await fsPromises.rename("./files/file1.txt", "./files/file2.txt");
const stats = await fsPromises.stat("./files/file2.txt");
console.log(stats);
} catch (error) {
console.log(error);
}
})();
We used the fs promises API which has file operations functions that return promises. This is much cleaner and takes advantage of the async and await the syntax for chaining promises. Note that the fs promises API has a warning that it’s experimental.
However, it has been quite stable so far and it’s good for basic file manipulation that requires chaining of multiple file operations. Note that we are catching errors with the try...catch block. async functions look like synchronous functions but can only return promises.
Each of the examples above will output something like the following:
Stats {
dev: 3605029386,
mode: 33206,
nlink: 1,
uid: 0,
gid: 0,
rdev: 0,
blksize: undefined,
ino: 6192449489177455,
size: 0,
blocks: undefined,
atimeMs: 1572568634188,
mtimeMs: 1572568838068,
ctimeMs: 1572569087450.1968,
birthtimeMs: 1572568634187.734,
atime: 2019-11-01T00:37:14.188Z,
mtime: 2019-11-01T00:40:38.068Z,
ctime: 2019-11-01T00:44:47.450Z,
birthtime: 2019-11-01T00:37:14.188Z }
Synchronous Operations
Synchronous file operations usually have the word Sync at the end of its name and are called line by line. We get the error with the try...catch block. If we rewrite the example above as a synchronous operation, we can write:
const fs = require('fs');
try {
fs.unlinkSync('./files/file1.txt');
console.log('successfully deleted ./files/file1.txt');
} catch (err) {
console.err(err);
}
We get the err binding with the catch clause to get the error data. The issue with synchronous operations in Node.js is that it holds up the processor until the process is finished, which makes long, resource-intensive operations hang the computer. Therefore, it slows down the execution of the program.
File Paths
Most file operation functions accept a path in the form of a string, a Buffer or a URL object using the file: protocol. String paths are interpreted as UTF-8 character sequences which identifies the absolute or relative path of the file or folder.
Relative paths are resolved relative to the current working directory, which is whatever is returned by the process.cwd() function. For example, we can open a file with a relative path like in the following code:
const fs = require("fs");
fs.open("./files/file.txt", "r", (err, fd) => {
if (err) throw err;
fs.close(fd, err => {
if (err) throw err;
});
});
The dot before the first slash indicates the current work directory of the relative path. Or using the promise API we can rewrite it to the following code:
(async () => {
try {
const fileHandle = await fsPromises.open("./files/file.txt", "r");
console.log(fileHandle);
} catch (error) {
console.log(error);
}
})();
Note that the promise API doesn’t have the close function. The regular fs API puts the file operation permission flag as the second argument. r stands for read-only.
The fs promise API has an open function and you can get the fd object, which stands by file descriptor, which is the reference to the file we opened. We can call the close function by passing the file descriptor fd .
The fs promise API doesn’t have this so the file can’t be closed with the promise API.
The fs API also accepts a URL object as a reference to the file location. It has to have the file: protocol and they must be absolute paths. For example, we can create a URL object and pass it into the read function to read a file. We can write the following code to do this:
const fs = require("fs");
const fileUrl = new URL(`file://${__dirname}/files/file.txt`);
fs.open(fileUrl, "r", (err, fd) => {
if (err) throw err;
fs.close(fd, err => {
if (err) throw err;
});
});
It does the exact same thing as with using relative paths. It’s just slightly more complex since we have to get the URL object and get the current directory of the code with the __dirname object. On Windows, any path that has a hostname prepended to the path is interpreted as a UNC path, which is the path to access a file over the local area network or Internet. For example, if we have the following:
fs.readFileSync(new URL('file://hostname/path/to/file'));
Then that will be interpreted as accessing a file on the server with the hostname hostname .
Any file URL that has a drive letter will be interpreted as an absolute path on Windows. For example, if we have the following path:
fs.readFileSync(new URL('file://c:/path/to/file'));
Then it’ll try to access the file in the c:\path\to\file path. Any file that has no hostname must have drive letters. Therefore only file paths with the same format above are valid file paths for a URL object on Windows. Paths that start with a drive letter must have a colon after the drive letter. On all other platforms, file: URLs with a hostname are unsupported and file paths like fs.readFileSync(new URL('file://hostname/path/to/file')); will throw an error.
Any file: URL with the escaped slash character will throw an error on all platforms, so any of the following examples would be invalid and throw errors. On Windows, these examples would throw errors:
fs.readFileSync(new URL('file:///C:/path/%2F'));
fs.readFileSync(new URL('file:///C:/path/%2f'));
And on POSIX systems, these would fail:
fs.readFileSync(new URL('file:///pathh/%2F'));
fs.readFileSync(new URL('file:///path/%2f'));
On Windows, file URLs with the encoded backslash character will throw an error, so the following examples are invalid:
fs.readFileSync(new URL('file:///D:/path/%5C'));
fs.readFileSync(new URL('file:///D:/path/%5c'));
On POSIX systems the kernel maintains a list of opened files and resources for every process. Each file that’s opened is assigned a simple numeric identifier called the file descriptor.
The operating system uses the file description to identify and track each specific file. On Windows, the tracking file uses a similar mechanism and file descriptors are still used for tracking files and resources opened by various processes.
Node.js does the hard work of assigning file descriptors to the resources so we don’t have to do it manually. This is handy for cleaning up resources that are opened.
In Node.js, the fs.open() function is used to open files and assign a new file descriptor to opened files. After processing is done, it can be closed by the close function so that the open resources can be closed and cleaned up. This can be used like in the following code:
const fs = require("fs");
fs.open("./files/file.txt", "r", (err, fd) => {
if (err) throw err;
console.log(fd);
fs.fstat(fd, (err, stat) => {
if (err) throw err;
console.log("Stat", stat); fs.close(fd, err => {
if (err) throw err;
console.log('Closed');
});
});
});
If we run the code above, we get output that resembles the following:
3
Stat Stats {
dev: 3605029386,
mode: 33206,
nlink: 1,
uid: 0,
gid: 0,
rdev: 0,
blksize: undefined,
ino: 22799473115106240,
size: 0,
blocks: undefined,
atimeMs: 1572569358035.625,
mtimeMs: 1572569358035.625,
ctimeMs: 1572569358035.625,
birthtimeMs: 1572569358035.625,
atime: 2019-11-01T00:49:18.036Z,
mtime: 2019-11-01T00:49:18.036Z,
ctime: 2019-11-01T00:49:18.036Z,
birthtime: 2019-11-01T00:49:18.036Z }
Closed
In the code above, we opened the file with the open function, which provided the fd object which contains the file descriptor in the callback function, we can use that to get the information about the file with the fstat function. And after we’re done, we can close the opened file resource with the close function.
We done the file open operation, read the file metadata and then close the file with the close function.
The promise API do not have the fstat or close function so we can’t do the same thing with it.
The Node.js run time platform has a fs module built into its standard library. It has many functions that can help with manipulating files and folders.
File and folder operations that are supported include basic ones like manipulating and opening files in directories.
Likewise, it can do the same for files. It can do this both synchronously and asynchronously. It has an asynchronous API that has functions that support promises.
Also, it can show statistics for a file. Almost all the file operations that we can think of can be done with the built-in fs module. In this article, we will introduce the fs module and how to construct paths that can be used to open files and open files with it.
We will also experiment with the fs promises API to do equivalent operations that exist in the regular fs module if they exist in the fs promises API.
The fs promise API functions return promises so this lets us run asynchronous operations sequentially much more easily. We barely scratch the surface, so stay tuned for Part 2 of this series.
