Manipulating files is a basic operation for any program. Since Node.js is a server-side platform and can interact with the computer that it’s running on directly, being able to manipulate files is a basic feature. Fortunately, Node.js has a fs module built into its library. It has many functions that can help with manipulating files and folders.

File and folder operations that are supported include basic ones like manipulating and opening files in directories. Likewise, it can do the same for files. It can do this both synchronously and asynchronously. It has an asynchronous API that has functions that support promises. Also, it can show statistics for a file.

Almost all the file operations that we can think of can be done with the built-in fs module. In this article, we will introduce the fs module and how to construct paths that can be used to open files and open files with it.

We will also experiment with the fs promises API to do equivalent operations that exist in the regular fs module if they exist in the fs promises API. The fs promise API functions return promises so this let us run asynchronous operations sequentially much more easily.

To use the fs module, we just have to require it like in the following line of code:

const fs = require('fs');

Asynchronous Operations

The asynchronous versions of the file functions take a callback that has an error and the result of the operation as the argument. If the operation is done successfully, then null or undefined will be passed in for the error argument, which should the first parameter of the callback function passed in for each function. Asynchronous operations aren’t executed sequentially. For example, if we want to delete a file called file, we can write:

const fs = require('fs');  
  
fs.unlink('/file', (err) => {  
  if (err) throw err;  
  console.log('successfully deleted file');  
});

We have the err parameter which has the error data if it exists, which will have it if an error occurred. Asynchronous operations aren’t done sequentially, to do multiple operations sequentially, we can either convert it to a promise or nest operations in the callback function of the earlier operation. For example, if we want to rename and then check for the existence of a file a file, we can shouldn’t write the following:

fs.rename('/file1', '/file2', (err) => {  
  if (err) throw err;  
  console.log('renamed complete');  
});  
fs.stat('/file1', (err, stats) => {  
  if (err) throw err;  
  console.log(stats);  
});

because they aren’t guaranteed to be run sequentially. Therefore, we should instead write:

fs.rename('./file1', './file2', (err) => {  
  if (err) throw err;  
  console.log('renamed complete'); 

  fs.stat('./file2', (err, stats) => {  
    if (err) throw err;  
    console.log(stats);  
  })  
;});

However, this gets messy if we want to do lots of operations sequentially. We have too much nesting of callback functions if we have multiple file operations. Too much nesting of callback functions is called callback hell and it makes reading and debugging the code very hard and confusing. Therefore, we should use something like promises do run asynchronous operations. For example, we should rewrite the example above like the following code:

const fsPromises = require("fs").promises;

(async () => {  
  try {  
    await fsPromises.rename("./files/file1.txt", "./files/file2.txt");  
    const stats = await fsPromises.stat("./files/file2.txt");  
    console.log(stats);  
  } catch (error) {  
    console.log(error);  
  }  
})();

We used the fs promises API which has file operations functions that return promises. This is much cleaner and takes advantage of the async and await the syntax for chaining promises. Note that the fs promises API has a warning that it’s experimental.

However, it has been quite stable so far and it’s good for basic file manipulation that requires chaining of multiple file operations. Note that we are catching errors with the try...catch block. async functions look like synchronous functions but can only return promises.

Each of the examples above will output something like the following:

Stats {  
  dev: 3605029386,  
  mode: 33206,  
  nlink: 1,  
  uid: 0,  
  gid: 0,  
  rdev: 0,  
  blksize: undefined,  
  ino: 6192449489177455,  
  size: 0,  
  blocks: undefined,  
  atimeMs: 1572568634188,  
  mtimeMs: 1572568838068,  
  ctimeMs: 1572569087450.1968,  
  birthtimeMs: 1572568634187.734,  
  atime: 2019-11-01T00:37:14.188Z,  
  mtime: 2019-11-01T00:40:38.068Z,  
  ctime: 2019-11-01T00:44:47.450Z,  
  birthtime: 2019-11-01T00:37:14.188Z }

Synchronous Operations

Synchronous file operations usually have the word Sync at the end of its name and are called line by line. We get the error with the try...catch block. If we rewrite the example above as a synchronous operation, we can write:

const fs = require('fs');  
  
try {  
  fs.unlinkSync('./files/file1.txt');  
  console.log('successfully deleted ./files/file1.txt');  
} catch (err) {  
  console.err(err);  
}

We get the err binding with the catch clause to get the error data. The issue with synchronous operations in Node.js is that it holds up the processor until the process is finished, which makes long, resource-intensive operations hang the computer. Therefore, it slows down the execution of the program.

File Paths

Most file operation functions accept a path in the form of a string, a Buffer or a URL object using the file: protocol. String paths are interpreted as UTF-8 character sequences which identifies the absolute or relative path of the file or folder.

Relative paths are resolved relative to the current working directory, which is whatever is returned by the process.cwd() function. For example, we can open a file with a relative path like in the following code:

const fs = require("fs");

fs.open("./files/file.txt", "r", (err, fd) => {  
  if (err) throw err;  
  fs.close(fd, err => {  
    if (err) throw err;  
  });  
});

The dot before the first slash indicates the current work directory of the relative path. Or using the promise API we can rewrite it to the following code:

(async () => {  
  try {  
    const fileHandle = await fsPromises.open("./files/file.txt", "r");  
    console.log(fileHandle);  
  } catch (error) {  
    console.log(error);  
  }  
})();

Note that the promise API doesn’t have the close function. The regular fs API puts the file operation permission flag as the second argument. r stands for read-only.

The fs promise API has an open function and you can get the fd object, which stands by file descriptor, which is the reference to the file we opened. We can call the close function by passing the file descriptor fd .

The fs promise API doesn’t have this so the file can’t be closed with the promise API.

The fs API also accepts a URL object as a reference to the file location. It has to have the file: protocol and they must be absolute paths. For example, we can create a URL object and pass it into the read function to read a file. We can write the following code to do this:

const fs = require("fs");  
const fileUrl = new URL(`file://${__dirname}/files/file.txt`);

fs.open(fileUrl, "r", (err, fd) => {  
  if (err) throw err;  
  fs.close(fd, err => {  
    if (err) throw err;  
  });  
});

It does the exact same thing as with using relative paths. It’s just slightly more complex since we have to get the URL object and get the current directory of the code with the __dirname object. On Windows, any path that has a hostname prepended to the path is interpreted as a UNC path, which is the path to access a file over the local area network or Internet. For example, if we have the following:

fs.readFileSync(new URL('file://hostname/path/to/file'));

Then that will be interpreted as accessing a file on the server with the hostname hostname .

Any file URL that has a drive letter will be interpreted as an absolute path on Windows. For example, if we have the following path:

fs.readFileSync(new URL('file://c:/path/to/file'));

Then it’ll try to access the file in the c:\path\to\file path. Any file that has no hostname must have drive letters. Therefore only file paths with the same format above are valid file paths for a URL object on Windows. Paths that start with a drive letter must have a colon after the drive letter. On all other platforms, file: URLs with a hostname are unsupported and file paths like fs.readFileSync(new URL('file://hostname/path/to/file')); will throw an error.

Any file: URL with the escaped slash character will throw an error on all platforms, so any of the following examples would be invalid and throw errors. On Windows, these examples would throw errors:

fs.readFileSync(new URL('file:///C:/path/%2F'));  
fs.readFileSync(new URL('file:///C:/path/%2f'));

And on POSIX systems, these would fail:

fs.readFileSync(new URL('file:///pathh/%2F'));  
fs.readFileSync(new URL('file:///path/%2f'));

On Windows, file URLs with the encoded backslash character will throw an error, so the following examples are invalid:

fs.readFileSync(new URL('file:///D:/path/%5C'));  
fs.readFileSync(new URL('file:///D:/path/%5c'));

On POSIX systems the kernel maintains a list of opened files and resources for every process. Each file that’s opened is assigned a simple numeric identifier called the file descriptor.

The operating system uses the file description to identify and track each specific file. On Windows, the tracking file uses a similar mechanism and file descriptors are still used for tracking files and resources opened by various processes.

Node.js does the hard work of assigning file descriptors to the resources so we don’t have to do it manually. This is handy for cleaning up resources that are opened.

In Node.js, the fs.open() function is used to open files and assign a new file descriptor to opened files. After processing is done, it can be closed by the close function so that the open resources can be closed and cleaned up. This can be used like in the following code:

const fs = require("fs");

fs.open("./files/file.txt", "r", (err, fd) => {  
  if (err) throw err;  
  console.log(fd);  
  fs.fstat(fd, (err, stat) => {  
    if (err) throw err;  
    console.log("Stat", stat); fs.close(fd, err => {  
      if (err) throw err;  
      console.log('Closed');  
    });  
  });  
});

If we run the code above, we get output that resembles the following:

3  
Stat Stats {  
  dev: 3605029386,  
  mode: 33206,  
  nlink: 1,  
  uid: 0,  
  gid: 0,  
  rdev: 0,  
  blksize: undefined,  
  ino: 22799473115106240,  
  size: 0,  
  blocks: undefined,  
  atimeMs: 1572569358035.625,  
  mtimeMs: 1572569358035.625,  
  ctimeMs: 1572569358035.625,  
  birthtimeMs: 1572569358035.625,  
  atime: 2019-11-01T00:49:18.036Z,  
  mtime: 2019-11-01T00:49:18.036Z,  
  ctime: 2019-11-01T00:49:18.036Z,  
  birthtime: 2019-11-01T00:49:18.036Z }  
Closed

In the code above, we opened the file with the open function, which provided the fd object which contains the file descriptor in the callback function, we can use that to get the information about the file with the fstat function. And after we’re done, we can close the opened file resource with the close function.

We done the file open operation, read the file metadata and then close the file with the close function.

The promise API do not have the fstat or close function so we can’t do the same thing with it.

The Node.js run time platform has a fs module built into its standard library. It has many functions that can help with manipulating files and folders.

File and folder operations that are supported include basic ones like manipulating and opening files in directories.

Likewise, it can do the same for files. It can do this both synchronously and asynchronously. It has an asynchronous API that has functions that support promises.

Also, it can show statistics for a file. Almost all the file operations that we can think of can be done with the built-in fs module. In this article, we will introduce the fs module and how to construct paths that can be used to open files and open files with it.

We will also experiment with the fs promises API to do equivalent operations that exist in the regular fs module if they exist in the fs promises API.

The fs promise API functions return promises so this lets us run asynchronous operations sequentially much more easily. We barely scratch the surface, so stay tuned for Part 2 of this series.

Manipulating files and directories are basic operations for any program. Since Node.js is a server side platform and can interact with the computer that it’s running on directly, being able to manipulate files is a basic feature. Fortunately, Node.js has a fs module built into its library. It has many functions that can help with manipulating files and folders. File and directory operation that are supported include basic ones like manipulating and opening files in directories. Likewise, it can do the same for files. It can do this both synchronously and asynchronously. It has an asynchronous API that have functions that support promises. Also it can show statistics for a file. Almost all the file operations that we can think of can be done with the built in fs module. In this article, we will use the functions in the fs module to open and manipulate directories. We will also experiment with the fs promises API to do equivalent operations that exist in the regular fs module if they exist in the fs promises API. The fs promise API functions return promises so this let us run asynchronous operations sequentially much more easily.

Opendir Promise

The fs module can open, manipulate and close directories. Opening directories with the module is a simple task. To open a directory in a Node.js program, we can use the opendir function to do it. It takes a path object as an argument. The path can be a string, Buffer or an URL object. To open a directory, we can write the following code with the fs promise API, which is available in Node.js 12 LTS or later:

const fs = require("fs");

(async (path) => {  
  const dir = await fs.promises.opendir(path);  
  for await (const dirent of dir) {  
    console.log(dirent.name);  
  }  
})('./files');

The code above will list all the files and folders that are in the given folder. If you run the code above, you should get something like the following:

.keep  
file.txt  
file2.txt

Readdir

If you’re using a Node.js version that’s before 12, then we have to use the older fs API, which doesn’t have functions that return promises. We have to use the readdir function instead to list the contents of a directory. The readdir function takes 2 arguments. The first is the path object, which can be a string, an URL object or a Buffer, and the second argument is a callback function that’s called when the result is produced. The first parameter of the callback is the error object which is defined when there’s an error, and the second is an array with the actual results. For example, if we want to view the content of the ./files folder, we can write:

const fs = require("fs");

fs.readdir("./files", (err, items) => {  
  for (const dirent of items) {  
    console.log(dirent);  
  }  
});

The code above will get us similar output if we run it. It will open the directory path asynchronously, and then list the contents when it’s ready. If we have the same content as the directory above, we will see the following:

.keep  
file.txt  
file2.txt

There’s also a synchronous version of the readdir function called the readdirSync . It takes one argument, which is the path object, which can be a string, an URL object or a Buffer. It returns an array with the content of the given directory path. We can do the same thing by writing the following code:

const fs = require("fs");

const items = fs.readdirSync("./files");  
for (const dirent of items) {  
  console.log(dirent);  
}

If we run the code above, we get the same output as we did before if we have the same contents in the given directory as before.

The opendir function and its synchronous version opendirSync function let us get the directory resource handle of the folder with the given path object. The path object can be a string, an URL object or a buffer. Then asynchronous version, opendir , takes the path object as the first argument and callback function which takes the error object as the first parameter and the directory resource handle object as the second parameter. It’s called when the directory is opened. With the directory handle

Whenever you’re done with opening the directory with the opendir, opendirSync, and manipulating it, we should close it to clean up any reference of it from memory to prevent memory leaks. To do this, Node.js has a close function, which takes a callback function with an error object parameter to get the error when there’s one. It will close the open directory’s resource handle asynchronously. There’s also a synchronous version called the closeSync function that takes no arguments and closes the directory’s resource handle like the close function did, but it holds up the Node.js program’s process until it’s finished. This doesn’t apply to the fs promise API version of opendir since it closes it automatically. For example, if we want to open a directory asynchronously, we can write:

fs.opendir("./files", (err, dir) => {  
  console.log(dir);  
  dir.read((err, dirent) => {  
    console.log(dirent);  
    dir.close();  
  });  
});

The code above will open the directory with the opendir function, which takes a path object and a callback that takes an error object and the directory handle as parameters. Then we call the read function to open the directory and get its description and the name..

Then we call close on the directory resource handle object in the callback passed into the read function which closes the resource and frees up the memory. If we run it, we will get output the looks something like the following:

Dir {  
  [Symbol(kDirHandle)]: DirHandle {},  
  [Symbol(kDirPath)]: './files',  
  [Symbol(kDirClosed)]: false,  
  [Symbol(kDirOptions)]: { encoding: 'utf8' },  
  [Symbol(kDirReadPromisified)]: [Function: bound read],  
  [Symbol(kDirClosePromisified)]: [Function: bound close]  
}  
Dirent { name: '\u0010S.u\u0001', [Symbol(type)]: 777167552 }

OpendirSync

A synchronous version of the opendir function, which is the opendirSync function, can be used with the readSync function to get the directory’s metadata. For example, we can write:

const fs = require("fs");const dir = fs.opendirSync("./files");  
const dirData = dir.readSync();  
console.log(dirData)  
dir.closeSync();

The dirData variable will have something like the following output:

Dirent { name: '.keep', \[Symbol(type)\]: 1 }

That is the data of the content of the directory. To get the path of the opened directory, we can use the path function, which is a function that’s part of the directory object. For example, we can write the following to get the path of the currently opened directory:

const fs = require("fs");fs.opendir("./files", (err, dir) => {  
  console.log(dir.path);  
});

If we run the code above, the we will get output that’s similar to the following:

./files

The output row is the output of the first console.log statement, which is the path of the currently open directory. This isn’t very useful here, but it can be used when the directory resource handle is from an indefinite origin, like when the directory resource handle object is passed in as an argument of a function.

To read the contents of the directory with a directory resource handle, we can use the read function, which is part of the directory resource handle object.

It takes no arguments and a promise is returned that’ll be resolved with the fs.Dirent object, which is an iterable object that has the names of the items in a directory.

The read function without any arguments returns a promise that resolves with the same Dirent object as the example above. For example, we can use the read like in the following code with the asynchronous opendir function:

const fs = require("fs");fs.opendir("./files", (err, dir) => {  
  dir.read((err, dirent) => {  
    console.log(dirent);  
    dir.close();  
  });  
});

If we run the code, we get something like the following output:

Dirent { name: '.keep', \[Symbol(type)\]: 1 }

The output above has the Dirent object, which has the name of the first file in the directory.

<img class="s t u gw ai" src="https://miro.medium.com/max/5854/0*M3wUa7pqdaGu7xMY" width="2927" height="3903" srcSet="https://miro.medium.com/max/552/0*M3wUa7pqdaGu7xMY 276w, https://miro.medium.com/max/1104/0*M3wUa7pqdaGu7xMY 552w, https://miro.medium.com/max/1280/0*M3wUa7pqdaGu7xMY 640w, https://miro.medium.com/max/1400/0*M3wUa7pqdaGu7xMY 700w" sizes="700px" role="presentation"/>

Photo by Omid Kashmari on Unsplash

Listing Multiple Pieces of Directory Content with Opendir

We can read more than one file in the directory, by nested the read function in the callback of the outer read function. For example, we can write:

const fs = require("fs");fs.opendir("./files", (err, dir) => {  
  console.log(dir.path);  
  dir.read((err, dirent) => {  
    console.log(dirent);  
    dir.read((err, dirent) => {  
      console.log(dirent);  
      dir.close();  
    });  
  });  
});

Then we get output with something like the following text:

./files  
Dirent { name: '.keep', \[Symbol(type)\]: 1 }  
Dirent { name: 'file.txt', \[Symbol(type)\]: 1 }

The first row of the output contains the folder path, which we got with dir.path . The next 2 rows are the first 2 items of the directory. Every time we call dir.read , we get the next item of the directory.

To read the contents of the directory with a directory resource handle, we can use the read function, which is part of the directory resource handle object. It takes no arguments and a promise is returned that’ll be resolved with the fs.Dirent object, which is an iterable object that has the names of the items in a directory. The read function without any arguments returns a promise that resolves with the same dirent object as the example above. For example, we can use the read like in the following code with the asynchronous opendir function:

const fs = require("fs");fs.opendir("./files", async (err, dir) => {  
  console.log(dir.path);  
  const item = await dir.read();  
  console.log(item)  
  dir.close();  
});

If we run the code above, we get something like the following:

./files  
Dirent { name: '.keep', \[Symbol(type)\]: 1 }

If we want to get the first 2 items in the directory, with the promise version of dir.read , we can write the following:

const fs = require("fs");fs.opendir("./files", async (err, dir) => {  
  console.log(dir.path);  
  const item1 = await dir.read();  
  console.log(item1)  
  const item2 = await dir.read();  
  console.log(item2)  
  dir.close();  
});

We should get the same output as what we had above assuming we got the same folder contents as before. As we can see, it’s not practical to use those methods to get more items inside the directory those 2 examples as have above of the read function. If we want to get more than 2 items in the directory with the read function, we can write the following with the promise version of the read function, which takes no arguments:

const fs = require("fs");fs.opendir("./files", async (err, dir) => {  
  while ((item = await dir.read())) {  
    console.log(item);  
  }  
  dir.close();  
});

If we run the code above, we get something like the following output:

Dirent { name: '.keep', \[Symbol(type)\]: 1 }  
Dirent { name: 'file.txt', \[Symbol(type)\]: 1 }  
Dirent { name: 'file2.txt', \[Symbol(type)\]: 1 }  
Dirent { name: 'folder1', \[Symbol(type)\]: 2 }  
Dirent { name: 'folder2', \[Symbol(type)\]: 2 }

Note that we can use the await keyword inside a while loop to repeatedly run the dir.read() function call until null or undefined is returned. In the code above, we looped through all the items in the directory and logged the content of the Dirent object.

To check if a Dirent object is a directory, we can use the isDirectory() function. Likewise, we can check if an Dirent is a file with the isFile() function. For example, we can write the following to check if a Dirent is a file or a folder for the contents of a given directory with the following code:

const fs = require("fs");

fs.opendir("./files", async (err, dir) => {  
  while ((item = await dir.read())) {  
    console.log(item);  
    console.log(`Is directory: ${item.isDirectory()}`);  
    console.log(`Is file: ${item.isFile()}\n`);  
  }  
  dir.close();  
});

When we run the code above, we get something like the following output:

Dirent { name: '.keep', [Symbol(type)]: 1 }  
Is directory: false  
Is file: trueDirent { name: 'file.txt', [Symbol(type)]: 1 }  
Is directory: false  
Is file: trueDirent { name: 'file2.txt', [Symbol(type)]: 1 }  
Is directory: false  
Is file: trueDirent { name: 'folder1', [Symbol(type)]: 2 }  
Is directory: true  
Is file: falseDirent { name: 'folder2', [Symbol(type)]: 2 }  
Is directory: true  
Is file: false

As we can see, the isDirectory and isFile functions can identify if an item in a directory is another directory and it can also identify whether it’s a file or not.

With the Node.js’ built in fs module, we can open directories and list its contents easily. It has many functions that can help with opening directories and files inside it. It can do this both synchronously and asynchronously. It has an asynchronous API that have functions that support promises.

Also, it can show statistics for a file. Almost all the file operations that we can think of can be done with the built-in fs module. In this article, we used the functions in the fs module to open directories and list directory contents.

We also used some functions in the fs module that returns promises. The fs functions that return promises let us run asynchronous operations sequentially much more easily.