Manipulating files and directories are basic operations for any program. Since Node.js is a server-side platform and can interact with the computer that it’s running on directly, being able to manipulate files is a basic feature.
Fortunately, Node.js has a fs module built into its library. It has many functions that can help with manipulating files and folders.
File and directory operation that are supported include basic ones like manipulating and opening files in directories.
Likewise, it can do the same for files. It can do this both synchronously and asynchronously. It has an asynchronous API that has functions that support promises.
Also, it can show statistics for a file. Almost all the file operations that we can think of can be done with the built-in fs module. In this article, we will create write streams to write a file’s data sequentially and listen to events from a write stream. Since Node.js WriteStreams are descendants of the Writable object, we will also listen to events to it.
Streams are collections of data that may not be available all at once and don’t have to fit in memory. This makes stream handy for processing large amounts of data.
It’s handy for files because files can be big and streams can let us get a small amount of data at one time. In the fs module, there are 2 kinds of streams. There’s the ReadStream and the WriteStream. We get the data into a WriteStream via a ReadStream.
WriteStream
A WriteStream is for used for writing data to a file. We can get the input into our WriteStream via the ReadStream’s pipe function. The data from the ReadStream is sent to the write stream in small chunks so that the host computer won’t run out of memory.
To define a WriteStream, we can use the fs.createWriteStream function. The function takes 2 arguments.
The first argument is the path of the file. The path can be in the form of a string, a Buffer object, or an URL object. The path can be in the form of a string, a Buffer object, or an URL object.
The second argument is an object that can have a variety of options as properties. The flag option is the file system flag for setting the mode for opening the file. The default flag is w. The list of flags are below:
'a'– Opens a file for appending, which means adding data to the existing file. The file is created if it does not exist.'ax'– Like'a'but exception is thrown if the path exists.'a+'– Open file for reading and appending. The file is created if it doesn’t exist.'ax+'– Like'a+'but exception is thrown if the path exists.'as'– Opens a file for appending in synchronous mode. The file is created if it does not exist.'as+'– Opens a file for reading and appending in synchronous mode. The file is created if it does not exist.'r'– Opens a file for reading. An exception is thrown if the file doesn’t exist.'r+'– Opens a file for reading and writing. An exception is thrown if the file doesn’t exist.'rs+'– Opens a file for reading and writing in synchronous mode.'w'– Opens a file for writing. The file is created (if it does not exist) or overwritten (if it exists).'wx'– Like'w'but fails if the path exists.'w+'– Opens a file for reading and writing. The file is created (if it does not exist) or overwritten (if it exists).'wx+'– Like'w+'but exception is thrown if the path exists.
The encoding option is a string that sets the character encoding in the form of the string. The default value is null .
The fd option is the integer file descriptor which can be obtained with the open function and its variants. If the fd option is set, then the path argument will be ignored. The default value is null .
The mode option is the file permission and sticky bits of the file, which is an octal number that is the same as Unix or Linux file permissions. It’s only set if the file is created.
The default value is 0o666. The autoClose option specifies that the file descriptor will be closed automatically. The default value is true . If it’s false , then the file descriptor won’t be closed even if there’s an error. It’s completely up to us to close it autoClose is set to false to make sure there’s no file descriptor leak.
Otherwise, the file descriptor will be closed automatically if there’s an error or end event emitted. The emitClose option will emit the close event when the write stream ends.
The default value is false . The start and end options specifies the beginning and end parts of the file to write.
Everything in between will be written in addition to the start and end . start and end are numbers that are the starting and ending bytes of the file to write.
To create a WriteStream, we can use the createWriteStream like in the following code:
const fs = require("fs");
const sourceFile = "./files/file.txt";
const destFile = "./files/newFile.txt";
const readStream = fs.createReadStream(sourceFile);
const writeStream = fs.createWriteStream(destFile, {
flags: "w",
encoding: "utf8",
mode: 0o666,
autoClose: true,
emitClose: true,
start: 0
});
readStream.pipe(writeStream);
writeStream.on("open", () => {
console.log("Stream opened");
});
writeStream.on("ready", () => {
console.log("Stream ready");
});
writeStream.on("pipe", src => {
console.log(src);
});
writeStream.on("unpipe", src => {
console.log(src);
});
writeStream.on("finish", () => {
console.log("Stream finished");
});
When we run the code above, we should get something like the following outputted to the screen:
Stream opened
Stream ready
ReadStream {
_readableState: ReadableState {
objectMode: false,
highWaterMark: 65536,
buffer: BufferList { head: null, tail: null, length: 0 },
length: 0,
pipes: null,
pipesCount: 0,
flowing: false,
ended: true,
endEmitted: true,
reading: false,
sync: false,
needReadable: false,
emittedReadable: false,
readableListening: false,
resumeScheduled: false,
paused: false,
emitClose: false,
autoDestroy: false,
destroyed: true,
defaultEncoding: 'utf8',
awaitDrain: 0,
readingMore: false,
decoder: null,
encoding: null
},
readable: false,
_events: [Object: null prototype] { end: [Function] },
_eventsCount: 1,
_maxListeners: undefined,
path: './files/file.txt',
fd: null,
flags: 'r',
mode: 438,
start: undefined,
end: Infinity,
autoClose: true,
pos: undefined,
bytesRead: 16,
closed: false
}
Stream finished
We have the Readable object sent when the pipe event is emitted. Also, we should get content that’s in file.txt in newFile.txt .
WriteStream Events
With a WriteStream, we can listen to the following events. There’s a close event which is emitted when the close event is emitted after the file is written. The open event is emitted when the stream is opened. The file descriptor number fd will be passed with the event when it’s emitted. The ready event is emitted when the WriteStream is ready to be used. It’s fired immediately after the open event is fired.
WriteStreams extend the Writable object, which emits events of its own. The close event is emitted when the stream any of its underlying resources like file descriptors have been closed.
This event indicates that there’re no more events to be emitted from the stream and nothing else will be run. If the emitClose option is set to true when creating the WriteStream then the close event will be emitted.
If a call to the stream.write function returns false , then the drain event will be emitted when writing data to the stream is resumed. The error event is emitted when an error occurred when piping data.
The listener callback function has an error object parameter to get the error information. The stream isn’t closed on error events unless the autoDestroy option is set to true when creating the stream.
After the error event is emitted, then no events other than close should be emitted. The finish event is emitted after the stream.end function is called and all data is flush to the underlying system like a file.
The pipe event is emitted when stream.pipe function is called with a readable stream passed in as an argument . The file specified when creating the ReadStream must exist before piping from it.
The unpipe event is emitted when the stream.unipipe function is called on the readable stream. It’s also emitted when the WriteStream emits an error event when a ReadStreanm is piped to it.
A WriteStream has a few properties. The bytesWritten property indicates the number of bytes being written by the WriteStream so far and doesn’t include data that’s still queued for writing.
The path property is the path of the file the WriteStream is writing to, which is the same as the first argument of the fs.createWriteStream function.
It will be of the same type as whatever we passed into the first argument of fs.createWriteStream . The pending property is a boolean that is true when the underlying file hasn’t been opened, that is, before the ready event is emitted.
By using the fs.createWriteStream function, we created read streams to read a file’s data sequentially and listen to events from a read stream. Since Node.js WriteStreams are descendants of the Writable object, we will also listen to events to it.
We have lots of control of how the write stream is created. We can set the path or file descriptor of the file. Also, we can set the mode of the file to be written and the permission and sticky bit of the file being read.
Also, we can choose to close the streams automatically or not or emit close event automatically.
We get data to a write stream by passing the WriteStream object as an argument of the ReadStream’s pipe function.
