Sometimes, we want to read a big file in Python.
In this article, we’ll look at how to read a big file in Python.
How to read a big file in Python?
To read a big file in Python, we can call the readlines method on the opened file object with the buffer size.
For instance, we write:
BUF_SIZE = 100
bigfile = open('bar.txt', 'r')
tmp_lines = bigfile.readlines(BUF_SIZE)
while tmp_lines:
print(tmp_lines)
tmp_lines = bigfile.readlines(BUF_SIZE)
to call open with the file path of the file and 'r' to open the file with read permission.
Then we call bigfile.readlines with the BUF_SIZE buffer size.
BUF_SIZE is in bytes.
Next, we have a while loop that runs while tmp_lines isn’t None.
In the while loop, we call bigfile_readlines to read the next chunk in the buffer.
Conclusion
To read a big file in Python, we can call the readlines method on the opened file object with the buffer size.
