To work with stored data, file handling becomes the core knowledge of every professional Python programmer. Right from its earliest release, both reading and writing data to files are built-in Python features. In comparison to other programming languages like C or Java, it is pretty simple and only requires a few lines of code. Furthermore, no extra module has to be loaded to do that properly.
In this article we will be explaining how to read files with Python through examples. Some examples include reading a file line-by-line, as a chunk (a defined number of lines at a time), and reading a file in one go. Also, we will show you a way to read a specific line from the file, only, without searching the entire file.
Basics of Files in Python
The common methods to operate with files are open() to open a file, seek() to set the file's current position at the given offset, and close() to close the file object when you're done using it. The built-in open() function returns a file handle that represents a file object to be used to access the file for reading, writing, or appending.
When opening a file for reading, Python needs to know exactly how the file should be opened with the system. Two access modes are available - reading and reading in binary mode. The respective flags used are 'r', and 'rb', and have to be specified when opening a file with the built-in open() function. The first mode includes the interpretation of special characters like "CR" (carriage return) and "LF" (linefeed) to represent line breaks, whereas the binary mode allows you to read the data in raw mode - where the data is stored as is without further interpretation.
Once you've opened a file, the open() function will return a file object to you. These file objects have methods like read(), readline(), seek()0, seek()1, and seek(). While some file objects (or file-like objects) have more methods than those listed here, these are the most common. Not all file objects need to implement all of the file methods.
Reading a File Line-by-Line
The first example is inspired by the two programming languages - C and C++. It is probably the most intuitive approach - open the file using the open() function, read the file line-by-line using the readline() method, and output the line immediately after reading.
In use here is a seek()5 loop that continuously reads from the file as long as the readline() method keeps returning data. In case the end of file (EOF) is reached the seek()5 loop stops and the file object is closed, freeing up the resources for other programs to use:
As you may have noted, we have explicitly opened and closed the file in this example. Although the Python interpreter closes the opened files automatically at the end of the execution of the Python program, explicitly closing the file via close() is good a programming style, and should not be forgotten.
As an improvement, the convenient iterator protocol was introduced in Python 2.3. This allows you to simplify the seek()9 loop:
In use here is a close()0 loop in combination with the close()1 iterator. The current line is identified with the help of the close()1 iterator, read from the file, and its content is output to close()3. Python covers the opening and closing of the file for you when it falls out of scope. While inefficient, this allows you to not have to deal with file handles anymore.
Unfortunately, the code above is less explicit and relies on Python's internal garbage collection to handle closing the file.
Introduced in Python 2.5, the close()4 command encapsulates the entire process even more, and also handles opening and closing files just once throughout the scoped code block:
The combination of the close()4 statement and the open() command opens the file only once. If successful the close()0 loop is executed, and the content of the line is printed on close()3.
Furthermore, the usage of the close()4 statement has a side effect. Internally, the Python interpreter creates a open()0-open()1-block to encapsulate reading from the file. The following example shows what is essentially happening internally in Python with the close()4 code blocks:
Reading a File as Chunks of Lines
Up to now, we have processed a file line by line. This is rather slow for huge files and can be improved by reading multiple lines at the same time. To achieve that, the open()3 method from the itertools module comes into play. Also, it works as an iterator and returns a chunk of data that consists of open()4 lines. At the end of the file, the result might be shorter, and finally, the call will return an empty list:
Reading a Specific Line from a File
Using the methods shown above we can also perform other useful actions, like reading a specific line from a file. To do this, we make use of a counter and print the appropriate line when we come to it while iterating through the file:
This should be simple to understand, but it's a bit longer than the previous examples. It can be shortened using the linecache module.
The following example shows how to simplify the code using the open()5 method. If the requested line number falls out of the range of valid lines in the file, then the open()5 method returns an empty string instead:
Reading the Entire File at Once
Last but not least we will have a look at a very different case than the previous example - reading an entire file in one go.
Keep in mind that in most cases you should have enough memory to read the entire file, since characters don't take up too much space, but be weary of large files. The following example uses a combination of the close()4 statement, and the read() method. In this case, we'll use read() to load the file content as a data stream:
Check out our hands-on, practical guide to learning Git, with best-practices, industry-accepted standards, and included cheat sheet. Stop Googling Git commands and actually learn it!
Python also offers the 'r'0 method, which is similar to the readline() method from the first example. In contrast to read(), the file content is stored in a list, where each line of the content is an item:
While 'r'0 will read content from the file until it hits EOF, keep in mind that you can also limit the amount of content read by providing the 'r'4 parameter, which is the number of bytes to read.
Conclusion
As usual, there is more than one way to read the contents of a file. In terms of speed, all of them are more or less in the same category. Which solution works best for you depends on your specific use case. We think it is quite helpful to see what is possible and then to choose the solution that suits best.
While Python greatly simplifies the process of reading files, it can still become tricky at times, in which case I'd recommend you take a look at the official Python documentation for more info.