File Handling in Python

Just to refresh our memory, so far, we have seen all basics of python including variables, data types, functions, conditional statements and loops. You can check out them all on my Python 101 series page.

In every software development, we have to do some kind of operation on the file. We can do n number of operations using python. We can:

1. List of all Files
2. Move & Rename File
3. Delete File
4. Read File
5. Write File
6. Append File

I am sure we can do a lot more that I might have forgotten to mention. But, these are the basic operation that we can do using Python. So, let’s get started!!!!

Just to save time and to create a better understanding, I have merged some topics. Don’t worry, you will know what I am talking about. 😁

List & Rename files

In this section, I will show you what we can do with the os module of python. If you are a geek like me and want to rename files according to the same conventions then you can do that with python.

So, to do this, what do we need? 🤔

First, we need a list of all files. After that, we need to pick each file and rename it.

So, how do we get a list of all files?

As far as I know, there is no direct method in the os module that returns only file names. So, once we get a list of all files and directories, we need to take out the directory or we need to keep just files. To do that we can use the os.path.isfile() method of the os module. This method will return true if the given argument is a file. Same way, os.path.isdir() will return true if the given argument is a directory.

import os

# returns name of all files & directory exist in current location
files_dir = os.listdir('./blogs')
print(files_dir)

only_files = []
for i in files_dir:
    if os.path.isfile('./blogs/'+i):
        only_files.append(i)

only_dir = []
for i in files_dir:
    if os.path.isdir('./blogs/'+i):
        only_dir.append(i)

print('-'*15)
print(only_files) # prints all files
print('-'*15)
print(only_dir) # prints all directories

"""
OUTPUT:
['1.txt', '2.txt', '3.txt', '4.txt', '5.txt', '6.txt', '7.txt', '8.txt', 'Test Directory 1', 'Test Directory 2']
---------------
['1.txt', '2.txt', '3.txt', '4.txt', '5.txt', '6.txt', '7.txt', '8.txt']
---------------
['Test Directory 1', 'Test Directory 2']
"""

So, we got a list of all files.

Now, how do we rename all these files? 🤔

To rename the files, we can use the rename() method and apply it to each file. Let’s assume, we want to rename files with odd and even prefixes based on the file index.

# only_files is a list of all files in the blogs directory

for index, file_name in enumerate(only_files):
    if index % 2 == 0:
        os.rename('./blogs/'+file_name, './blogs/'+'Even-'+file_name)
    else:
        os.rename('./blogs/'+file_name, './blogs/'+'Odd-'+file_name)

Here, enumerate() method returns the counter (starting from 0) and value from iterator (only_files). So, we are checking if the index/counter is even then the code will add the prefix ‘Even-’ to the file name otherwise it will add ‘Odd-’. Find out more on enumerate() on Programiz.

You might have guessed the syntax of the os.rename() method but just in case:
Syntax: os.rename(source, destination) or os.rename(current_name, new_name)

One thing to remember here is that we need to give a whole or proper relative path to use this method. In our case, I have given ‘./blog’ and the ‘/’ to go inside the blogs directory. So the path of the file will become ‘./blogs/1.txt’.

Move & Delete Files

To move a file, we can use either the os or shutil module. In this blog, I will show you how to move a file using the rename() method of the os module. The syntax of rename() is the same, but the only catch is, as the second argument we have to use the destination path with a file name. It might be confusing, so let me show you.

import os

os.rename('./blogs/1.txt', './blogs/Test Directory 1/1.txt')

os.rename('./blogs/2.txt', './blogs/Test Directory 2/1.txt')

So, in the first rename method, we are taking 1.txt from the blogs directory and moving it to Test Directory 1 which is a subdirectory of blogs. In the second scenario, we are taking 2.txt and moving it to the Test Directory 1 directory with the name 1.txt. Yes, we just moved and renamed the file at the same time. If you know a little bit about the Linux commands then you might have observed that os.rename() is similar to the mv Linux command.

Now, how do we delete these files? 🤔

No worries! OS module got your back there as well. By using the remove() method we can remove the file. For example, if we want to remove the 3.txt file from the blogs directory. To do this, we can write the following code:

import os

os.remove('./blogs/3.txt')

Read, Write & Append Operations on File

To do any kind of operation we have to follow 3 basic steps:

1. Open File
2. Do Operation
3. Close File

We can follow these steps using two patterns. It is very hard to explain these patterns in words so, I will explain you using code.

# Open the file
file = open('./blogs/1.txt', 'r', encoding="UTF-8")

# Do operation
data = file.read()
print(data)

# Close the file
file.close()

"""
OUTPUT:

Python 101 Series
"""

So, in this first pattern, we need to open a file, do the operation and close the file manually. Here, open() is provided by Python.
Syntax: open(file, access_mode, encoding)
Here, the file represents the location or a location-like object of a file. The access_mode represents in which mode we want to open the file. We can do more than one task at a time which I will list out later. The encoding represents the encoding format in which we want to operate the file but it is optional. In most cases, we use UTF-8.

with open('./blogs/1.txt', 'r', encoding="UTF-8") as file:
    data = file.read()
    print(data)

In this second pattern, we are using the context manager pattern. If I have to say in short, I would say we use this context manager pattern to use our resources mindfully. The issue with the first approach is that sometimes we might forget to close the file. In this case, it won't affect much but when we handle thousands and millions of files it will affect our system. Context manager pattern will free up the resources as soon as we are done with the more efficient task. If you want to know more about it, you can check out this geeksforgeeks blog.

So, on the 1st line, we open a file using the with open() syntax and assign it to the file variable. It is the same as file = open(). But the main benefit of using this syntax is, python will clear this resource automatically after line 3. As we are not in the context of with open() all resources will be freed related to the file variable after line number 3. In industries, while handling files, most of the time you will see context manager patterns.

As we read the file, we can write and append data to the file in a similar manner. So, to write data into the file we can use the following code snippet:

with open('./blogs/temp.txt', 'w', encoding="UTF-8") as file:
    data = "Python 101 Series"
    file.write(data)

In the case of the write operation, if a file does not exist, it will create a new file and write data into it. And if a file is already with the same name, it will override it with the new data. This will only create just a new file, this open() method will not create a directory for use. So, if I write open("./blogs/temp/temp.txt", "w"), it will throw an error as there is no directory named temp exist.

Now, to solve the overriding issue of writing, we can use the append method. The append method will append the new data at the end of the file we opened.

with open('./blogs/temp.txt', 'a') as file:
    data = "\n New Python 101 Series"
    file.write(data)

So, from the above code, we can say that python will check if the file named temp.txt exists then it will open and append \n New Python 101 Series at the end of it. Otherwise, it will create a new file and write the data. You can see the output in the image below.

File Access Modes

In the table below, I have shown basic access modes. But some modes let you operate on binary data and also in binary format. You can check out the full list on this blog post.

Mode	Explanation
r	read operation
w	write operation
a	append operation
r+	To read and write data into the file. The previous data in the file will not be deleted.
w+	To write and read data. It will override existing data.
a+	To append and read data from the file. It won’t override existing data.

Conclusion

Finally! I hope you learned something after 8 minutes of reading😁.

In this blog, we understood how to handle files in python with some basic programs. I know it is a lot to take in at a time. But, you don't need to remember everything that I mentioned here.

---

That was it. Thank you for reading.

I know it is a lot, but, I hope, you were able to absorb some knowledge. Let me know if you need any help or want to discuss something. Reach out to me on Twitter or LinkedIn. Make sure to share any thoughts, questions, or concerns. I would love to see them. If you want more amazing content subscribe to the Newsletter 👇🏼

You can subscribe to my Newsletter for more amazing resources and articles.

Till the next time 👋