Category: Python Answers

Sometimes, we want to use os.walk() to recursively traverse directories in Python.

In this article, we’ll look at how to use os.walk() to recursively traverse directories in Python.

How to use os.walk() to recursively traverse directories in Python?

To use os.walk() to recursively traverse directories in Python, we call os.walk with the root directory.

For instance, we write

import os

for root, dirs, files in os.walk("."):
    path = root.split(os.sep)
    print(os.path.basename(root))
    for file in files:
        print(file)

to call os.walk with the root directory that we want to traverse.

Then we loop through the dirs directories and files files with a for loop.

We get the split path with the string split method called with the path separator os.sep.

And we print the base folder name of root with basename.

Then we loop through the files in the directory with another for loop.

Conclusion

To use os.walk() to recursively traverse directories in Python, we call os.walk with the root directory.

Sometimes, we want to compute list difference with Python.

In this article, we’ll look at how to compute list difference with Python.

How to compute list difference with Python?

To compute list difference with Python, we can create a function that uses the filter function.

For instance, we write

diff = lambda l1, l2: filter(lambda x: x not in l2, l1)
d1 = diff(A, B)
d2 = diff(B, A)

to create the diff function that calls filter with a function that checks if x in list l1 isn’t in l2.

Then we call diff and lists A and B in different orders to get the difference A and B and vice versa.

Conclusion

To compute list difference with Python, we can create a function that uses the filter function.

Sometimes, we want to run scp in Python.

In this article, we’ll look at how to run scp in Python.

How to run scp in Python?

To run scp in Python, we can use paramiko.

To install it, we run

pip install paramiko

Then we use it by writing

import paramiko
from scp import SCPClient

def create_ssh_client(server, port, user, password):
    client = paramiko.SSHClient()
    client.load_system_host_keys()
    client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    client.connect(server, port, user, password)
    return client

ssh = create_ssh_client(server, port, user, password)
scp = SCPClient(ssh.get_transport())

to create the create_ssh_client function.

In it, we create a SSHClient object.

Then we load host keys on the system with load_system_host_keys.

We call set_missing_host_key_policy to set the policy to use when connecting to servers without a host key.

Then we call connect to connect to the server with the port and credentials.

Next, we call create_ssh_client create an SSH client.

And then we create a SCPClient with the object returned by ssh.get_transport() to create the SCP client.

Conclusion

To run scp in Python, we can use paramiko.

Sometimes, we want to efficiently parse fixed width files with Python.

In this article, we’ll look at how to efficiently parse fixed width files with Python.

How to efficiently parse fixed width files with Python?

To efficiently parse fixed width files with Python, we can use the Pandas’ read_fwf method.

For instance, we write

import pandas as pd

path = 'filename.txt'

col_specification = [(0, 20), (21, 30), (31, 50), (51, 100)]
data = pd.read_fwf(path, colspecs=col_specification)

to define the col_specification list with the column specifications for filename.txt.

Then we call read.fwf with the path and the colspecs argument set to col_specification to parse the fixed with file into a data frame.

Conclusion

To efficiently parse fixed width files with Python, we can use the Pandas’ read_fwf method.

Sometimes, we want to time a code segment for testing performance with Python timeit.

In this article, we’ll look at how to time a code segment for testing performance with Python timeit.

How to time a code segment for testing performance with Python timeit?

To time a code segment for testing performance with Python timeit, we can use the time module.

For instance, we write

import time

t0 = time.time()
foo()
t1 = time.time()

total = t1 - t0

to call time.time to get the current time.

We call foo in between the 2 time calls to get the time at the start and the end of the execution of foo.

And then we get the time elapsed when foo is run with

total = t1 - t0

Conclusion

To time a code segment for testing performance with Python timeit, we can use the time module.