Author: John Au-Yeung

Sometimes, we want to compute list difference with Python.

In this article, we’ll look at how to compute list difference with Python.

How to compute list difference with Python?

To compute list difference with Python, we can create a function that uses the filter function.

For instance, we write

diff = lambda l1, l2: filter(lambda x: x not in l2, l1)
d1 = diff(A, B)
d2 = diff(B, A)

to create the diff function that calls filter with a function that checks if x in list l1 isn’t in l2.

Then we call diff and lists A and B in different orders to get the difference A and B and vice versa.

Conclusion

To compute list difference with Python, we can create a function that uses the filter function.

Sometimes, we want to run scp in Python.

In this article, we’ll look at how to run scp in Python.

How to run scp in Python?

To run scp in Python, we can use paramiko.

To install it, we run

pip install paramiko

Then we use it by writing

import paramiko
from scp import SCPClient

def create_ssh_client(server, port, user, password):
    client = paramiko.SSHClient()
    client.load_system_host_keys()
    client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    client.connect(server, port, user, password)
    return client

ssh = create_ssh_client(server, port, user, password)
scp = SCPClient(ssh.get_transport())

to create the create_ssh_client function.

In it, we create a SSHClient object.

Then we load host keys on the system with load_system_host_keys.

We call set_missing_host_key_policy to set the policy to use when connecting to servers without a host key.

Then we call connect to connect to the server with the port and credentials.

Next, we call create_ssh_client create an SSH client.

And then we create a SCPClient with the object returned by ssh.get_transport() to create the SCP client.

Conclusion

To run scp in Python, we can use paramiko.

Sometimes, we want to efficiently parse fixed width files with Python.

In this article, we’ll look at how to efficiently parse fixed width files with Python.

How to efficiently parse fixed width files with Python?

To efficiently parse fixed width files with Python, we can use the Pandas’ read_fwf method.

For instance, we write

import pandas as pd

path = 'filename.txt'

col_specification = [(0, 20), (21, 30), (31, 50), (51, 100)]
data = pd.read_fwf(path, colspecs=col_specification)

to define the col_specification list with the column specifications for filename.txt.

Then we call read.fwf with the path and the colspecs argument set to col_specification to parse the fixed with file into a data frame.

Conclusion

To efficiently parse fixed width files with Python, we can use the Pandas’ read_fwf method.

Sometimes, we want to time a code segment for testing performance with Python timeit.

In this article, we’ll look at how to time a code segment for testing performance with Python timeit.

How to time a code segment for testing performance with Python timeit?

To time a code segment for testing performance with Python timeit, we can use the time module.

For instance, we write

import time

t0 = time.time()
foo()
t1 = time.time()

total = t1 - t0

to call time.time to get the current time.

We call foo in between the 2 time calls to get the time at the start and the end of the execution of foo.

And then we get the time elapsed when foo is run with

total = t1 - t0

Conclusion

To time a code segment for testing performance with Python timeit, we can use the time module.

Sometimes, we want to jump to a particular line in a huge text file with Python.

In this article, we’ll look at how to jump to a particular line in a huge text file with Python.

How to jump to a particular line in a huge text file with Python?

To jump to a particular line in a huge text file with Python, we’ve to read the file.

For instance, we write

# ...
line_offset = []
offset = 0
for line in file:
    line_offset.append(offset)
    offset += len(line)
file.seek(0)

# ...

file.seek(line_offset[n])

to loop through the file and append the offset to the line_offset list.

Then we add the line‘s length to the offset.

Next, we rewind back to the start of the file with file.seek called with 0.

And then we call file_seek again to jump to the offset with

file.seek(line_offset[n])

Conclusion

To jump to a particular line in a huge text file with Python, we’ve to read the file.