Category: Python Answers

Sometimes, we want to get the return value of a function passed to Python multiprocessing.Process.

In this article, we’ll look at how to get the return value of a function passed to Python multiprocessing.Process.

How to get the return value of a function passed to Python multiprocessing.Process?

To get the return value of a function passed to Python multiprocessing.Process, we can use the manager.dict method to create a shared variable.

For instance, we write

import multiprocessing


def worker(procnum, return_dict):
    print(str(procnum) + " represent!")
    return_dict[procnum] = procnum


if __name__ == "__main__":
    manager = multiprocessing.Manager()
    return_dict = manager.dict()
    jobs = []
    for i in range(5):
        p = multiprocessing.Process(target=worker, args=(i, return_dict))
        jobs.append(p)
        p.start()

    for proc in jobs:
        proc.join()
    print(return_dict.values())

to create a Process object with the target set to the worker function that runs for each process.

Then we set the args to a tuple with the arguments for worker.

Then we call jobs.append to append the process p to jobs.

And then we call p.start to start the process.

Next, we have a for loop that calls join to block the main process from running until the proc process terminates.

And then we get the returned values of worker with return_dict.values().

Conclusion

To get the return value of a function passed to Python multiprocessing.Process, we can use the manager.dict method to create a shared variable.

Sometimes, we want to make asynchronous requests with Python requests.

In this article, we’ll look at how to make asynchronous requests with Python requests.

How to make asynchronous requests with Python requests?

To make asynchronous requests with Python requests, we can use the grequests module.

To install it, we run

pip install grequests

Then we use it by writing

import grequests

urls = [
    'http://www.heroku.com',
    'http://example.org',
    'http://httpbin.org',
    'http://python-requests.org',
    'http://abc.com'
]

rs = (grequests.get(u) for u in urls)

to call grequests.get with the URLs in urls to make async GET requests with each URL.

Conclusion

To make asynchronous requests with Python requests, we can use the grequests module.

Sometimes, we want to get last n lines of a file, similar to tail with Python.

In this article, we’ll look at how to get last n lines of a file, similar to tail with Python.

How to get last n lines of a file, similar to tail with Python?

To get last n lines of a file, similar to tail with Python, we can run tail with Popen.

For instance, we write

import subprocess

def tail(f, n, offset=0):
    proc = subprocess.Popen(['tail', '-n', n + offset, f], stdout=subprocess.PIPE)
    lines = proc.stdout.readlines()
    return lines[:, -offset]

to create the tail function that runs tail with the arguments to return the last n lines from file with path f.

And then we set stdout to subprocess.PIPE to return the output.

Next, we call proc.stdout.readlines to get the output and return it.

Finally, we use lines[:, -offset] to get the list with the last n lines.

Conclusion

To get last n lines of a file, similar to tail with Python, we can run tail with Popen.

Sometimes, we want to get the total memory used by a Python process.

In this article, we’ll look at how to get the total memory used by a Python process.

How to get the total memory used by a Python process?

To get the total memory used by a Python process, we can use the psutil module.

To install it, we run

pip install psutil

Then, we write

import os, psutil

process = psutil.Process(os.getpid())
print(process.memory_info().rss) 

to create a Process object with

process = psutil.Process(os.getpid())

We call os.getpid() to get the process ID of the current script.

And then we call memory_info to get the amount of memory used in bytes from the rss property.

Conclusion

To get the total memory used by a Python process, we can use the psutil module.

Sometimes, we want to find elements by class with Python.

In this article, we’ll look at how to find elements by class with Python.

How to find elements by class with Python?

To find elements by class with Python, we can use the find_all method.

For instance, we write

soup.find_all("a", class_="sister")

to call find_all on a BeautifulSoup instance with the tag of the element and the class of the element to return.

Setting class_ searches for the elements with the given class.

Conclusion

To find elements by class with Python, we can use the find_all method.