Sometimes, we want to log while using multiprocessing in Python.
In this article, we’ll look at how to log while using multiprocessing in Python.
How to log while using multiprocessing in Python?
To log while using multiprocessing in Python, we can create a function to create a logger.
For instance, we write
def create_logger():
import multiprocessing, logging
logger = multiprocessing.get_logger()
logger.setLevel(logging.INFO)
formatter = logging.Formatter(\
'[%(asctime)s| %(levelname)s| %(processName)s] %(message)s')
handler = logging.FileHandler('logs/your_file_name.log')
handler.setFormatter(formatter)
if not len(logger.handlers):
logger.addHandler(handler)
return logger
if __name__ == '__main__':
from multiprocessing import Pool
logger = create_logger()
logger.info('Starting pooling')
p = Pool()
to create the create_logger function.
In it, we call multiprocessing.get_logger to create a logger object.
And then we get the handler for the logger with FileHandler to save log messages to logs/your_file_name.log.
Then we call setFormatter to use the formatter we get from Formatter.
Next, we call logger.addHandler with handler to use the FileHandler.
Next, we call create_logger to create the logger.
And then we call logger.info to log a message.
Conclusion
To log while using multiprocessing in Python, we can create a function to create a logger.
