Logging

By now, you probably have an understanding of what logs are. Now, it is time to take a step further and explore the built-in Python logging module.

The logging module

According to the Official Documentation, the logging module defines functions and classes that implement a flexible event logging system for applications and libraries. In other words, this module provides a flexible framework for outputting log messages.

To use this module, we need to import it and configure a logger. Let's take a look at the example below:

import logging

logging.warning("Your %s was executed successfully, but the %s is wrong!", "script", "output")


# WARNING:root: Your script was executed successfully, but the output is wrong!

By default, the output message has the following format: {LEVEL}:{LOGGER}:{MESSAGE}. This format can be changed by using the format attribute when configuring the logger. To do it, we need to call the basicConfig() method:

import logging

logging.basicConfig(format='%(levelname)s:%(message)s')
logging.info("Your program is running excellent")
logging.warning("Your %s was executed successfully, but the %s is wrong!", "script", "output")


# WARNING: Your script was executed successfully, but the output is wrong!

There are other attributes that you can use to merge data from the log record into the format string, such as asctime and lineno. To get more information and other available attributes, refer to the LogRecord attributes page of the Official Documentation.

You probably noticed that only the warning message was printed. This happens because the default configuration of the logging module is set to the WARNING level, so the info level is not shown in the output. Let's explore what log levels are.

The log levels

It is important to provide a priority scale when logging. A log that notifies the developer that the application is going to crash should be more visible than a log that indicates that there are no problems.

The logging priority scale in Python consists of five different hierarchical log levels. Each log level has a numeric value associated with it. It is much easier to determine if a specific level has to be logged or not.

• DEBUG: 10 — the messages at this level provide detailed insights about the application. It is typically used by developers when they have a bug. If you log a message at the debug level instead of print(), you can avoid removing the print statement when your debugging phase is over;
• INFO: 20 — at this level, the messages confirm that things are business as usual. We can regard the messages at this level as checkpoints of our program;
• WARNING: 30 — the messages at this level indicate that something went sour. The program is still working, but the output could be weird;
• ERROR: 40 — as suggested by the name, the messages at this level suggest that the application is not working correctly. The output will be affected;
• CRITICAL: 50 — the highest level. The program may be unable to continue running and will no longer produce an output.

The logger class

Logger is the main class of the logging module. It can help us interact with the log system and output log messages:

• logging.debug("message") — to log a message at the debug level;
• logging.info("message") — to log a message at the info level;
• logging.warning("message") — to log a message at the warning level;
• logging.error("message") — to log a message at the error level;
• logging.critical("message") — to log a message at the critical level.

As a practical example, we can run the following script:

import logging
logging.basicConfig(format="%(message)s")

logging.critical("It is critical to understand logs!")
logging.error("Running this line will result in an error message!")
logging.warning("You must catch that bug! It is a warning!")
logging.info("My info is that you are here!")
logging.debug("I'm debugging!")

The output to the console will be:

It is critical to understand logs!
Running this line will result in an error message!
You must catch that bug! It is a warning!

In the previous example, only the messages from the WARNING (or higher) level can be output! This happens because the default configuration of the logging module is set to the WARNING level. To output all the messages, we need to set the level attribute to the DEBUG level:

import logging

logging.basicConfig(format='%(message)s', level='DEBUG')

logging.critical("It is critical to understand logs!")
logging.error("Running this line will result in an error message!")
logging.warning("You must catch that bug! It is a warning!")
logging.info("My info is that you are here!")
logging.debug("I'm debugging!")

The output of this piece of code will be as follows:

It is critical to understand logs!
Running this line will result in an error message!
You must catch that bug! It is a warning!
My info is that you are here!
I'm debugging!

If you add filename='' and filemode='' parameters to the basicConfig() method, you will also be able to save the output into a file for later use:

logging.basicConfig(filename='log_file.txt',
                    filemode='a',
                    format='%(message)s',
                    level='DEBUG')

In this case, the messages won't be printed but directly written into the file.

Better logging

If you would like to do more complicated logging tasks, you may need to use a more professional approach to it.

At the advanced level, we use a logger object to track events in the code. We need to set up our logger object in four steps:

import logging

# STEP 1
# create a logger object instance
logger = logging.getLogger()

# STEP 2
# specify the lowest boundary for logging
logger.setLevel(logging.DEBUG)

# STEP 3
# set a destination for your logs or a handler
# here, we choose to print on console (a console handler)
console_handler = logging.StreamHandler()

# STEP 4
# set the logging format for your handler
log_format = '%(asctime)s | %(levelname)s: %(message)s'
console_handler.setFormatter(logging.Formatter(log_format))

# finally, add the handler to the logger
logger.addHandler(console_handler)

# start the logging and show the messages
logger.debug('Here you have some information for debugging.')
logger.info('Everything is OK. Keep going!')
logger.warning("Something strange has happened, but it's not critical.")
logger.error('Something unexpected and critical has happened.')
logger.critical('A critical error! The code cannot run!')

If you run the script, you will get the following output:

2021-09-19 22:55:05,829 | DEBUG: Here you have some information for debugging.
2021-09-19 22:55:05,830 | INFO: Everything is OK. Keep going!
2021-09-19 22:55:05,831 | WARNING: Something strange has happened, but it's not critical.
2021-09-19 22:55:05,831 | ERROR: Something unexpected and critical has happened.
2021-09-19 22:55:05,831 | CRITICAL: A critical error! The code cannot run!

As you can see, we have created a logger object and set its level. Then, we have created the console handler and set its format. We have also added the console handler to the logger object and, finally, we initiated the logging. Now, you can imagine how hard it would be to replicate the same functionality with print() statements.

There are many advanced tools and settings that you can use in your logger. We won't get into detail now. You can learn more about all available tools and settings in the Logging Cookbook.

Conclusion

All developers make errors, and all programs may crash. It is important to identify and resolve the issues quickly. The logging module is one of the most powerful tools at our disposal to easily determine the source of issues in our code.

In this topic, we have introduced you to the main player of the logging module: the logger class. You also have got a sense of what log levels are and how to set them. There is more to discuss on logging, but for now, let's practice what you have learned!