Stop Printing, Start Logging: A Practical Deep Dive into Python's logging Module

Almost every Python program starts the same way: you sprinkle a few print() calls around to see what's happening, and it works well enough. But the moment your code grows into a real application — a web service, a scheduled job, a CLI tool that other people run — print() starts to hurt. You can't easily turn messages off in production. You can't tell an important warning apart from routine chatter. You can't send errors to a file while keeping debug output on screen. And you certainly can't capture a full traceback without wrapping everything in clumsy boilerplate.

Python's built-in logging module solves all of this, and it ships in the standard library — no installation required. It has a reputation for being fiddly, but most of that reputation comes from people copying snippets without understanding the three pieces that matter: loggers, handlers, and formatters. Once those click, logging becomes one of the most useful tools in your kit. This guide walks through it from the ground up, with runnable examples and the pitfalls that trip people up.

The five levels, and why they matter

Logging is built around severity levels. Each message you emit is tagged with one, and you can later decide which levels are worth showing. The five built-in levels, from least to most severe, are DEBUG, INFO, WARNING, ERROR, and CRITICAL.

import logging

logging.debug("Detailed diagnostic information")
logging.info("Confirmation that things are working")
logging.warning("Something unexpected, but we carry on")
logging.error("A serious problem; an operation failed")
logging.critical("A fatal error; the program may stop")

Run this and you'll only see the last three messages, printed as WARNING:root:Something unexpected, but we carry on. That surprises beginners, but it's intentional: the default level is WARNING, so DEBUG and INFO are filtered out until you ask for them. The level is a dial — set it to DEBUG while developing, raise it to INFO or WARNING in production, and you change how chatty your app is without touching a single log statement.

Use a named logger, not the root logger

The module-level functions like logging.info() all write to the single root logger. That's fine for a throwaway script, but in real code you want your own named logger. The idiom is always the same:

import logging

logger = logging.getLogger(__name__)

def divide(a, b):
    logger.debug("divide(%s, %s) called", a, b)
    return a / b

Using __name__ means the logger is named after the module it lives in — myapp.services.billing, for example. This gives you two things for free: every message can show where it came from, and you can later tune logging per-package (turn myapp.services up to DEBUG while leaving everything else at WARNING). Calling getLogger() with the same name anywhere in your program returns the same object, so there's no need to pass loggers around.

Handlers and formatters: where logs go and how they look

A logger decides whether a message should be processed. A handler decides where it goes — the console, a file, the system journal, the network. A formatter decides what each line looks like. A single logger can fan out to several handlers at once, each with its own level and format.

Here's the pattern that covers most needs: everything at DEBUG and above goes to a file, but only INFO and above clutters the console.

import logging

logger = logging.getLogger("myapp")
logger.setLevel(logging.DEBUG)          # the logger passes everything through

console = logging.StreamHandler()        # defaults to stderr
console.setLevel(logging.INFO)           # console is quieter

file_handler = logging.FileHandler("app.log")
file_handler.setLevel(logging.DEBUG)     # the file keeps everything

formatter = logging.Formatter(
    "%(asctime)s | %(levelname)-8s | %(name)s | %(message)s",
    datefmt="%Y-%m-%d %H:%M:%S",
)
console.setFormatter(formatter)
file_handler.setFormatter(formatter)

logger.addHandler(console)
logger.addHandler(file_handler)

logger.debug("Written to the file only")
logger.info("Written to both the console and the file")

This is the part people miss: a record has to clear two gates — the logger's level and the handler's level. The logger is set to DEBUG so nothing is dropped early; then each handler applies its own threshold. That two-stage filtering is exactly what lets the same debug line land in your log file while staying off the screen.

Useful format fields

Formatters use %-style placeholders drawn from the log record. The ones you'll reach for most are %(asctime)s (timestamp), %(levelname)s (DEBUG/INFO/…), %(name)s (the logger name), %(message)s (your text), and for debugging, %(filename)s and %(lineno)d to pin down the exact source line.

Configure once, at the entry point

For small programs, logging.basicConfig() sets up a sensible console handler in one call. Call it once, early, before you emit anything:

import logging

logging.basicConfig(
    level=logging.DEBUG,
    format="%(asctime)s | %(levelname)-8s | %(name)s | %(message)s",
    datefmt="%Y-%m-%d %H:%M:%S",
)

logging.getLogger(__name__).info("Now INFO and DEBUG are visible")

One catch worth memorizing: basicConfig() does nothing if the root logger already has handlers. So if it seems to be ignored, something configured logging before you. (Python 3.8+ lets you pass force=True to wipe existing handlers and start clean.)

For anything larger, configure logging declaratively with dictConfig. It keeps all your handlers, formatters, and levels in one readable structure that's easy to load from a config file:

import logging.config

LOGGING = {
    "version": 1,
    "disable_existing_loggers": False,
    "formatters": {
        "standard": {
            "format": "%(asctime)s | %(levelname)-8s | %(name)s | %(message)s",
        },
    },
    "handlers": {
        "console": {
            "class": "logging.StreamHandler",
            "level": "INFO",
            "formatter": "standard",
        },
        "file": {
            "class": "logging.handlers.RotatingFileHandler",
            "level": "DEBUG",
            "formatter": "standard",
            "filename": "app.log",
            "maxBytes": 1_000_000,
            "backupCount": 3,
        },
    },
    "root": {
        "level": "DEBUG",
        "handlers": ["console", "file"],
    },
}

logging.config.dictConfig(LOGGING)
logging.getLogger(__name__).info("Configured from a dictionary")

Log exceptions with the full traceback

When you catch an exception, you almost always want the traceback in your logs, not just a one-line message. Inside an except block, logger.exception() does exactly that — it logs at ERROR level and automatically attaches the traceback:

import logging

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

try:
    result = 10 / 0
except ZeroDivisionError:
    logger.exception("Failed to compute result")

The output includes your message followed by the complete Traceback (most recent call last): … ZeroDivisionError: division by zero. logger.exception() only makes sense inside an exception handler; elsewhere, pass exc_info=True to any logging call (for example logger.error("...", exc_info=True)) to get the same effect.

Lazy formatting: a small habit that pays off

You'll often see two ways to log a value. They look similar but behave differently:

# Eager: the string is always built, even if DEBUG is turned off
logger.debug("User payload: " + str(payload))

# Lazy: pass arguments and let logging format them only if needed
logger.debug("User payload: %s", payload)

In the second form, logging holds onto payload and only performs the %s substitution — which calls str(payload) — if a handler actually emits the record. When DEBUG is filtered out, that string conversion never runs. For cheap values the savings are tiny, but when the object has an expensive __str__ (think a large dataframe or a serialized request), this habit keeps suppressed debug logs from doing real work. Make passing arguments your default; it costs nothing and occasionally saves a lot.

Rotating log files so they don't eat your disk

A plain FileHandler grows forever. In production you want rotation — old logs get archived and eventually deleted. The logging.handlers module has two workhorses:

from logging.handlers import RotatingFileHandler, TimedRotatingFileHandler

# Roll over once the file passes 5 MB, keep 5 old files
size_based = RotatingFileHandler(
    "app.log", maxBytes=5_000_000, backupCount=5,
)

# Roll over at midnight, keep one week of history
time_based = TimedRotatingFileHandler(
    "app.log", when="midnight", backupCount=7,
)

With backupCount=5, you'll see app.log plus app.log.1 through app.log.5; the oldest is dropped on the next rotation. It's a one-line change that prevents a runaway log file from filling a server's disk at 3 a.m.

Common pitfalls

Duplicate log lines. If a message appears twice, you've usually added handlers to a logger and to the root, or you've run handler-adding code more than once (common when a module is imported repeatedly or in notebooks). Configure logging in exactly one place, and guard against re-adding handlers.

Libraries shouldn't configure logging. If you publish a package, never call basicConfig() or add handlers at import time — that's the application's job. Instead, attach a do-nothing handler so your library stays quiet until the app opts in:

# In your library's top-level __init__.py
import logging

logging.getLogger(__name__).addHandler(logging.NullHandler())

Reaching for the root logger everywhere. Calling logging.info() directly works, but you lose the per-module names that make large logs navigable. Spend the one extra line on logger = logging.getLogger(__name__) at the top of each module.

Wrap-up and next steps

The mental model is small once it lands: a logger you obtain with getLogger(__name__) decides whether to process a message, one or more handlers decide where it goes, and formatters decide how each line reads. Set levels deliberately, configure everything once at your program's entry point (with basicConfig for small tools or dictConfig for real apps), use logger.exception() in your error handlers, and pass arguments instead of pre-building strings.

From here, three directions are worth exploring: structured/JSON logging (with a library like structlog or a custom formatter) so machines can parse your logs; adding contextual fields such as a request ID via logging.LoggerAdapter or filters; and centralizing logs from many services. But you don't need any of that to start — swap your next batch of print() calls for a named logger today, and your future self debugging production will thank you.