The with Statement, Demystified: A Practical Deep Dive into Context Managers and contextlib

You have almost certainly written with open("file.txt") as f: without thinking twice about it. The file closes itself, even if your code raises an exception halfway through. That reliability is not magic — it is the context manager protocol, one of Python's most elegant features for guaranteeing that setup is always paired with cleanup.

Once you understand how with actually works, you can wrap any "do this, then always undo it" pattern in the same clean syntax: database transactions, locks, temporary directories, timers, redirecting output, or silencing expected errors. In this deep dive we will build context managers from scratch, then tour the contextlib module — the standard library's toolbox that removes almost all of the boilerplate.

Why context managers exist

The problem they solve is deterministic cleanup. Consider the naive way to work with a resource that must be released:

f = open("data.txt")
data = f.read()
process(data)   # if this raises, f.close() below never runs
f.close()

If process(data) throws, the file handle leaks. You could reach for try/finally, and that works, but it is verbose and easy to forget. The with statement bakes the try/finally in for you:

with open("data.txt") as f:
    data = f.read()
    process(data)
# f is closed here no matter what happened above

The object handed to with just needs to implement two methods. That is the whole protocol.

The protocol: __enter__ and __exit__

A context manager is any object with an __enter__ method (run on entering the block) and an __exit__ method (run on leaving it, whether normally or via an exception). Here is a working timer:

import time

class Timer:
    def __enter__(self):
        self.start = time.perf_counter()
        return self          # value bound to the "as" name

    def __exit__(self, exc_type, exc_val, exc_tb):
        self.elapsed = time.perf_counter() - self.start
        print(f"Elapsed: {self.elapsed:.4f}s")
        return False         # do not suppress exceptions

with Timer() as t:
    total = sum(range(1_000_000))

print(t.elapsed)

Two details matter. First, whatever __enter__ returns is what gets bound to the name after as — often self, but it can be anything. Second, __exit__ receives the details of any exception raised inside the block: its type, value, and traceback. If the block exits cleanly, all three arguments are None.

Suppressing exceptions with __exit__

The return value of __exit__ is a signal. If it returns a truthy value, Python treats the exception as handled and swallows it. Return False (or nothing) and the exception propagates normally. This is powerful but easy to abuse:

class IgnoreValueError:
    def __enter__(self):
        return self

    def __exit__(self, exc_type, exc_val, exc_tb):
        # Only swallow ValueError; let everything else bubble up
        return exc_type is ValueError

with IgnoreValueError():
    raise ValueError("this disappears quietly")

print("execution continues")

Pitfall: accidentally returning a truthy value from __exit__ (for example, returning self or a non-empty message) will silently hide every exception, including bugs. Be explicit and return False unless you truly intend to suppress.

contextlib.contextmanager: the generator shortcut

Writing a whole class for a simple manager is a lot of ceremony. The @contextmanager decorator lets you write one as a generator function instead. Everything before the yield is the "enter" phase; everything after is the "exit" phase. The value you yield becomes the as target.

from contextlib import contextmanager

@contextmanager
def managed_connection(dsn):
    conn = connect(dsn)          # setup
    try:
        yield conn               # hand control to the with-block
    finally:
        conn.close()             # teardown, always runs

with managed_connection("postgres://...") as conn:
    conn.execute("SELECT 1")

The try/finally around the yield is the key idiom. If the with block raises, the exception is re-raised at the point of the yield, so your finally still runs. You can even catch it if you want:

@contextmanager
def transaction(db):
    db.begin()
    try:
        yield db
    except Exception:
        db.rollback()
        raise            # re-raise after cleaning up
    else:
        db.commit()

This reads almost like plain English: begin, do the work, roll back on failure, otherwise commit. That clarity is why @contextmanager is the most common way to write custom managers in modern Python.

Bonus: they double as decorators

A manager built with @contextmanager can also be applied directly to a function, wrapping every call in the context automatically:

@contextmanager
def timed(label):
    start = time.perf_counter()
    try:
        yield
    finally:
        print(f"{label}: {time.perf_counter() - start:.4f}s")

@timed("build")
def build_report():
    ...   # timed on every call, no with-block needed

Ready-made tools in contextlib

Before writing your own, check whether the standard library already has what you need.

suppress: skip expected errors cleanly

contextlib.suppress replaces the noisy try/except/pass pattern:

import os
from contextlib import suppress

# Instead of try/except FileNotFoundError: pass
with suppress(FileNotFoundError):
    os.remove("cache.tmp")

redirect_stdout and redirect_stderr

Temporarily capture or reroute output — handy for testing or for taming a chatty third-party function:

import io
from contextlib import redirect_stdout

buffer = io.StringIO()
with redirect_stdout(buffer):
    print("this goes into the buffer")

captured = buffer.getvalue()
print(f"Captured: {captured!r}")

closing: adapt objects that only have close()

Some objects have a close() method but do not implement the context manager protocol. closing wraps them so they work with with:

from contextlib import closing
from urllib.request import urlopen

with closing(urlopen("https://example.com")) as page:
    html = page.read()
# page.close() is guaranteed here

ExitStack: managing a dynamic number of resources

Sometimes you do not know at coding time how many resources you need — imagine opening a list of files whose length is only known at runtime. Nesting with statements does not work when the count is variable. ExitStack solves this by letting you register context managers dynamically; it unwinds them in reverse order (last in, first out) when the block ends:

from contextlib import ExitStack

filenames = ["a.txt", "b.txt", "c.txt"]

with ExitStack() as stack:
    files = [stack.enter_context(open(name)) for name in filenames]
    # all files are open here
    for f in files:
        process(f.read())
# every file is closed automatically, in reverse order

enter_context enters a manager and remembers it for cleanup. You can also register plain callbacks with stack.callback(func, *args), and — for advanced cases — transfer ownership of the cleanup with stack.pop_all() so resources survive past the block only when setup fully succeeds.

A cleaner way to nest multiple managers

When you have a fixed, known set of managers, you do not need ExitStack. Since Python 3.10 you can group them in parentheses for readability:

with (
    open("input.txt") as src,
    open("output.txt", "w") as dst,
):
    dst.write(src.read())

Common pitfalls

Yielding more than once. A @contextmanager generator must yield exactly once. A second yield raises RuntimeError. Keep the generator linear: setup, one yield, teardown.

Forgetting the finally. If you write @contextmanager without wrapping the yield in try/finally, an exception in the block will skip your cleanup — defeating the entire purpose. Always guard teardown with finally.

Reusing a single-shot manager. Generator-based managers are not reusable; entering the same object twice will fail. If you need reuse, create a fresh instance each time or write a class that resets its state in __enter__.

Over-suppressing. Returning a truthy value from __exit__ to hide errors is occasionally right and frequently a footgun. Suppress narrowly, and prefer contextlib.suppress so your intent is explicit.

Wrap-up and next steps

Context managers turn the fragile "remember to clean up" pattern into a guarantee the language enforces for you. Start by recognizing the shape: any time you write try/finally to pair setup with teardown, a context manager will read better and be reusable. Reach for @contextmanager first — it covers the vast majority of cases with the least code — and drop down to a class with __enter__/__exit__ only when you need richer state or reuse. Keep suppress, redirect_stdout, closing, and ExitStack in your back pocket for the situations they were built for.

From here, try refactoring one try/finally block in your own code into a context manager, then experiment with ExitStack to manage a variable-length group of resources. Once the pattern clicks, you will start seeing "enter and exit" pairs everywhere — and your resource handling will be the better for it.