HTTP the Modern Way: A Practical Deep Dive into Python's httpx

For years, requests was the default way to talk to the web from Python. It is still excellent, but it has one hard limit: it is synchronous only. The moment you need to fire off dozens of requests concurrently, use HTTP/2, or share one async client across your web app, you are working against the grain. httpx was built to fix exactly this. It gives you a nearly identical, requests-style API for everyday scripts, plus a first-class async client, HTTP/2 support, and timeouts that are switched on by default.

This guide walks through httpx from a first request to production-shaped patterns: query parameters and JSON bodies, the reusable Client, async concurrency, timeouts, error handling, and streaming. Every snippet below was checked against httpx 0.28.1 on Python 3.10.

Installing and your first request

httpx is pure-Python with a small dependency tree. Install it, optionally with HTTP/2 support:

pip install httpx
# For HTTP/2:
pip install "httpx[http2]"

The module-level functions mirror requests almost exactly, so a first request feels familiar:

import httpx

response = httpx.get("https://api.github.com/repos/encode/httpx")
response.raise_for_status()

data = response.json()
print(response.status_code)          # 200
print(data["stargazers_count"])      # an integer
print(response.headers["content-type"])

A Response object carries everything you need: .status_code, .headers, .text, .content (raw bytes), and .json() for decoding a JSON body. Convenient, but for anything beyond a throwaway script you should not use these module-level helpers, and we will see why shortly.

Query parameters, headers, and JSON bodies

You rarely send a bare URL. httpx builds query strings for you from a dict passed to params, sets headers from a dict, and serializes a Python object to a JSON request body when you pass json=:

import httpx

# GET with query parameters -> ...?q=python&page=2
r = httpx.get(
    "https://httpbin.org/get",
    params={"q": "python", "page": 2},
    headers={"User-Agent": "pykit-demo"},
)
print(r.request.url)   # inspect the final, encoded URL

# POST a JSON body (Content-Type is set to application/json for you)
r = httpx.post("https://httpbin.org/post", json={"name": "PyKIT", "active": True})
print(r.json()["json"])   # {'name': 'PyKIT', 'active': True}

Note the difference between json= and data=. Use json= for a JSON API; use data= with a dict for classic application/x-www-form-urlencoded form posts, and files= for multipart uploads. Mixing these up is a common source of "why is my API returning 400?" confusion.

The Client: connection pooling done right

Here is the single most important httpx habit. Every module-level call (httpx.get, httpx.post, ...) spins up a fresh connection, performs the TLS handshake, sends one request, and tears everything down. Do that in a loop and you pay the setup cost every single time. A Client keeps a connection pool alive and reuses it, which is dramatically faster and lets you set shared configuration once:

import httpx

with httpx.Client(
    base_url="https://api.github.com",
    headers={"Accept": "application/vnd.github+json"},
    timeout=10.0,
) as client:
    repo = client.get("/repos/encode/httpx").json()
    issues = client.get("/repos/encode/httpx/issues", params={"state": "open"}).json()
    print(repo["name"], "has", len(issues), "issues on this page")

Using the client as a context manager (with ...) guarantees the pool is closed cleanly when you are done. The base_url and default headers apply to every request, so you configure authentication and content negotiation in exactly one place. Think of the module-level functions as a convenience for interactive experiments, and the Client as what you actually ship.

Going async with AsyncClient

This is httpx's headline feature. AsyncClient exposes the same methods as Client, but they are coroutines you await. Combined with asyncio.gather, you can run many requests concurrently instead of one after another:

import asyncio
import httpx

async def fetch_all(urls):
    async with httpx.AsyncClient(timeout=10.0) as client:
        tasks = [client.get(url) for url in urls]
        responses = await asyncio.gather(*tasks)
        return [r.status_code for r in responses]

urls = [f"https://httpbin.org/get?i={i}" for i in range(20)]
print(asyncio.run(fetch_all(urls)))

Those twenty requests overlap in flight rather than blocking one at a time, so total wall-clock time is roughly that of the slowest single request instead of the sum of all of them. Because the sync and async APIs are mirror images, porting code between them is mostly a matter of adding async/await and swapping the client class.

Timeouts are on by default

A subtle but important difference from requests: httpx applies a default timeout of five seconds, so a hung server cannot freeze your program forever. You can tune it globally or split it into phases (connect, read, write, pool):

import httpx

# One number applies to all phases
client = httpx.Client(timeout=15.0)

# Or control each phase independently
timeout = httpx.Timeout(10.0, connect=5.0)
client = httpx.Client(timeout=timeout)

# Opt out entirely for a specific call (use sparingly!)
r = client.get("https://example.com/slow", timeout=None)

Disabling timeouts with None is occasionally necessary for long-lived streams, but treat it as a deliberate exception, not a default.

Handling errors properly

httpx does not raise on a 4xx or 5xx status automatically. A "404 Not Found" is still a valid HTTP response, so you get a Response object back. Call raise_for_status() to turn bad statuses into exceptions, and catch the specific error types httpx provides:

import httpx

try:
    r = httpx.get("https://httpbin.org/status/503", timeout=5.0)
    r.raise_for_status()
    data = r.json()
except httpx.HTTPStatusError as exc:
    # The server responded, but with an error status
    print("Bad status:", exc.response.status_code)
except httpx.TimeoutException:
    print("Request timed out")
except httpx.RequestError as exc:
    # Connection refused, DNS failure, etc. -- no response at all
    print("Network problem:", exc)

The hierarchy is worth remembering: httpx.RequestError is the base for transport-level failures (no response), while httpx.HTTPStatusError is only raised by raise_for_status() and always carries a .response. Both share the common ancestor httpx.HTTPError, which is handy as a catch-all.

Streaming large responses

Calling .content or .json() loads the entire body into memory. For a multi-gigabyte download or a long server-sent-events feed, that is a fast route to an out-of-memory crash. Use client.stream() to consume the body in chunks as it arrives:

import httpx

with httpx.Client() as client:
    with client.stream("GET", "https://httpbin.org/bytes/10485760") as response:
        response.raise_for_status()
        downloaded = 0
        with open("big.bin", "wb") as f:
            for chunk in response.iter_bytes():
                f.write(chunk)
                downloaded += len(chunk)
        print("Downloaded", downloaded, "bytes")

Inside a streaming block you can iterate with iter_bytes(), iter_text(), or iter_lines(). Just remember that the response body is consumed as you go, so you cannot also call .text or .json() on the same response afterward.

Tips and common pitfalls

A few habits will save you real debugging time. Reuse one client for the lifetime of your program or request handler instead of creating a new one per call; creating and discarding clients throws away the connection pool you are paying for. Do not forget to close the client if you are not using a with block, or you will leak sockets. Set an explicit timeout on every external call rather than relying on the default when latency matters. Enable HTTP/2 explicitly with httpx.Client(http2=True) if your target supports it, since it is off by default. And when testing, reach for httpx.MockTransport or the pytest-httpx plugin to simulate responses without touching the network at all.

Wrap-up and next steps

httpx gives you the ergonomics of requests with the concurrency story modern Python needs. Start by replacing bare httpx.get calls with a shared Client, lean on raise_for_status() and the typed exceptions for robust error handling, and set timeouts deliberately. When you have workloads that fan out to many endpoints, move them to AsyncClient and let asyncio.gather do the heavy lifting. From there, explore the parts we did not cover: authentication flows, event hooks for logging and retries, custom transports, and streaming uploads. The API surface is small and consistent, so each of those builds naturally on the patterns you have already seen here.