JSON Without the Footguns: A Practical Deep Dive into Python's json Module

JSON is the lingua franca of the modern web. Every API you call, every config file you parse, and every message queue you read from probably speaks it. Python's json module makes the easy cases trivial — json.dumps to serialize, json.loads to parse — which is exactly why most developers never learn what it can actually do. Then a datetime shows up in a payload, a duplicate key silently overwrites your data, or a rogue NaN produces output that no other parser will accept, and suddenly the "simple" module has teeth.

This is a practical tour of the parts of json that matter in real code: serializing types the module doesn't understand, customizing decoding, controlling output format, and — most importantly — the sharp edges that cause production bugs. Everything here works on the standard library alone, no dependencies.

The four functions you already half-know

The module has exactly four entry points. Two work with strings, two work with file-like objects. The s suffix means "string."

import json

data = {"name": "Ada", "langs": ["Python", "C"], "active": True, "score": None}

# Object -> string, and back
text = json.dumps(data)
print(text)
# {"name": "Ada", "langs": ["Python", "C"], "active": true, "score": null}

restored = json.loads(text)
print(restored == data)  # True

Notice the type translation: Python's True becomes true, None becomes null, and a dict becomes a JSON object. Going the other way, JSON objects become dicts, arrays become lists, and numbers become int or float. The file-based pair, json.dump and json.load, do the same thing straight to and from a file handle:

with open("config.json", "w", encoding="utf-8") as f:
    json.dump(data, f, indent=2)

with open("config.json", encoding="utf-8") as f:
    loaded = json.load(f)

Always pass encoding="utf-8" when you open the file. JSON is defined as UTF-8, but Python opens text files using your platform's default encoding, which on some Windows machines is not UTF-8. Being explicit prevents a class of "works on my machine" bugs.

Making output readable — and compact

The default output is a dense single line. Three keyword arguments change that. indent turns on pretty-printing, sort_keys gives you deterministic key ordering (invaluable for diffs and cache keys), and separators lets you strip every wasted byte.

obj = {"b": 2, "a": 1}

print(json.dumps(obj, indent=2, sort_keys=True))
# {
#   "a": 1,
#   "b": 2
# }

# Compact: no spaces after separators. Great for network payloads.
print(json.dumps(obj, separators=(",", ":")))
# {"a":1,"b":2}

The default separators are (", ", ": "). When you enable indent, Python automatically drops the trailing whitespace from the item separator, so you don't get ugly trailing spaces at the end of each line.

The ensure_ascii trap

By default, json.dumps escapes every non-ASCII character into a \uXXXX sequence. This is technically valid JSON and safe to transmit over any channel, but it makes human-readable text unreadable and bloats the output.

print(json.dumps({"city": "München"}))
# {"city": "München"}

print(json.dumps({"city": "München"}, ensure_ascii=False))
# {"city": "München"}

Set ensure_ascii=False when you're writing UTF-8 files or logs meant for humans. Keep the default when you can't guarantee the downstream channel handles raw Unicode.

Serializing types JSON doesn't understand

The moment you try to serialize a datetime, a Decimal, a set, or your own class, you get a TypeError: Object of type ... is not JSON serializable. The clean fix is the default parameter: a function that json.dumps calls for any object it can't handle. Return something serializable, or raise TypeError to signal you can't help either.

from datetime import datetime, date
from decimal import Decimal

def to_serializable(obj):
    if isinstance(obj, (datetime, date)):
        return obj.isoformat()
    if isinstance(obj, Decimal):
        return float(obj)
    if isinstance(obj, set):
        return sorted(obj)
    raise TypeError(f"Cannot serialize {type(obj).__name__}")

payload = {"when": datetime(2026, 8, 2, 9, 30), "price": Decimal("19.99")}
print(json.dumps(payload, default=to_serializable))
# {"when": "2026-08-02T09:30:00", "price": 19.99}

For reusable logic, subclass JSONEncoder instead and pass it with cls. This pairs beautifully with dataclasses:

import dataclasses

class DataclassEncoder(json.JSONEncoder):
    def default(self, obj):
        if dataclasses.is_dataclass(obj) and not isinstance(obj, type):
            return dataclasses.asdict(obj)
        return super().default(obj)

@dataclasses.dataclass
class Point:
    x: int
    y: int

print(json.dumps(Point(1, 2), cls=DataclassEncoder))
# {"x": 1, "y": 2}

Calling super().default(obj) for anything you don't recognize is important — it raises the standard TypeError so unexpected types still fail loudly instead of silently vanishing.

Customizing the parse: hooks that rebuild your types

Decoding is symmetric. Three hooks let you intercept parsing and reconstruct richer types. object_hook receives every JSON object as a dict and returns whatever you want in its place:

def as_point(d):
    if "x" in d and "y" in d:
        return Point(d["x"], d["y"])
    return d

print(json.loads('{"x": 3, "y": 4}', object_hook=as_point))
# Point(x=3, y=4)

If you care about numeric precision — money is the classic case — parse_float lets you swap Python's lossy binary float for Decimal:

from decimal import Decimal

result = json.loads('{"price": 19.99}', parse_float=Decimal)
print(result)  # {'price': Decimal('19.99')}

This matters because 0.1 + 0.2 is famously not 0.3 in binary floating point. If your JSON carries prices or financial figures, parsing them straight into Decimal keeps them exact from the moment they enter your program.

The footguns

Now the part that actually saves you from 2 a.m. debugging.

Duplicate keys vanish silently

JSON technically allows duplicate keys, and Python's parser resolves them by simply keeping the last one — no warning, no error.

print(json.loads('{"a": 1, "a": 2}'))
# {'a': 2}   <- the first value is gone

If you're parsing data from an untrusted or bug-prone source, use object_pairs_hook, which receives the raw key/value pairs before they're collapsed into a dict, so you can detect collisions:

def reject_duplicates(pairs):
    seen = {}
    for key, value in pairs:
        if key in seen:
            raise ValueError(f"Duplicate key: {key!r}")
        seen[key] = value
    return seen

json.loads('{"a": 1, "a": 2}', object_pairs_hook=reject_duplicates)
# ValueError: Duplicate key: 'a'

NaN and Infinity are not valid JSON

By default Python happily emits NaN, Infinity, and -Infinity — literals that the JSON specification forbids and that many other parsers (JavaScript's JSON.parse, most strict validators) will reject.

print(json.dumps(float("nan")))  # NaN  <- other parsers choke on this

# Fail fast instead:
json.dumps(float("nan"), allow_nan=False)
# ValueError: Out of range float values are not JSON compliant

Pass allow_nan=False whenever you produce JSON for external systems. It converts a silent interoperability bug into an immediate, obvious error.

Dictionary keys are always coerced to strings

JSON object keys must be strings, so Python quietly stringifies integer, float, and boolean keys on the way out — and does not restore them on the way in.

encoded = json.dumps({1: "a", 2: "b"})
print(encoded)                 # {"1": "a", "2": "b"}
print(json.loads(encoded))     # {'1': 'a', '2': 'b'}  <- keys are now strings

The round trip is not lossless. If you need integer keys preserved, convert them yourself after loading, or restructure the data as a list of objects.

Handle JSONDecodeError with real diagnostics

When parsing fails, don't just catch a bare exception. JSONDecodeError carries the exact location of the problem, which turns "invalid JSON somewhere" into an actionable message.

try:
    json.loads('{"broken": }')
except json.JSONDecodeError as e:
    print(f"{e.msg} at line {e.lineno}, column {e.colno} (char {e.pos})")
# Expecting value at line 1, column 12 (char 11)

Handling big data: JSON Lines

The json module loads an entire document into memory at once, which is fine until your file is gigabytes. The pragmatic answer for large or streaming datasets is JSON Lines (.jsonl): one independent JSON object per line. You process it record by record, with constant memory.

records = [{"id": 1, "name": "a"}, {"id": 2, "name": "b"}]

# Write one object per line
with open("data.jsonl", "w", encoding="utf-8") as f:
    for record in records:
        f.write(json.dumps(record) + "\n")

# Read back lazily — never more than one record in memory
with open("data.jsonl", encoding="utf-8") as f:
    for line in f:
        line = line.strip()
        if line:
            record = json.loads(line)
            print(record["id"])

This pattern is the backbone of log files, data-pipeline dumps, and machine-learning datasets. It also recovers gracefully: one corrupt line doesn't destroy the whole file.

Wrap-up and next steps

The json module rewards a few minutes of real attention. Reach for default and a JSONEncoder subclass to serialize your own types, object_hook and parse_float to reconstruct rich objects on the way in, and indent, sort_keys, and separators to control the shape of the output. Just as importantly, defend against the footguns: reject duplicate keys with object_pairs_hook, forbid non-compliant floats with allow_nan=False, remember that dict keys become strings, and always read JSONDecodeError for the exact failure location.

From here, two directions are worth exploring. If raw speed matters — parsing millions of records — benchmark a drop-in third-party library such as orjson or ujson against the standard library for your workload. And if you want validated, typed structures rather than loose dicts, pair json with a data-modeling layer like dataclasses or Pydantic so that malformed payloads are caught at the boundary of your program rather than deep inside it.