Python Memory Profiling

tracemalloc, in the standard library since 3.4, hooks into Python's memory allocator and records, for every single object allocated while tracing is active, exactly which line of code triggered that allocation. tracemalloc.start() begins that tracking; tracemalloc.take_snapshot() captures the complete current state — every currently-live traced allocation, grouped and attributable back to its originating source line.

snapshot.statistics("lineno") aggregates that snapshot by source line, ranking lines by how much memory is currently attributable to allocations made there — printing the top few entries answers, with actual precision, 'which specific lines in my code are responsible for the most memory currently in use', rather than a guess based on which parts of the code *look* memory-intensive.

This line-level attribution is what makes tracemalloc fundamentally more useful for diagnosis than a single aggregate 'total memory used' number (from sys.getsizeof on some suspected culprit, or from an OS-level process monitor) — it directly answers the actual debugging question ('which line is responsible') instead of requiring you to manually narrow down the cause through trial and error.

A single snapshot answers 'what's using memory right now', which includes everything allocated since the program started — a lot of that is baseline overhead, imported libraries, and normal working state that isn't the actual problem. Diagnosing 'this specific operation seems to leak memory' requires isolating exactly what grew *during* that operation specifically, filtering out everything that was already present before it ran.

snapshot2.compare_to(snapshot1, "lineno") does exactly this: it computes the difference between two snapshots, line by line, showing precisely what memory was newly allocated (or freed) between the two points in time they were taken. Wrapping a suspected operation between two take_snapshot() calls and comparing them turns 'I think this function leaks memory' into a precise, ranked list of exactly which lines allocated memory during that specific operation and how much.

This before/after comparison technique is the standard, most effective tracemalloc workflow for hunting a real memory leak: rather than staring at one large, undifferentiated snapshot trying to guess what's abnormal, you isolate the suspected operation between two snapshots and let the diff show you precisely, and only, what that operation itself contributed — the memory-profiling equivalent of profiling's 'sort by cumulative time to find the real bottleneck' discipline from the previous lesson.

sys.getsizeof(obj) reports the number of bytes an individual object itself occupies in memory — useful for quick, direct comparisons, like confirming that a tuple is generally more memory-efficient than an equivalent list, or that a frozenset costs less than a set with the same elements. It's a fast, simple check for comparing the raw container overhead of different data structure choices.

The important caveat, and a common source of misleading conclusions: getsizeof() measures only the object's *own* immediate footprint, not the recursive size of everything it references. sys.getsizeof(large_list) for a list containing 1000 large strings reports only the size of the list's internal array of *pointers* to those strings — not the actual memory the 1000 string objects themselves consume, which could be dramatically larger than what getsizeof() reports for the list alone.

For a genuinely complete picture of a complex object graph's total memory footprint — a list of dictionaries, each containing nested lists, for instance — sys.getsizeof() alone is insufficient; you'd need either a recursive traversal that sums getsizeof() across every reachable object (careful to handle shared references and avoid double-counting), or a purpose-built tool like pympler or tracemalloc's own snapshot-based line-level accounting, which correctly attributes memory to where it was actually allocated regardless of how deeply nested the resulting object graph becomes.

'Something is leaking memory' is one of the scariest bug reports to get without a tool that can point at the exact line responsible. tracemalloc is that tool.

tracemalloc.start() begins tracking every memory allocation -- take_snapshot() then captures the current state for analysis.

Comparing two snapshots reveals exactly what grew BETWEEN them -- the single most useful technique for finding a memory leak.

Checkpoint: Why is comparing two tracemalloc snapshots (before/after a suspected operation) more useful than looking at just one snapshot?

• →It isolates exactly what memory was allocated specifically during that operation, filtering out everything already present before it
• →Comparing two snapshots runs significantly faster than taking one

sys.getsizeof() gives you the size of a SINGLE object -- useful for comparing data structure choices directly.

Checkpoint: Does sys.getsizeof(large_list) include the memory used by the objects the list contains?

• →No — it only measures the container itself, not the objects it references
• →Yes — it includes the full recursive size of everything the list contains

With time and memory both measurable, Benchmarking gives you the rigor to compare two implementations fairly, not just intuitively.

Measure Real Memory Allocation. Finish measure_allocation(): tracemalloc reports real allocation, not a guess.