itertools is a toolbox of fast, memory-efficient building blocks for iteration, all lazy (returning iterators, per the Iterators lesson) and all composable with each other. This lesson covers the four you'll reach for most often in real code.
1chain(): Flattening Multiple Iterables Lazily
chain(list1, list2, list3) produces a single iterator that yields every item from list1, then every item from list2, then every item from list3 ā functionally similar to list1 + list2 + list3, but critically different in one respect: + on lists eagerly builds a brand new combined list in memory immediately, while chain() yields items lazily, one at a time, pulling from each source iterable only as needed and never materializing a combined sequence at all.
This matters most when the sources are themselves large or lazy (generators, file iterators) ā chain() lets you treat several separate lazy sources as one logical stream without forcing any of them to fully materialize just to combine them, preserving the exact memory-efficiency benefits generators exist to provide (as covered in the Generators lesson).
chain.from_iterable(iterable_of_iterables) is a companion variant for when you have a single iterable *containing* several iterables to flatten (rather than several separate iterable arguments) ā useful when the list of things to chain is itself computed dynamically rather than known and spelled out at the call site.
from itertools import chain
list1 = [1, 2, 3]
list2 = [4, 5, 6]
list3 = [7, 8, 9]
for item in chain(list1, list2, list3):
print(item) # 1, 2, 3, 4, 5, 6, 7, 8, 9 -- iterated lazily, one source at a timeYields items lazily ā no combined list ever built in memory
2groupby(): Consecutive Runs, Not a Full Grouping ā Sort First
groupby(data, key=...) is a streaming operation: it walks through data in order, and starts a *new* group every single time the computed key changes from the previous item ā it has no memory of keys it saw earlier in the stream. This is precisely why the example produces two separate "A" groups instead of one: the unsorted input [("A",1), ("A",2), ("B",3), ("A",4)] has "A" appearing, then "B", then "A" again ā from groupby's consecutive-only perspective, that's genuinely three separate runs, not two groups.
The fix, and the correct idiomatic usage of groupby, is to sort the data by the same key *before* grouping: sorted(data, key=lambda x: x[0]) first guarantees every item sharing a key is consecutive in the sequence, so groupby then produces exactly one group per distinct key value, matching the intuitive 'group everything with this key together' behavior most people expect from the name.
A second detail worth internalizing: each group groupby yields is itself a lazy iterator, not a list ā list(group) (as shown) materializes it, but if you don't consume a group before advancing to the next one (e.g. by storing groups in a list without converting them first), the underlying shared iterator state can produce surprising empty results, since groupby's internal iterator is shared and advances as you consume each group. Materializing groups you intend to keep (list(group)) as you go is the safe default.
from itertools import groupby
data = [("A", 1), ("A", 2), ("B", 3), ("A", 4)] # NOT sorted by first element
for key, group in groupby(data, key=lambda x: x[0]):
print(key, list(group))
# A [(A,1),(A,2)] B [(B,3)] A [(A,4)] -- 'A' appears TWICE because it's not sorted first!Guarantees one group per distinct key
3product() and islice(): Combinations and Lazy Slicing
product(sizes, colors) computes the full Cartesian product ā every possible combination of one item from sizes and one item from colors ā which is exactly what a nested for size in sizes: for color in colors: loop computes manually, expressed instead as a single, flat, composable iterator call. product() also accepts a repeat argument for computing the Cartesian product of an iterable with itself N times (product(range(2), repeat=3) generates every 3-bit binary combination), which would require a proportionally deeper nested loop to replicate by hand.
islice(iterable, stop) (or islice(iterable, start, stop, step)) provides slice-like behavior ā [start:stop:step] semantics ā for *any* iterator, including generators and other lazy sources that don't support Python's normal subscript slicing syntax at all, since slicing requires __getitem__, which a plain generator object (whose only real interface is __next__) doesn't implement. islice(infinite_counter(), 5) is the only way to safely get 'the first 5 values' from a genuinely infinite generator ā trying to convert it to a list first (list(infinite_counter())[:5]) would hang forever, since list() tries to exhaust the generator completely before slicing ever happens.
The unifying theme across all of itertools, and the reason it's worth learning as a toolbox rather than memorizing function-by-function: every tool here composes with every other ā islice(chain(gen_a(), gen_b()), 10), groupby(sorted(product(a, b))) ā because they all consume and produce the same lazy iterator protocol, letting you build sophisticated iteration pipelines out of small, well-understood, memory-efficient pieces.
from itertools import product
sizes = ["S", "M", "L"]
colors = ["red", "blue"]
for size, color in product(sizes, colors):
print(size, color)
# S red, S blue, M red, M blue, L red, L blue -- replaces a nested for loop[0, 1, 2, 3, 4] ā safe slicing of an infinite generator
4Step-by-Step Breakdown
Nested for loops building intermediate lists are often a sign itertools has a purpose-built, lazier tool for exactly that job. Let's meet the essentials.
chain() flattens multiple iterables into one, lazily -- no intermediate combined list is ever built.
groupby() groups CONSECUTIVE items sharing a key -- it does NOT group across the whole iterable like a dict would, unless the data is already sorted by that key.
Checkpoint: Why does groupby() produce TWO separate 'A' groups in the example, instead of one combined group?
- āgroupby only groups consecutive items sharing a key ā the data was not sorted by that key first
- āThis is unexpected behavior ā groupby should always group all matching items together
product() generates the Cartesian product -- every combination -- replacing nested for loops entirely.
islice() lazily slices an iterator -- crucial because a generator can't be sliced with normal [start:stop] syntax.
Checkpoint: Why is islice() needed to get the first 5 values from infinite_counter(), instead of just infinite_counter()[0:5]?
- āA generator object does not support slice syntax ([start:stop]) ā it only supports next()
- āSlice syntax would work but would loop forever trying to compute the length first
itertools composes iteration; functools composes and caches FUNCTIONS ā the next natural tool in the same 'functional building blocks' family.
Chain Real Iterables Lazily. Finish flatten_lazily(): chain() flattens multiple iterables without building an intermediate list.
Level Up š
Advanced cheat sheets, SEO tricks, and interview prep for this topic.
Browser Support
Fully supported (via server-side Python execution).
Fully supported (via server-side Python execution).
Fully supported (via server-side Python execution).
Fully supported (via server-side Python execution).
Best Practices
Always sort data by the grouping key before calling groupby()
groupby() only groups CONSECUTIVE matching items ā without a prior sort, data with a repeated but non-adjacent key silently produces multiple separate groups for what should logically be one group.
Reach for itertools tools over manual nested loops and intermediate lists when composing iteration logic
chain, product, and islice express common iteration patterns declaratively and lazily, avoiding both the verbosity of manual loops and the memory cost of intermediate materialized lists.
Frequent Bugs
Calling groupby() on unsorted data and assuming it behaves like a full grouping (as a dict-based groupby would), silently producing multiple fragmented groups for the same key.
Always sort the data by the same key function passed to groupby() immediately before calling it, guaranteeing every item sharing a key value is consecutive.
Real-World Examples
Grouping Sorted Transaction Records by Date for a Report
A financial report needs to group a list of transaction records by date, producing one summary per distinct date, where the transaction list can be sorted by date first.
from itertools import groupby
transactions = sorted(all_transactions, key=lambda t: t.date)
for date, group in groupby(transactions, key=lambda t: t.date):
daily_transactions = list(group)
total = sum(t.amount for t in daily_transactions)
print(f"{date}: {len(daily_transactions)} transactions, total ${total:.2f}")