Python Itertools

The itertools module is part of Python’s standard library and provides a set of fast, memory-efficient building blocks for working with iterators. Instead of writing manual loops to generate combinations, group data, or build infinite sequences, you can compose small, well-tested functions from itertools that do the job lazily — producing values one at a time instead of building entire lists in memory. Mastering it lets you write code that is both shorter and dramatically more efficient when dealing with large or infinite data streams.

Overview / How Itertools Works

Every function in itertools returns an iterator: an object that implements the iterator protocol (__iter__ and __next__) and yields values on demand rather than all at once. This is the same protocol that powers for loops, generator functions, and generator expressions. The difference is that itertools functions are implemented in C for speed, and they are designed to be combined like Lego bricks: the output of one iterator can feed directly into another.

The module’s functions fall into three broad families:

  • Infinite iterators — count(), cycle(), and repeat() never run out on their own; you must limit them explicitly (usually with islice() or a break).
  • Terminating iterators — functions like chain(), groupby(), accumulate(), compress(), islice(), zip_longest(), takewhile(), and dropwhile() transform or combine one or more finite iterables and stop when their inputs are exhausted.
  • Combinatoric generators — product(), permutations(), combinations(), and combinations_with_replacement() generate arrangements or selections of items from an input iterable.

Because everything is lazy, memory use stays flat even for huge inputs: itertools.count() can represent an infinite counting sequence using a single small object, not a list of numbers. This makes itertools the tool of choice whenever you’d otherwise write a manual accumulator loop, a nested-loop combination generator, or a hand-rolled grouping algorithm.

Syntax

import itertools

itertools.count(start=0, step=1)
itertools.cycle(iterable)
itertools.repeat(object, times=None)
itertools.chain(*iterables)
itertools.islice(iterable, stop)
itertools.groupby(iterable, key=None)
itertools.accumulate(iterable, func=None)
itertools.product(*iterables, repeat=1)
itertools.permutations(iterable, r=None)
itertools.combinations(iterable, r)

The table below summarizes the most commonly used functions:

Function Category Description
count(start, step) Infinite Counts upward (or downward) forever from start by step.
cycle(iterable) Infinite Repeats the items of iterable forever, in order.
repeat(obj, times) Infinite Yields obj repeatedly; forever, or times times if given.
chain(*iterables) Terminating Yields items from each iterable in sequence, one after another.
islice(iterable, [start,] stop, [step]) Terminating Slices any iterable lazily, like list[start:stop:step] without building a list.
accumulate(iterable, func) Terminating Yields running totals (or running results of func) as it consumes the input.
groupby(iterable, key) Terminating Groups consecutive items that share the same key.
zip_longest(*iterables, fillvalue) Terminating Like zip() but pads shorter iterables with fillvalue.
product(*iterables, repeat) Combinatoric Cartesian product — every combination of one item from each iterable.
permutations(iterable, r) Combinatoric All ordered arrangements of length r (order matters).
combinations(iterable, r) Combinatoric All unordered selections of length r, no repeats (order doesn’t matter).

Examples

Example 1: Infinite iterators with islice

import itertools

# count() generates an infinite arithmetic sequence
counter = itertools.count(start=10, step=5)
first_five = list(itertools.islice(counter, 5))
print(first_five)

# cycle() repeats a sequence forever
colors = itertools.cycle(["red", "green", "blue"])
repeated = list(itertools.islice(colors, 7))
print(repeated)

# repeat() repeats a single value, optionally a fixed number of times
fours = list(itertools.repeat(4, 4))
print(fours)

Output:

[10, 15, 20, 25, 30]
['red', 'green', 'blue', 'red', 'green', 'blue', 'red']
[4, 4, 4, 4]

None of count(), cycle(), or repeat() would ever finish on their own — itertools.islice() is what makes them safe to use by cutting them off after a fixed number of items, exactly like a list slice but without ever materializing the infinite sequence.

Example 2: Grouping consecutive data with groupby

import itertools

words = ["apple", "avocado", "banana", "blueberry", "cherry", "clementine"]

grouped = itertools.groupby(words, key=lambda w: w[0])
for letter, group in grouped:
    print(letter, list(group))

Output:

a ['apple', 'avocado']
b ['banana', 'blueberry']
c ['cherry', 'clementine']

groupby() only groups consecutive items that share a key — it does not scan the whole iterable looking for matches. Because words is already sorted alphabetically, every word starting with the same letter is adjacent, so the grouping works as expected. We’ll see what happens when the data isn’t sorted in the Common Mistakes section.

Example 3: Combinatoric generators

import itertools

players = ["Ana", "Bo", "Cy", "Dee"]

print("All possible 2-player matchups:")
for pair in itertools.combinations(players, 2):
    print(pair)

print()
print("All possible 2-player batting orders:")
for order in itertools.permutations(players, 2):
    print(order)

sizes = ["S", "M", "L"]
colors = ["Red", "Blue"]
print()
print("All shirt size/color combos:")
for combo in itertools.product(sizes, colors):
    print(combo)

Output:

All possible 2-player matchups:
('Ana', 'Bo')
('Ana', 'Cy')
('Ana', 'Dee')
('Bo', 'Cy')
('Bo', 'Dee')
('Cy', 'Dee')

All possible 2-player batting orders:
('Ana', 'Bo')
('Ana', 'Cy')
('Ana', 'Dee')
('Bo', 'Ana')
('Bo', 'Cy')
('Bo', 'Dee')
('Cy', 'Ana')
('Cy', 'Bo')
('Cy', 'Dee')
('Dee', 'Ana')
('Dee', 'Bo')
('Dee', 'Cy')

All shirt size/color combos:
('S', 'Red')
('S', 'Blue')
('M', 'Red')
('M', 'Blue')
('L', 'Red')
('L', 'Blue')

combinations() treats ('Ana', 'Bo') and ('Bo', 'Ana') as the same selection, so it appears once. permutations() treats order as meaningful, so both appear. product() generates the full Cartesian product across multiple independent iterables — useful for generating every configuration of a set of options.

Under the Hood: Laziness in Action

Because every itertools function returns an iterator rather than a list, values are pulled one at a time only when something actually asks for the next item (via a for loop, next(), or a call to list()). You can observe this directly by wrapping an iterable in a function that logs each value as it is produced:

import itertools

def logged(iterable):
    for item in iterable:
        print(f"pulling {item}")
        yield item

pipeline = itertools.islice(logged(itertools.count()), 3)
result = list(pipeline)
print(result)

Output:

pulling 0
pulling 1
pulling 2
[0, 1, 2]

Even though itertools.count() could generate numbers forever, only three values are ever pulled, because islice() stops requesting new items from its upstream iterator once it has yielded three. This lazy, pull-based chaining is exactly how itertools stays memory-efficient: each stage in a pipeline only computes the next value when the stage after it asks for one, and nothing downstream of the point you stop consuming ever gets computed at all.

Common Mistakes

Mistake 1: Reusing an exhausted iterator

Iterators returned by itertools functions are single-use. Once you’ve consumed them (for example, by passing them to list()), a second pass yields nothing:

import itertools

pairs = itertools.combinations([1, 2, 3], 2)
first_pass = list(pairs)
second_pass = list(pairs)  # pairs is already exhausted!
print(first_pass)
print(second_pass)

Output:

[(1, 2), (1, 3), (2, 3)]
[]

The fix is to materialize the result once into a list (or tuple) and reuse that list, or to construct a fresh itertools.combinations(...) call each time you need to iterate again:

import itertools

pairs = list(itertools.combinations([1, 2, 3], 2))
first_pass = list(pairs)
second_pass = list(pairs)
print(first_pass)
print(second_pass)

Output:

[(1, 2), (1, 3), (2, 3)]
[(1, 2), (1, 3), (2, 3)]

Mistake 2: Calling groupby() on unsorted data

groupby() only merges keys that are adjacent. If the same key appears in multiple, non-consecutive positions, you’ll silently get separate groups instead of one combined group:

import itertools

data = [("a", 1), ("b", 2), ("a", 3)]
for key, group in itertools.groupby(data, key=lambda x: x[0]):
    print(key, list(group))

Output:

a [('a', 1)]
b [('b', 2)]
a [('a', 3)]

Notice "a" appears twice, as two separate one-item groups, because the second ("a", 3) isn’t adjacent to the first. The fix is to sort by the same key before grouping:

import itertools

data = [("a", 1), ("b", 2), ("a", 3)]
data_sorted = sorted(data, key=lambda x: x[0])
for key, group in itertools.groupby(data_sorted, key=lambda x: x[0]):
    print(key, list(group))

Output:

a [('a', 1), ('a', 3)]
b [('b', 2)]

Best Practices

  • Always pair infinite iterators (count(), cycle(), repeat() with no times) with islice() or an explicit break so your program can’t hang.
  • Sort an iterable by the same key you pass to groupby() before grouping, unless you already know the data is pre-grouped (e.g. it came from a database ORDER BY clause).
  • Convert an itertools iterator to a list or tuple if you need to iterate over the result more than once, check its length, or index into it.
  • Prefer itertools.chain(a, b, c) over a + b + c when concatenating iterables lazily, especially for generators that shouldn’t be fully materialized.
  • Use itertools.product(*iterables, repeat=n) instead of nested for loops when generating every combination of settings, test cases, or coordinates.
  • Be cautious with itertools.tee(): cloned iterators buffer items internally until every clone has consumed them, so tee-ing a huge or infinite iterator and consuming one clone far ahead of the other can use a surprising amount of memory.
  • On Python 3.10+, reach for the built-in itertools.pairwise() for consecutive-pair iteration instead of hand-rolling it with zip() and slicing.
  • Keep pipelines readable: chain a small number of itertools calls together, and fall back to a plain for loop or generator function if a pipeline starts needing more than two or three stages to explain.

Practice Exercises

  1. Write a function first_n_evens(n) that uses itertools.count() and itertools.islice() to return a list of the first n non-negative even numbers, without writing a manual loop that checks each number for evenness.
  2. Given logs = [("INFO", "start"), ("INFO", "loaded"), ("ERROR", "failed"), ("ERROR", "retry"), ("INFO", "done")], sort the list by log level and use itertools.groupby() to print how many messages exist for each level.
  3. Use itertools.product() to generate all 4-digit PIN codes where each digit is one of 0, 1, 2, or 3. How many total PINs are produced? (Hint: use repeat=4.)

Summary

  • itertools provides fast, memory-efficient, lazy iterator building blocks implemented in C.
  • Infinite iterators (count, cycle, repeat) must be limited manually, typically with islice().
  • Terminating iterators like chain(), accumulate(), and groupby() transform or combine finite iterables.
  • groupby() only merges adjacent matching keys — sort your data first.
  • Combinatoric generators (product, permutations, combinations) replace nested loops for generating arrangements and selections.
  • All itertools iterators are single-use; materialize them with list() if you need to reuse the results.
  • Because everything is lazy, values are only computed when explicitly requested downstream, which keeps pipelines memory-efficient even over huge or infinite inputs.