Python Iterators

An iterator is an object that produces the elements of a collection one at a time, on demand, without exposing how those elements are actually stored. Iterators are the engine that powers every for loop in Python, along with functions like sum(), sorted(), and list() — anywhere you see something being "looped over", an iterator is quietly doing the work behind the scenes. Once you understand the iterator protocol, generators, comprehensions, and the internals of for loops stop feeling like separate topics and start looking like one consistent design.

Overview: How Iteration Works in Python

Python draws a sharp distinction between two related but different concepts: iterables and iterators.

An iterable is any object you can get an iterator from — lists, tuples, strings, dictionaries, sets, files, and range objects are all iterables. Formally, an object is iterable if it defines an __iter__() method that returns an iterator.

An iterator is the object that actually does the stepping. It is a stateful object that remembers where it is in a sequence, and produces the next value each time you ask for one. Formally, an object is an iterator if it defines both:

• __iter__() — which simply returns self, so that iterators are themselves iterable.
• __next__() — which returns the next value, or raises the built-in StopIteration exception when there are no more values left.

This pair of rules is called the iterator protocol. Anything that follows it can be used in a for loop, unpacked, passed to list(), or consumed by sum() — Python doesn’t need to know what kind of object it is, only that it obeys the protocol. This is Python’s version of duck typing applied to looping.

Here is the key mental model: calling the built-in function iter(obj) asks the object to hand back an iterator (by calling obj.__iter__() internally). Calling the built-in function next(it) asks that iterator to hand back its next value (by calling it.__next__() internally). A list is an iterable, not an iterator — you cannot call next() directly on a list, you first have to obtain an iterator from it with iter(). That iterator is a separate object with its own internal position counter, which is exactly why you can have two independent loops over the same list running at once without interfering with each other.

When Python evaluates a for item in collection: statement, it desugars, roughly, into repeatedly calling iter() once and then next() in a loop, catching StopIteration to know when to stop — you’ll see this made explicit in the "Under the Hood" section below.

Syntax

There is no special syntax for using iterators — they are used through two built-in functions and implemented through two dunder (double-underscore) methods.

iterator = iter(iterable)   # get an iterator from an iterable
value = next(iterator)      # get the next value, or raise StopIteration
value = next(iterator, default)  # get the next value, or return 'default' instead of raising

Piece	Meaning
iter(x)	Calls x.__iter__(). Returns an iterator object positioned before the first element.
next(it)	Calls it.__next__(). Returns the next value and advances the iterator; raises StopIteration when exhausted.
next(it, default)	Same as above, but returns default instead of raising when the iterator is exhausted.
__iter__(self)	Method a class implements to become iterable; for an iterator class it should simply return self.
__next__(self)	Method a class implements to become an iterator; computes and returns the next value or raises StopIteration.

Examples

Example 1: Manually driving an iterator

This example bypasses the for loop entirely so you can see the protocol in its raw form.

numbers = [10, 20, 30]
iterator = iter(numbers)

print(next(iterator))
print(next(iterator))
print(next(iterator))

try:
    print(next(iterator))
except StopIteration:
    print("No more items")

Output:

10
20
30
No more items

Each call to next() advances the iterator’s internal position by one. Once all three elements are consumed, a fourth call raises StopIteration, which we catch explicitly here to show the mechanism — in a normal for loop, Python catches this exception for you automatically.

Example 2: Writing a custom iterator class

Any class that implements __iter__ (returning self) and __next__ can be used directly in a for loop.

class Countdown:
    def __init__(self, start: int):
        self.current = start

    def __iter__(self):
        return self

    def __next__(self):
        if self.current <= 0:
            raise StopIteration
        value = self.current
        self.current -= 1
        return value

for number in Countdown(5):
    print(number)

Output:

5
4
3
2
1

Countdown is both an iterable (because it has __iter__) and an iterator (because it has __next__ and its __iter__ just returns itself). Each call to __next__ mutates self.current and returns the current value, until the guard condition raises StopIteration to end the loop.

Example 3: Iterable vs. iterator — supporting multiple passes

Because Countdown above is its own iterator, it can only be looped over once — after it's exhausted, its self.current stays at 0 forever. A well-designed iterable should usually hand back a fresh iterator every time, so it can be looped over repeatedly, just like a list can.

class EvenNumbers:
    def __init__(self, limit: int):
        self.limit = limit

    def __iter__(self):
        return iter(range(0, self.limit, 2))

evens = EvenNumbers(10)
print(list(evens))
print(list(evens))

Output:

[0, 2, 4, 6, 8]
[0, 2, 4, 6, 8]

Here EvenNumbers is an iterable but deliberately not an iterator itself — its __iter__ method builds and returns a brand-new range iterator each time it's called. That's why list(evens) can be called twice and produce the same result both times, mirroring how built-in containers like lists and tuples behave.

How It Works Step by Step (Under the Hood)

A for loop like for word in words: is syntactic sugar. Written out longhand using the iterator protocol, it looks like this:

words = ["alpha", "beta", "gamma"]
iterator = iter(words)
while True:
    try:
        word = next(iterator)
    except StopIteration:
        break
    print(word.upper())

Output:

ALPHA
BETA
GAMMA

Step by step, this is exactly what the CPython interpreter does whenever it executes a for statement:

1. It calls iter(words) once, up front, to obtain an iterator object (for a list, this is an internal list_iterator object — you'd see <list_iterator object at 0x...> if you printed it).
2. It calls next() on that iterator repeatedly, once per loop iteration, binding the result to the loop variable (word).
3. When next() raises StopIteration, the interpreter catches it internally and exits the loop cleanly — no traceback is ever shown to you, and no exception escapes the loop.

This is also why an object only needs one iterator obtained at the start, not a fresh one per element — the same iterator object is reused and mutated across the whole loop, which is precisely why iterators are described as stateful: the object itself remembers "where it is".

Common Mistakes

Mistake 1: Confusing an iterable with an iterator

A very common beginner error is calling next() directly on something like a list, forgetting that lists are iterables, not iterators.

values = [1, 2, 3]
try:
    print(next(values))
except TypeError as error:
    print(f"Error: {error}")

Output:

Error: 'list' object is not an iterator

The fix is to call iter() first to obtain an actual iterator object: next(iter(values)). Remember: containers (list, tuple, dict, set, str) are iterables that produce iterators — they are not iterators themselves.

Mistake 2: Exhausting an iterator without realizing it

Unlike a list, an iterator can only be consumed once. If you pass it to one function that consumes it, there's nothing left for the next function.

numbers = [1, 2, 3, 4]
squared_iterator = (n ** 2 for n in numbers)

total = sum(squared_iterator)
remaining = list(squared_iterator)

print(total)
print(remaining)

Output:

30
[]

The generator expression squared_iterator is an iterator. sum() fully drains it to compute 30, so by the time list() runs there is nothing left, producing an empty list instead of an error — this is a silent bug, not a crash, which makes it especially easy to miss. If you need to use the values more than once, materialize them into a list first, e.g. values = list(squared_iterator), and then reuse values.

Best Practices

• Prefer a plain for loop over manual iter()/next() calls unless you specifically need fine-grained control (e.g. reading a few items, then doing other work, then reading more).
• When writing a class meant to be looped over multiple times (like a container), make __iter__ return a new iterator each call rather than self, so multiple independent loops can run over it safely.
• Only make __iter__ return self when the object is genuinely meant to be a one-shot, single-pass iterator (like Countdown in Example 2).
• Use next(iterator, default) instead of a try/except StopIteration block when a sensible fallback value exists — it's shorter and just as clear.
• Don't reuse a consumed iterator or generator expecting fresh values — if you need the data more than once, convert it to a list or tuple with list(it) right after creating it.
• For most custom iteration needs, prefer writing a generator function (using yield) over a hand-written class with __iter__/__next__ — it does the same job with far less boilerplate, since Python builds the iterator protocol for you automatically.
• Reach for the itertools module (e.g. itertools.islice, itertools.chain) before hand-rolling iterator logic for common patterns like slicing, chaining, or infinite counting.

Practice Exercises

1. Write a class Repeater whose constructor takes a value and a count, and whose instances are iterators that yield that value count times, then stop. For example, iterating over Repeater("hi", 3) should print hi three times.
2. Given data = iter([5, 10, 15]), use next() with a default value of None four times in a row and print each result. Confirm that the fourth call returns None instead of raising StopIteration.
3. Write an iterable class Squares that, given a limit, can be iterated over multiple times (like Example 3) to yield the squares of 0, 1, 2, ..., limit - 1. Verify by calling list(Squares(4)) twice and confirming both calls return [0, 1, 4, 9].

Summary

• An iterable is anything with __iter__(); an iterator additionally has __next__() and raises StopIteration when exhausted.
• iter(obj) obtains an iterator; next(it) pulls the next value from it.
• A for loop is syntactic sugar for calling iter() once, then next() repeatedly, catching StopIteration automatically.
• Iterators are stateful and single-use — once exhausted, they stay exhausted; well-designed iterables return a fresh iterator on each call to __iter__ so they can be looped over repeatedly.
• Custom iterator classes implement __iter__ (returning self) and __next__; for most real code, a generator function is a simpler alternative to writing this boilerplate by hand.