Explaining Iterable vs Iterator in Python

The aim of this page📝is to demonstrate the dynamics of the 2 iteration protocols:

1. iterable
2. iterator

1. BUT FIRST (TO ADD TO CONFUSINGLY SIMILAR WORDS), LET'S ADDRESS ITERATION

• iteration - of course - is taking items one by one from a source and doing something with each in turn
• in python, this is commonly used in
  • a) for/while loops and
  • b) comprehensions
• by default - these structures iterate over the whole structure
• sometimes, however, a more fine-grained control could be needed - like in generators
• for this, there are 2 important concepts/protocols, on top of which much of Python is constructed:
• a) iterable objects
• b) iterator objects
• both are reflected in standard python protocols
• this is not something extra: actually, for/while loops and comprehensions are built directly upon these lower-level elements of iteration protocols

2. ITER() METHOD CREATES AN ITERATOR FROM AN ITERABLE

• iterable object (collection or stream of objects) is any object that can be passed into the built-in iter() function
• once passed the built-in iter() function and which returns an iterator object of a passed type, i.e. a string iterator is created with

>>> example_iterator = iter('abc')
>>> example_iterator
<str_iterator object at 0x063DCE38>

• note that iterator is an implicit sequence object providing sequential (not random!) access to an underlying sequential dataset
• for example range object itself is not an iterator
• iterator does not allow the access to arbitrary element of the underlying series
• they provide access only to the next element of the series
• they provide sequential access

<!-- THIS IS NOT AN ITERATOR -->
>>> r = range(10)[5]
>>> r
5

3. NEXT() FUNCTION RETURNS THE NEXT VALUE FROM AN ITERATOR

• the built-in next() requires an iterator object - and it returns the next value in the iteration of a collection
• iterator consists of 2 components:
• mechanism for retrieving the next element of a collection
• mechanism for signalling the end of the series

In programming languages with built-in object systems, this abstraction typically corresponds to a particular interface that can be implemented by classes

• next() allows to consider each item in turn / on request - not the whole series from beginning to an end
• there are 2 messages iterator interface includes
  • next → query for the next element
  • iter → return the iterator
• constraint: iterator can be iterated over once

4. CLASSROOM EXAMPLE - FROM ITERABLE TO ITERATOR TO STOPITERATION EXCEPTION

• Python, liberally, raises an exception of the type StopIteration

iterable = ['Spring', 'Summer', 'Autumn', 'Winter']
iterator = iter(iterable)
next(iterator)
# >>> 'Spring'
next(iterator)
# >>> 'Summer'a  r
next(iterator)
# >>> 'Autumn'
next(iterator)
# >>> 'Winter'
next(iterator)
# >>> Traceback (most recent call last):
# >>> File "<stdin>", line 1, in <module>
# StopIteration

5. REAL-LIFE EXAMPLE - UNIT TESTING MULTIPLE COMMAND LINE INPUTS

1. define/get an iterable object such as a list ["20.01", "y"]
2. pass an iterable object into iter() → create an iterator object
3. pass an iterator object into a next() to yield the next value of the list each time the input function is called in the code

def test_pubDoc(monkeypatch):
    inputs = iter(["20.01", "y"])
    monkeypatch.setattr("builtins.input", lambda _: next(inputs))
    #...

• the first time input() is encountered, the "20.01" value is passed,
• the second time it is "y"
• the third time it would be an exception

6. LINKS

• https://mypy.readthedocs.io/en/stable/protocols.html#iteration-protocols
• 5.2 Implicit Sequences - SICP in Python