This Python FAQ, addresses a problem new Python programmers always run into: lambda in list comprehension (in essence, in a for
loop). However, it is poorly explained.
I tried to figure out a way to explain it by breaking down and refactoring the code just like what you would do in a math proof problem. Tell me whether this "proof" makes sense or whether "proof" makes better explanation.
Credit: Thanks to YouTube channel mCoding
for showing the internals of Python list comprehensions!
increment_by_i = [lambda x: x + i for i in range(5)]
is equalivent to:
increment_by_i = list(lambda x: x + i for i in range(5))
, where
lambda(x: x + i for i in range(10))
is a generator expression, which is equalivent to:
def lambda_generator(): # Define a generator function
for i in range(5):
yield lambda x: x + i
generator = lambda_generator() # Then call it
By the property of lambda
in Python, this is eqivalent to:
def lambda_generator():
for i in range(5):
def lambda_function(x):
return x + i
yield lambda_function
generator = lambda_generator()
If you run this code with the inspcect.getclosurevars
function, you can already see from the printed output that during the definition of lambda_generator()
, the i
variable have already been iterated from 0 to 4. So lambda_generator()
function becomes a function that adds 4 to its inputs.
from inspect import getclosurevars
def lambda_generator():
for i in range(5):
def lambda_function(x):
return x + i
yield lambda_function
print(getclosurevars(lambda_function))
generator = lambda_generator()
list(generator)[3](4)
# Output:
# ClosureVars(nonlocals={'i': 0}, globals={}, builtins={}, unbound=set())
# ClosureVars(nonlocals={'i': 1}, globals={}, builtins={}, unbound=set())
# ClosureVars(nonlocals={'i': 2}, globals={}, builtins={}, unbound=set())
# ClosureVars(nonlocals={'i': 3}, globals={}, builtins={}, unbound=set())
# ClosureVars(nonlocals={'i': 4}, globals={}, builtins={}, unbound=set())
# 8
(Lemma: A small example demonstrating the underlying mechanism of for
loops in Python)
"""
for i in range(5):
print(i)
is equivalent to
"""
itr = iter(range(5)) # Apply `iter()` to range object to get iterator `itr`
while True:
try:
print(next(itr))
except StopIteration:
break
By the underlying mechanism of for
loops (create an iterator with iter()
function, then repeatedly apply next()
to the iterator until StopIteration
) in Python, lambda_generator()
is equalivent to:
def lambda_generator():
itr = iter(range(5)) # Apply `iter()` to range object to get iterator `itr`
while True:
try:
def lambda_function(x):
return x + next(itr) # `next(iter)` is the `i`
print(lambda_function(0)) # Set `x` to be 0, so that the printed result = next(itr)
except StopIteration:
yield lambda_function
break # This break may not be necessary though
In every while
loop (before StopIteration
), next(itr)
is called, modifying the itr
iterator outside the while
loop (inside the lambda_generator()
function). Therefore, it is OK to refactor the itr
iterator inside the while
loop:
def lambda_generator():
def lambda_function(x):
itr = iter(range(4))
result = None
while True:
try:
result = x + next(itr)
except StopIteration:
break
return result
yield lambda_function
Now we can easily see that by the time lambda_function()
is yielded to lambda_generator()
, lambda_function()
have already become a function that adds 4 to its input.
def lambda_function(x):
itr = iter(range(5))
result = None
while True:
try:
result = x + next(itr)
except StopIteration:
break
return result
print(lambda_function(0)) # Set x = 0, so that the printed result = the final next(iter) value
# Output: 4
Top comments (0)