[FIXED] List comprehension rebinds names even after scope of comprehension. Is this right?

Issue

Comprehensions show unusual interactions with scoping. Is this the expected behavior?

x = "original value"
squares = [x**2 for x in range(5)]
print(x)  # Prints 4 in Python 2!

At the risk of whining, this is a brutal source of errors. As I write new code, I just occasionally find very weird errors due to rebinding -- even now that I know it's a problem. I need to make a rule like "always preface temp vars in list comprehensions with underscore", but even that's not foolproof. The fact that there's this random time-bomb waiting kind of negates all the nice "ease of use" of list comprehensions.

Solution

List comprehensions leak the loop control variable in Python 2 but not in Python 3. Here's Guido van Rossum (creator of Python) explaining the history behind this:

We also made another change in Python 3, to improve equivalence between list comprehensions and generator expressions. In Python 2, the list comprehension "leaks" the loop control variable into the surrounding scope:

x = 'before'
a = [x for x in 1, 2, 3]
print x # this prints '3', not 'before'

This was an artifact of the original implementation of list comprehensions; it was one of Python's "dirty little secrets" for years. It started out as an intentional compromise to make list comprehensions blindingly fast, and while it was not a common pitfall for beginners, it definitely stung people occasionally. For generator expressions we could not do this. Generator expressions are implemented using generators, whose execution requires a separate execution frame. Thus, generator expressions (especially if they iterate over a short sequence) were less efficient than list comprehensions.

However, in Python 3, we decided to fix the "dirty little secret" of list comprehensions by using the same implementation strategy as for generator expressions. Thus, in Python 3, the above example (after modification to use print(x) :-) will print 'before', proving that the 'x' in the list comprehension temporarily shadows but does not override the 'x' in the surrounding scope.

Answered By - Steven Rumbalski