[FIXED] How does Future's exec_ work?

Issue

How does this code, which forward-ports exec, work?

# Implementation of exec_ is from ``six``:
if PY3:
    import builtins
    exec_ = getattr(builtins, "exec")
else:
    def exec_(code, globs=None, locs=None):
        """Execute code in a namespace."""
        if globs is None:
            frame = sys._getframe(1)
            globs = frame.f_globals
            if locs is None:
                locs = frame.f_locals
            del frame
        elif locs is None:
            locs = globs
        exec("""exec code in globs, locs""")

The snippet is copied from Python Future because I was too lazy to find the original from Six which Martijn Pieters linked to.

Answers specific to Six's (identical-bar reraise) version are welcome too.

Solution

For Python 3:

if PY3:

This is relatively straighforward:

    import builtins
    exec_ = getattr(builtins, "exec")

The reason to use getattr is that on Python 2 exec is a statment and you want to avoid:

>>> builtins.exec
  File "<stdin>", line 1
    builtins.exec
                ^
SyntaxError: invalid syntax

Using a string to do lookup gets around the problem of exec not being a valid identifier.

If Python 2:

else:

We want to define exec_ to be like exec on Python 3, which means it looks like so:

    def exec_(code, globs=None, locs=None):
        """Execute code in a namespace."""

Let's quickly check the docs:

help(exec)
#>>> Help on built-in function exec in module builtins:
#>>>
#>>> exec(...)
#>>>     exec(object[, globals[, locals]])
#>>>     
#>>>     Read and execute code from an object, which can be a string or a code
#>>>     object.
#>>>     The globals and locals are dictionaries, defaulting to the current
#>>>     globals and locals.  If only globals is given, locals defaults to it.
#>>>

This should help explain the next part:

        if globs is None:
            frame = sys._getframe(1)
            globs = frame.f_globals

If globs is None, we want it to default to the globals of the caller's scope. This is actually quite involved.

First we get the outer frame:

import sys
help(sys._getframe)
#>>> Help on built-in function _getframe in module sys:
#>>>
#>>> _getframe(...)
#>>>     _getframe([depth]) -> frameobject
#>>>     
#>>>     Return a frame object from the call stack.  If optional integer depth is
#>>>     given, return the frame object that many calls below the top of the stack.
#>>>     If that is deeper than the call stack, ValueError is raised.  The default
#>>>     for depth is zero, returning the frame at the top of the call stack.
#>>>     
#>>>     This function should be used for internal and specialized
#>>>     purposes only.
#>>>

sys._getframe is just the caller's scope, then. A frame is the environment in which function execution happens, and stores several interesting properties such as f_globals - the globals from the point of view of that function.

Then if does:

            if locs is None:
                locs = frame.f_locals

to default locs to the locals.

The frame is deleted to prevent this from keeping things alive unnecessarily during the exec call.

            del frame

Penultimately,

        elif locs is None:
            locs = globs

which is to fufull the "If only globals is given, locals defaults to it." part.

Then the most fun part:

        exec("""exec code in globs, locs""")

Running just exec code in globs, locs would mean that when compiled on Python 3, this would error as exec is no longer a statement. Thus, an outer exec("""exec code in globs, locs""") call is made.

On Python 3, this will never be run but will compile. On Python 2, this will capture the locals globs and locs, allowing it to run exec code in globs, locs.

---

Please do look at Antti Haapala's answer, though as these shennanigans are actually made somewhat irrelevant by what he points out, including some information that Martijn Pieters brought forward.

Answered By - Veedrac