[FIXED] outer async context manager finalized before inner async generator

Issue

Given the following minimal example:

@asynccontextmanager
async def async_context():
    try:
        yield
    finally:
        await asyncio.sleep(1)
        print('finalize context')


async def async_gen():
    try:
        yield
    finally:
        await asyncio.sleep(2)
        # will never be called if timeout is larger than in async_context
        print('finalize gen')


async def main():
    async with async_context():
        async for _ in async_gen():
            break


if __name__ == "__main__":
    asyncio.run(main())

I'm breaking while iterating over the async generator and I want the finally block to complete before my async context manager finally block runs. In this example "finalize gen" will never be printed because the program exits before that happens.

Note that I intentionally chose a timeout of 2 in the generators finally block so the context managers finally has a chance to run before. If I chose 1 for both timeouts both messages will be printed.

Is this kind of a race condition? I expected all finally blocks to complete before the program finishes.

How can I prevent the context mangers finally block to run before the generators finally block has completed?

For context:

I use playwright to control a chromium browser. The outer context manager provides a page that it closes in the finally block.

I'm using python 3.9.0.

Try this example: https://repl.it/@trixn86/AsyncGeneratorRaceCondition

Solution

The async context manager doesn't know anything about the asynchronous generator. Nothing in main knows about the asynchronous generator after you break, in fact. You've given yourself no way to wait for the generator's finalization.

If you want to wait for the generator to close, you need to handle closure explicitly:

async def main():
    async with async_context():
        gen = async_gen()
        try:
            async for _ in gen:
                break
        finally:
            await gen.aclose()

In Python 3.10, you'll be able to use contextlib.aclosing instead of the try/finally:

async def main():
    async with async_context():
        gen = async_gen()
        async with contextlib.aclosing(gen):
            async for _ in gen:
                break

Answered By - user2357112 supports Monica