[FIXED] Performance of asyncio

Issue

I'm trying to get familiar with asyncio, so I've decided to write a database client. However, performance exactly matches synchronous code. I'm sure this is my misunderstanding of a concept. Could someone explain what am I doing wriong?

Please see example of code below:

class Connection:
    def __init__(self, reader, writer, loop):
        self.futures = deque()

        # ...

        self.reader_task = asyncio.async(self.recv_data(), loop=self.loop)

    @asyncio.coroutine
    def recv_data(self):
        while 1:
            try:
                response = yield from self.reader.readexactly(4)
                size, = struct.unpack('I', response)
                response = yield from self.reader.readexactly(size)

                # ...                

                future = self.futures.popleft()

                if not future.cancelled():
                    future.set_result(response)

            except Exception:
                break

    def send_data(self, data):
        future = asyncio.Future(loop=self.loop)
        self.futures.append(future)

        self.writer.write(data)

        return future


loop = asyncio.get_event_loop()


@asyncio.coroutine
def benchmark():
    connection = yield from create_connection(loop=loop, ...)

    for i in range(10000):
        yield from connection.send_data(...)


s = time.monotonic()

loop.run_until_complete(benchmark())

e = time.monotonic()
print('Requests per second:', int(10000 / (e - s)))

Thanks in advance.

Solution

You've made a mistake in the way you're calling send_data. Right now, you've got this:

@asyncio.coroutine
def benchmark():
    connection = yield from create_connection(loop=loop, ...)

    for i in range(10000):
        yield from connection.send_data(...)

By using yield from inside the for loop, you're waiting for the future you're returning from send_data to yield a result before moving on to the next call. This makes your program basically synchronous. You want to make all your calls to send_data, and then wait for results:

@asyncio.coroutine
def benchmark():
    connection = yield from create_connection(loop=loop, ...)
    yield from asyncio.wait([connection.send_data(..) for _ in range(10000)])

Answered By - dano