Why Is Reading And Calling An Api From A File Slower Using Python Async Than Synchronously?

I have a large file, with a JSON record on each line. I'm writing a script to upload a subset of these records to CouchDB via the API, and experimenting with different approaches t

Solution 1:

your code uses async but it does the work synchronously and in this case it will be slower than the sync approach. Asyc won't speed up the execution if not constructed/used effectively.

You can create 2 coroutines and make them run in parallel.. perhaps that speeds up the operation.

Example:

#!/usr/bin/env python3import asyncio


asyncdefupload(event, queue):
    # This logic is not so correct when it comes to shutdown,# but gives the ideawhilenot event.is_set():
        record = await queue.get()
        print(f'uploading record : {record}')
    returnasyncdefread(event, queue):
    # dummy logic : instead read here and populate the queue.for i inrange(1, 10):
        await queue.put(i)
    # Initiate shutdown..
    event.set()


asyncdefmain():
    event = asyncio.Event()
    queue = asyncio.Queue()

    uploader = asyncio.create_task(upload(event, queue))
    reader = asyncio.create_task(read(event, queue))
    tasks = [uploader, reader]

    await asyncio.gather(*tasks)


if __name__ == '__main__':
    asyncio.run(main())