Design Of Asynchronous Request And Blocking Processing Using Tornado

I'm trying to implement a Python app that uses async functions to receive and emit messages using NATS, using a client based on Tornado. Once a message is received, a blocking func

Solution 1:

General rule (credits to NYKevin) is:

• multiprocessing for CPU- and GPU-bound computations.
• Event-driven stuff for non-blocking I/O (which should be preferred over blocking I/O where possible, since it scales much more effectively).
• Threads for blocking I/O (you can also use multiprocessing, but the per-process overhead probably isn't worth it).

ThreadPoolExecutor for IO, ProcessPoolExecutor for CPU. Both have internal queue, both scale to at most specified max_workers. More info about concurrent executors in docs.

So answer are:

1. Reimplementing pool is an overhead. Thread or Process depends on what you plan to do.
2. while True is not blocking if you have e.g. some yielded async calls (even yield gen.sleep(0.01)), it gives back control to ioloop
3. qsize() is the right to call, but since I have not run/debug this and I would take a different approach (existing pool), it is hard to find a problem here.