Making Requests Non-blocking in Tornado

Tornado is one of the most popular web framework for Python, which is based on a single thread IO loop (aka event loop). You can handle high concurrency with optimal performance. However, Tornado is single threaded (in its common usage, although in supports multiple threads in advanced configurations), therefore any "blocking" task will block the whole server. This means that a blocking task will not allow the framework to pick the next task waiting to be processed.

For example, this is a wrong way of using IOLoop:

class MainHandler(tornado.web.RequestHandler): def get(self): self.write('[MainHandler] Hello, world') class ComplexHandler(tornado.web.RequestHandler): def get(self): result = yield self.get_complex_result() self.write('[ComplexHandler] Result = %d.\n' % result) def get_complex_result(self): time.sleep(5) # Assume the complex calculation takes 5 seconds return 100 # Assume the final result is 100

Note that get_complex_result() is called correctly, but it is blocked by time.sleep(5), which will prevent the execution of the following tasks (such as a second request to the same function). Only when the first request is finished, the second request will be handled by IOLoop.

The solution to the blocking problem is to use asynchronous or coroutine annotation.

class MainHandler(tornado.web.RequestHandler): @tornado.web.asynchronous def get(self): self.write('[MainHandler] Hello, world') self.finish() # finish() must be invoked to close connection class ComplexHandler(tornado.web.RequestHandler): @tornado.gen.coroutine def get(self): yield tornado.gen.sleep(10) self.write('[ComplexHandler] Hello, world')

In the above code, the Tornado application can handle requests in MainHandler and ComplexHandler simultaneously. However, in some situations, some packages do not support asynchronous and will still block requests. ThreadPoolExecutor are incorporated to tackle this problem.

class ComplexHandler(tornado.web.RequestHandler): executor = concurrent.futures.ThreadPoolExecutor(5) @tornado.gen.coroutine def get(self): result = yield self.get_complex_result() self.write('The final result is %d.\n' % result) @tornado.concurrent.run_on_executor def get_complex_result(self): print('Before Sleep.') time.sleep(5) # Assume the complex calculation takes 5 seconds print('After Sleep.') return 100 # Assume the final result is 100

In Python 2, you need to install a package called futures.

ThreadPoolExecutor is a high level encapsulation of threading in the standard library, making functions run asynchronously with the help of threads.

References

  • https://hexiangyu.me/posts/15
  • https://gist.github.com/lbolla/3826189
  • https://www.peterbe.com/plog/worrying-about-io-blocking
  • http://www.tornadoweb.org/en/stable/concurrent.html
  • http://www.tornadoweb.org/en/stable/guide/coroutines.html
Contact Us
  • Tencent AI Lab, Shenzhen, China
  • cshzxie [at] gmail [dot] com