Description
Using Coroutines to Create Efficient, High-Concurrency Web Applications
Presented by Matt Spitz
Creating high-concurrency python web applications is inherently difficult for a variety of reasons. In this talk, I'll discuss the various iterations of application server paradigms we've used at meebo, the advantages/disadvantages of each approach, and why we've settled on a coroutine-based WSGI setup to handle our high-concurrency web applications going forward.
Abstract
There are a number of ways in which to create a web application in python. Some examples include a straight-up CGI scripts that run anew with each request, preforked Apache workers that each handle multiple requests, and using an asynchronous web framework like Twisted.
At meebo, we've settled on using gunicorn, a lightweight WSGI server, which supports gevent, a coroutine-based network library for python. Gevent monkeypatches python's system modules to make network requests asynchronous using an event loop based on libevent. This trick allows the developer to use a simple blocking CGI as a non-blocking web application that can handle many concurrent requests.
I'll discuss our iteration process through these approaches to building web applications, why we ended up choosing gunicorn+gevent, the challenges this new framework presents, and how we've dealt with them.