# Scaling Python/Django application with Apache and mod\_wsgi

DevFeed: [Scaling Python/Django application with Apache and mod\_wsgi](<https://devfeed.tech/articles/scaling-python-django-application-with-apache-and-mod-wsgi-19986.md>)

Original publisher: [Read original article](<http://engineering.hackerearth.com/2013/11/21/scaling-python-django-application-apache-mod_wsgi/>)

Published: 2013-11-21T00:00:00Z

Content type: article

Language: en

Sources: [HackerEarth](<https://devfeed.tech/sources/hackerearth.md>)

Topics: [Django](<https://devfeed.tech/topics/django.md>), [Python](<https://devfeed.tech/topics/python.md>), [web applications](<https://devfeed.tech/topics/web-applications.md>), [configuration](<https://devfeed.tech/topics/configuration.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [Testing](<https://devfeed.tech/topics/testing.md>), [Server](<https://devfeed.tech/topics/server.md>)

Tags: [apache](<https://devfeed.tech/tags/apache.md>), [applications](<https://devfeed.tech/tags/applications.md>), [concurrent](<https://devfeed.tech/tags/concurrent.md>), [configuration](<https://devfeed.tech/tags/configuration.md>), [django](<https://devfeed.tech/tags/django.md>), [hosting](<https://devfeed.tech/tags/hosting.md>), [latency](<https://devfeed.tech/tags/latency.md>), [load](<https://devfeed.tech/tags/load.md>), [memory](<https://devfeed.tech/tags/memory.md>), [modules](<https://devfeed.tech/tags/modules.md>), [production](<https://devfeed.tech/tags/production.md>), [python](<https://devfeed.tech/tags/python.md>), [scale](<https://devfeed.tech/tags/scale.md>), [server](<https://devfeed.tech/tags/server.md>), [servers](<https://devfeed.tech/tags/servers.md>), [testing](<https://devfeed.tech/tags/testing.md>), [web-applications](<https://devfeed.tech/tags/web-applications.md>)

## AI overview

This article explains how HackerEarth improved the scalability of its Python/Django application hosted with Apache and mod_wsgi. It attributes performance problems primarily to application bottlenecks, database latency, poor Apache configuration, excessive modules, and the default prefork multiprocessing model. It describes replica-based experiments and load testing with thousands of concurrent connections to measure configuration improvements.

## Source excerpt

HackerEarth is primarily based on Python & Django. And we use Apache with mod_wsgi for hosting the application. There is a general complaint that Apache sucks when it comes to hosting Python web applications. It's said that it's slow, bloated, uses lots of memory and doesn't perform very well. It's also said that it doesn't handle a high number of concurrent requests. All that is true if you are not running the Python application in the right way. If configured properly, Apache works fantastically and is usually never the reason of slowness. That is almost always due to the application bottlenecks and database latency. Now I am ashamed to admit that we have been running HackerEarth frontend servers for a long time under bad configuration of Apache and mod_wsgi. This came up when we started load testing our servers with thousands of concurrent connections. Also, we previously used to see the memory usage going on a rocket trajectory when faced with sudden spike in traffic, which used to force us to scale up more than that was actually required. There are couple reasons for excessive memory usage when running Python applications in Apache. First of all, they are very heavy to start with. The multiprocessing module (apache2-mpm-prefork) that comes by default with apache makes it even worse. Their poor configuration is just waiting for disaster again. And most importantly, there are tons of apache modules installed and loaded into memory, while most of them are never going to be used. If Apache is setup properly keeping the Python web application and the machine resource contraints into account, Apache is fast and reliable. I will explain further what we did at HackerEarth. But before that, I will present some real data from the experiments that we did. We created an exact replica of one of our production machine - which directly handles the requests and is running Apache server. Then, we tweaked apached configuration one by one and recorded the improvement. Everytime w