# Programming challenges, uptime, and mistakes in 2013

DevFeed: [Programming challenges, uptime, and mistakes in 2013](<https://devfeed.tech/articles/programming-challenges-uptime-and-mistakes-in-2013-19988.md>)

Original publisher: [Read original article](<http://engineering.hackerearth.com/2014/01/22/programming-challenges-uptime-mistakes/>)

Published: 2014-01-22T00:00:00Z

Content type: opinion

Language: en

Sources: [HackerEarth](<https://devfeed.tech/sources/hackerearth.md>)

Topics: [Programming](<https://devfeed.tech/topics/programming.md>), [Server](<https://devfeed.tech/topics/server.md>), [Code](<https://devfeed.tech/topics/code.md>), [Databases](<https://devfeed.tech/topics/databases.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [Front end](<https://devfeed.tech/topics/frontend.md>), [Processes](<https://devfeed.tech/topics/processes.md>), [Django](<https://devfeed.tech/topics/django.md>)

Tags: [browser](<https://devfeed.tech/tags/browser.md>), [code](<https://devfeed.tech/tags/code.md>), [database](<https://devfeed.tech/tags/database.md>), [frontend](<https://devfeed.tech/tags/frontend.md>), [latency](<https://devfeed.tech/tags/latency.md>), [process](<https://devfeed.tech/tags/process.md>), [programming](<https://devfeed.tech/tags/programming.md>), [server](<https://devfeed.tech/tags/server.md>), [servers](<https://devfeed.tech/tags/servers.md>), [uptime](<https://devfeed.tech/tags/uptime.md>)

## AI overview

HackerEarth recounts the operational challenges of hosting more than a thousand programming contests in 2013. The article describes traffic bursts, scaling limitations, the struggle to maintain uptime, and engineering changes including an asynchronous code-checker queue, a Tornado realtime server, database sharding, and database routers to reduce latency.

## Source excerpt

HackerEarth hosted more than thousand contests in the year 2013 alone. Out of them, there were more than two dozen public programming contests by HackerEarth itself. They include our monthly challenges and hiring challenges. There were over 200 internal and public contests by colleges in the previous year. They include IIT Delhi, IIT Guwahati, IIT Ropar, NIT Warangal, IIIT Jabalpur, NIT Raipur, NIT Calicut, BITS Pilani and many others. And we have been able to do that without any sweat. But sometimes, we made mistakes too, most of them in the early half of 2013. ####Mayhem To tell you the truth, in the beginning it was chaotic, mayhem and scary. We would have to monitor that everything was working right. Sometimes, we would give in everything just to keep the site up and running. The problem of scaling always takes a toll on you. And that too when you want to build a word-class product. And particularly for a platform like ours where the concept of putting more servers on demand(auto-scaling) fails due to sudden burst in traffic, giving no time to bring more servers in action. Below is a request graph from production server on an usual day. It's important to realize that nothing scales automatically. 100% uptime is a constant struggle. But we were ready to roll up our sleeves and move towards that. And in later half of 2013, things have moved ahead at a really amazing pace. On a related note, whenever I read the Deploying Django post by Randall Degges, it gives me a good laugh any day. Particularly these lines: Yes, grasshopper! You now see it: you have only begun to discover the amount of work that lays ahead. You've barely scratched the surface as to the tools, methods, and skills necessary to manage and operate even the simplest of production sites. Your work is cut out for you. ####What We Did We undertook a series of steps to make the experience nicer for end user: We rewrote our code-checker server queueing system in early 2013 to make it asynchronous. This si