# The Robust Realtime Server

DevFeed: [The Robust Realtime Server](<https://devfeed.tech/articles/the-robust-realtime-server-19980.md>)

Original publisher: [Read original article](<http://engineering.hackerearth.com/2013/05/31/the-robust-realtime-server/>)

Published: 2013-05-31T00:00:00Z

Content type: article

Language: en

Sources: [HackerEarth](<https://devfeed.tech/sources/hackerearth.md>)

Topics: [Server](<https://devfeed.tech/topics/server.md>), [Socket.IO](<https://devfeed.tech/topics/socket-io.md>), [Meteor](<https://devfeed.tech/topics/meteor.md>), [Database](<https://devfeed.tech/topics/database.md>), [Web](<https://devfeed.tech/topics/web.md>), [Cache](<https://devfeed.tech/topics/cache.md>)

Tags: [blog-post](<https://devfeed.tech/tags/blog-post.md>), [browser](<https://devfeed.tech/tags/browser.md>), [cache](<https://devfeed.tech/tags/cache.md>), [code](<https://devfeed.tech/tags/code.md>), [communication](<https://devfeed.tech/tags/communication.md>), [database](<https://devfeed.tech/tags/database.md>), [message-queue](<https://devfeed.tech/tags/message-queue.md>), [server](<https://devfeed.tech/tags/server.md>), [web](<https://devfeed.tech/tags/web.md>)

## AI overview

This developer article describes a realtime server that updates webpages when database or cache data changes. It discusses scaling problems with nowjs and socket.io, including connection limits and file descriptor leaks, then explains choosing Tornado as an alternative. The described code-submission workflow uses a web server, RabbitMQ message queues, a code-checker engine, asynchronous processing, and browser notifications.

## Source excerpt

This is going to be a long blog post but I promise you will find some interesting piece of engineering here, so stay till the end. The realtime server manages the live update of webpages when the data changes in the data storage system (database or cache). We had a realtime server in-place but there was a big problem with scaling it. ####Problem with nowjs I was told beforehand that I will be primarily working first on writing a realtime server beside many other things. Vivek Prakash told me he had written a realtime server implementation sometime ago with nowjs. But the problem with it is that it doesn't scale well beyond ~200 simultaneous connections. In a conversation on Google Groups, I came across this: In my experience, the underlying "socket.io" module is not able to scale well (more than 150 connections was a problem for me), so I had to retreat from using "nowjs" or more specifically, "socket.io" in one of my applications. After further inspection, we also saw that there was an issue with file descriptor leak and nowjs server reported ENFILE/EMFILE (Too many open files). Also nowjs project was abandoned in 2012 and last commit in github repo is that of 1 year ago. So there was need of some good alternative which can handle large number of simultaneous connections (or users). I didn't have to do much research as Vivek had already researched about it. He found Tornado and Meteor.js to be good alternative. Going by order of preference and popularity I chose Tornado, and also because it's integration with existing system looked simpler and more efficient. ####The Use Case Vivek pretty much explained me how different components of code submission works. Here is a quick explanation of it. User submits the code and a POST request is sent to webserver which further sends submission details to a message queue in RabbitMQ server (a message broker to connect various application components). Code-checker engine (consumer of RabbitMQ here) gets the submission details, e