# Sending emails to our half million and growing user community

DevFeed: [Sending emails to our half million and growing user community](<https://devfeed.tech/articles/sending-emails-to-our-half-million-and-growing-user-community-20005.md>)

Original publisher: [Read original article](<http://engineering.hackerearth.com/2016/02/11/sending-emails-to-our-half-million-and-growing-user-community/>)

Published: 2016-02-11T00:00:00Z

Content type: article

Language: en

Sources: [HackerEarth](<https://devfeed.tech/sources/hackerearth.md>)

Topics: [API](<https://devfeed.tech/topics/api.md>), [MongoDB](<https://devfeed.tech/topics/mongodb.md>), [MySQL](<https://devfeed.tech/topics/mysql.md>), [Django](<https://devfeed.tech/topics/django.md>), [HTML](<https://devfeed.tech/topics/html.md>), [html elements](<https://devfeed.tech/topics/html-elements.md>)

Tags: [api](<https://devfeed.tech/tags/api.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [asynchronous-architecture](<https://devfeed.tech/tags/asynchronous-architecture.md>), [database](<https://devfeed.tech/tags/database.md>), [infrastructure-monitoring](<https://devfeed.tech/tags/infrastructure-monitoring.md>), [mongodb](<https://devfeed.tech/tags/mongodb.md>), [mysql](<https://devfeed.tech/tags/mysql.md>), [queue](<https://devfeed.tech/tags/queue.md>), [rabbitmq](<https://devfeed.tech/tags/rabbitmq.md>), [sendgrid](<https://devfeed.tech/tags/sendgrid.md>)

## AI overview

HackerEarth describes an asynchronous email delivery architecture for sending large volumes of user notifications. Emails are serialized and stored, metadata is placed in RabbitMQ queues, and workers reconstruct and deliver messages through SendGrid, with priority queues used to reduce waiting time.

## Source excerpt

At hackerearth we send emails to keep our users updated on upcoming challenges and their activities, for example, when a user successfully solves a problem, receives test-invitation, updates on user comments. Basically whenever it is appropriate. Architecture It takes lot of computational power to send emails in such large quantities synchronously. So we have implemented an asynchronous architecture to send emails. Here is brief overview of the architecture: Step 1: Construct an email and save the serialized email object in database. Step 2: Queue the metadata for later consumption. Step 3: Consume the metadata, recreate the email object and deliver. The diagram below shows high level architecture of emailing system. The solid line represents the data flow between different components. The dotted line represents the communications. Hackerearth email infrastructure consists of MySQL database, MongoDB database, RabbitMQ queues. Journey Of Email Step 1 - Construct email: There are two different type of emails. Text - Plain text emails Html - Emails with rich interface using html elements. These emails are made using django templates API used by hackerearth developers for sending email - send_email(ctx, template, subject, from_email, html=False, async=True, **kwargs) The above API creates Sendgrid Mail object, serializes and saves it in the db with some additional data. A piece of code similar to the bit shown below is used to create sendgrid Mail object import sendgrid sg = sendgrid.SendGridClient('YOUR_SENDGRID_API_KEY') message = sendgrid.Mail() message.add_to('John Doe <john@email.com>') message.set_subject('Example') message.set_html('Body') message.set_text('Body') message.set_from('Doe John <doe@email.com>') status, msg = sg.send(message) Model below is used for storing the serialized mail object and additional data. class Message(): # The actual data - a pickled sendgrid.Mail object message_data = models.TextField() when_added = models.DateTimeField(default=date