# Go memory ballast: How I learnt to stop worrying and love the heap

DevFeed: [Go memory ballast: How I learnt to stop worrying and love the heap](<https://devfeed.tech/articles/go-memory-ballast-how-i-learnt-to-stop-worrying-and-love-the-heap-20449.md>)

Original publisher: [Read original article](<https://medium.com/twitch-news/go-memory-ballast-how-i-learnt-to-stop-worrying-and-love-the-heap-26c2462549a2?source=rss----3ae745429979--engineering>)

Author: Ross Engers

Published: 2019-04-10T21:44:55Z

Content type: article

Language: en

Sources: [Twitch](<https://devfeed.tech/sources/twitch.md>)

Topics: [Go Language](<https://devfeed.tech/topics/go-language.md>), [Twitch](<https://devfeed.tech/topics/twitch.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [Optimization](<https://devfeed.tech/topics/optimization.md>), [API](<https://devfeed.tech/topics/api.md>), [Amazon EC2](<https://devfeed.tech/topics/amazon-ec2.md>), [GraphQL](<https://devfeed.tech/topics/graphql.md>), [nginx](<https://devfeed.tech/topics/nginx.md>)

Tags: [api](<https://devfeed.tech/tags/api.md>), [ec2](<https://devfeed.tech/tags/ec2.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [gateway](<https://devfeed.tech/tags/gateway.md>), [go](<https://devfeed.tech/tags/go.md>), [golang](<https://devfeed.tech/tags/golang.md>), [graphql](<https://devfeed.tech/tags/graphql.md>), [latency](<https://devfeed.tech/tags/latency.md>), [load-balancer](<https://devfeed.tech/tags/load-balancer.md>), [memory](<https://devfeed.tech/tags/memory.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [programming](<https://devfeed.tech/tags/programming.md>), [server](<https://devfeed.tech/tags/server.md>), [twitch](<https://devfeed.tech/tags/twitch.md>)

## AI overview

A Twitch engineering article explains how a small Go memory-ballast change reduced CPU utilization on API frontend servers by about 30% and peak-load 99th-percentile API latency by about 45%. It describes the Visage gateway, traffic spikes, and the investigation behind the optimization.

## Source excerpt

Go memory ballast: How I learned to stop worrying and love the heap I'm a big fan of small code changes that can have large impact. This may seem like an obvious thing to state, but let me explain: These type of changes often involve diving into and understanding things one is not familiar with. Even with the most well factored code, there is a maintenance cost to each optimization you add, and it's usually (although not always) pretty linear with the amount of lines of code you end up adding/changing. We recently rolled out a small change that reduced the CPU utilization of our API frontend servers at Twitch by ~30% and reduced overall 99th percentile API latency during peak load by ~45%. This blog post is about the change, the process of finding it and explaining how it works. Setting the stage We have a service at Twitch called Visage, that functions as our API frontend. Visage is the central gateway for all externally originating API traffic. It is responsible for a bunch of things, from authorization to request routing, to (more recently) server-side GraphQL. As such, it has to scale to handle user traffic patterns that are somewhat out of our control. As an example, a common traffic pattern we see is a "refresh storm." This occurs when a popular broadcaster's stream drops due to a blip in their internet connectivity. In response, the broadcaster restarts the stream. This usually causes the viewers to repeatedly refresh their pages, and suddenly we have a lot more API traffic to deal with. Visage is a Go application (built with Go 1.11 at the time of this change) that runs on EC2 behind a load balancer. Being on EC2 it scales well horizontally, for the most part. However, even with the magic of EC2 and Auto Scaling groups, we still have the problem of dealing with very large traffic spikes. During refresh storms, we frequently have surges of millions of requests over a few seconds, on the order of 20x our normal load. On top of this, we would see API latency de