# Stack Overflow: How We Do App Caching - 2019 Edition

DevFeed: [Stack Overflow: How We Do App Caching - 2019 Edition](<https://devfeed.tech/articles/stack-overflow-how-we-do-app-caching-2019-edition-21595.md>)

Original publisher: [Read original article](<https://nickcraver.com/blog/2019/08/06/stack-overflow-how-we-do-app-caching/>)

Published: 2019-08-06T00:00:00Z

Content type: tutorial

Language: en

Sources: [Nick Craver](<https://devfeed.tech/sources/nick-craver.md>)

Topics: [Caching](<https://devfeed.tech/topics/caching.md>), [Stack Overflow](<https://devfeed.tech/topics/stackoverflow.md>), [App](<https://devfeed.tech/topics/app.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [API](<https://devfeed.tech/topics/api.md>), [cpu](<https://devfeed.tech/topics/cpu.md>), [Database](<https://devfeed.tech/topics/database.md>), [Web](<https://devfeed.tech/topics/web.md>)

Tags: [api](<https://devfeed.tech/tags/api.md>), [app](<https://devfeed.tech/tags/app.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [blog](<https://devfeed.tech/tags/blog.md>), [cache](<https://devfeed.tech/tags/cache.md>), [caching](<https://devfeed.tech/tags/caching.md>), [computer](<https://devfeed.tech/tags/computer.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cycles](<https://devfeed.tech/tags/cycles.md>), [database](<https://devfeed.tech/tags/database.md>), [latency](<https://devfeed.tech/tags/latency.md>), [memory](<https://devfeed.tech/tags/memory.md>), [performance](<https://devfeed.tech/tags/performance.md>), [processor](<https://devfeed.tech/tags/processor.md>), [server](<https://devfeed.tech/tags/server.md>), [web](<https://devfeed.tech/tags/web.md>)

## AI overview

This article explains how caching avoids repeatedly recalculating or fetching data, improving performance and reducing costs. It examines cache layers in computer processors and memory, their capacity and latency trade-offs, and the role of storage in Stack Overflow's production architecture.

## Source excerpt

This is #5 in a very long series of posts on Stack Overflow's architecture. Previous post (#4): Stack Overflow: How We Do Monitoring - 2018 Edition So...caching. What is it? It's a way to get a quick payoff by not re-calculating or fetching things over and over, resulting in performance and cost wins. That's even where the name comes from, it's a short form of the "ca-ching!" cash register sound from the dark ages of 2014 when physical currency was still a thing, before Apple Pay. I'm a dad now, deal with it. Let's say we need to call an API or query a database server or just take a bajillion numbers (Google says that's an actual word, I checked) and add them up. Those are all relatively crazy expensive. So we cache the result - we keep it handy for re-use. Why Do We Cache? I think it's important here to discuss just how expensive some of the above things are. There are several layers of caching already in play in your modern computer. As a concrete example, we're going to use one of our web servers which currently houses a pair of Intel Xeon E5-2960 v3 CPUs and 2133MHz DIMMs. Cache access is a "how many cycles" feature of a processor, so by knowing that we always run at 3.06GHz (performance power mode), we can derive the latencies (Intel architecture reference here - these processors are in the Haswell generation): L1 (per core): 4 cycles or ~1.3ns latency - 12x 32KB+32KB L2 (per core): 12 cycles or ~3.92ns latency - 12x 256KB L3 (shared): 34 cycles or ~11.11ns latency - 30MB System memory: ~100ns latency - 8x 8GB Each cache layer is able to store more, but is farther away. It's a trade-off in processor design with balances in play. For example, more memory per core means (almost certainly) on average putting it farther away on the chip from the core and that has costs in latency, opportunity costs, and power consumption. How far an electric charge has to travel has substantial impact at this scale; remember that distance is multiplied by billions every second. And I