# Caches, Modes, and Unstable Systems

DevFeed: [Caches, Modes, and Unstable Systems](<https://devfeed.tech/articles/caches-modes-and-unstable-systems-12511.md>)

Original publisher: [Read original article](<http://brooker.co.za/blog/2021/08/27/caches.html>)

Author: Marc Brooker

Published: 2021-08-27T00:00:00Z

Content type: article

Language: en

Sources: [Marc Brooker's Blog](<https://devfeed.tech/sources/marc-brooker-s-blog.md>), [Marc Brooker's Blog](<https://devfeed.tech/sources/marc-brooker-s-blog-2.md>)

Topics: [Caching](<https://devfeed.tech/topics/caching.md>), [distributed-systems](<https://devfeed.tech/topics/distributed-systems.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [Concurrency](<https://devfeed.tech/topics/concurrency.md>), [Database](<https://devfeed.tech/topics/database.md>), [systems](<https://devfeed.tech/topics/systems.md>)

Tags: [best-practices](<https://devfeed.tech/tags/best-practices.md>), [caching](<https://devfeed.tech/tags/caching.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [database](<https://devfeed.tech/tags/database.md>), [distributed-systems](<https://devfeed.tech/tags/distributed-systems.md>), [latency](<https://devfeed.tech/tags/latency.md>), [reduce](<https://devfeed.tech/tags/reduce.md>), [scale](<https://devfeed.tech/tags/scale.md>), [systems](<https://devfeed.tech/tags/systems.md>)

## AI overview

The article explains how caching can reduce database load and latency but, when applied incorrectly, can make distributed systems unstable or metastable. It connects offered load, goodput, concurrency, and latency to congestive collapse, including the effects of an empty cache.

## Source excerpt

Caches, Modes, and Unstable Systems Best practices are seldom the best. Is your system having scaling trouble? A bit too slow? Sending too much traffic to the database? Add a caching layer! After all, caches are a best practice and a standard way to build systems. What trouble could following a best practice cause? Lots of trouble, as it turns out. In the context of distributed systems, caches are a powerful and useful tool. Unfortunately, applied incorrectly, caching can introduce some highly undesirable system behaviors. Applied incorrectly, caches can make your system unstable. Or worse, metastable. To understand why that is, we need to understand a bit about how systems scale. Let's start with the basics. Your system (hopefully) has some customers who send requests to it. Most often, you have lots of customers, and each one sends requests fairly infrequently. Those requests coming in from your customers are the offered load, generally measured in something like requests per second. Then, your system does some work on those requests, and eventually gives the results to some happy customers. The rate it does that is the goodput. The number of requests inside your system, the concurrency, is related to the offered load and goodput. When they're the same, the concurrency varies a small amount, but is relatively stable. The amount of concurrency in your system depends on the offered load and the time it takes to handle each request (latency). So far, so good. But there's some bad news. The bad news is that latency isn't really a constant. In most systems, and maybe all systems, it increases with concurrency. And concurrency increases with latency. Maybe you can see where this is going. Most real systems like this have a congestive collapse mode, where they can't get rid of requests as fast as they arrive, concurrency builds up, and the goodput drops, making the issue worse. You can use tools like Little's law to think about those situations. What does this have to do