# Why do we need distributed systems?

DevFeed: [Why do we need distributed systems?](<https://devfeed.tech/articles/why-do-we-need-distributed-systems-12490.md>)

Original publisher: [Read original article](<http://brooker.co.za/blog/2020/01/02/why-distributed.html>)

Author: Marc Brooker

Published: 2020-01-02T00:00:00Z

Content type: article

Language: en

Sources: [Marc Brooker's](<https://devfeed.tech/sources/marc-brooker-s-blog.md>)

Topics: [distributed-systems](<https://devfeed.tech/topics/distributed-systems.md>), [systems](<https://devfeed.tech/topics/systems.md>), [Developer Tools](<https://devfeed.tech/topics/developer-tools.md>), [Load Balancing](<https://devfeed.tech/topics/load-balancing.md>)

Tags: [availability](<https://devfeed.tech/tags/availability.md>), [better](<https://devfeed.tech/tags/better.md>), [complex](<https://devfeed.tech/tags/complex.md>), [complexity](<https://devfeed.tech/tags/complexity.md>), [computing](<https://devfeed.tech/tags/computing.md>), [cost](<https://devfeed.tech/tags/cost.md>), [datacenter](<https://devfeed.tech/tags/datacenter.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [distributed-computing](<https://devfeed.tech/tags/distributed-computing.md>), [distributed-systems](<https://devfeed.tech/tags/distributed-systems.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [fail](<https://devfeed.tech/tags/fail.md>), [failure](<https://devfeed.tech/tags/failure.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [john](<https://devfeed.tech/tags/john.md>), [memory](<https://devfeed.tech/tags/memory.md>), [modern](<https://devfeed.tech/tags/modern.md>), [need](<https://devfeed.tech/tags/need.md>), [performance](<https://devfeed.tech/tags/performance.md>), [systems](<https://devfeed.tech/tags/systems.md>)

## AI overview

Distributed systems can improve availability, durability, scalability and efficiency by spreading state and workloads across machines. Replication and consensus avoid dependence on one server; distributed storage keeps multiple copies and can tolerate correlated failures; shared infrastructure and automatic scaling can match capacity to changing demand. Stateless systems are comparatively easy to scale, while stateful systems require decisions about data placement and routing, for which existing cloud services and software can help. These benefits come with real costs: distributed systems are harder to build, understand and operate, and single-machine designs can be faster and more efficient. The author argues that better tooling and services, including serverless, could make distributed capabilities simpler to use; the theory does not guarantee practical results.

## Source excerpt

Why do we need distributed systems? Building distributed systems is hard. It's expensive. It's complex. But we do it anyway. I grew up reading John Carmack's .plan file. His stories about the development of Doom, Quake and the rest were a formative experience for me, and a big reason I was interested in computers beyond just gaming1. I was a little bit disappointed to see this tweet: My formative memory of Python was when the Quake Live team used it for the back end work, and we wound up having serious performance problems with a few million users. My bias is that a lot (not all!) of complex "scalable" systems can be done with a simple, single C++ server. -- John Carmack (@ID_AA_Carmack) December 28, 2019 This isn't an isolated opinion, but I don't think it's a particularly good one. To be fair, there are a lot of good reasons not to build distributed systems. Complexity is one: distributed systems are legitimately harder to build, and significantly harder to understand and operate. Efficiency is another. As McSherry et al point out in Scalability! But at what COST?, single-system designs can have great performance and efficiency. Modern computers are huge and fast. I was not so much disappointed in John, as in our success at building distributed systems tools that make this untrue. Distributed computing could be much easier, and needs to be much easier. We need to get to a point, with services, tooling and technology, that monolithic systems aren't a good default. To understand why, let me answer the question in the post's title. Distributed systems offer better availability The availability of a monolithic system is limited to the availability of the piece of hardware it runs on. Modern hardware is pretty great, and combined with a good datacenter and good management practices servers can be expected to fail with an annual failure rate (AFR) in the single-digit percentages. That's OK, but not great in two ways. First, if you run a lot of systems fixing these server