# The Fundamental Mechanism of Scaling

DevFeed: [The Fundamental Mechanism of Scaling](<https://devfeed.tech/articles/the-fundamental-mechanism-of-scaling-12503.md>)

Original publisher: [Read original article](<http://brooker.co.za/blog/2021/01/22/cloud-scale.html>)

Author: Marc Brooker

Published: 2021-01-22T00:00:00Z

Content type: article

Language: en

Sources: [Marc Brooker's Blog](<https://devfeed.tech/sources/marc-brooker-s-blog.md>), [Marc Brooker's Blog](<https://devfeed.tech/sources/marc-brooker-s-blog-2.md>)

Topics: [distributed-systems](<https://devfeed.tech/topics/distributed-systems.md>), [Cloud](<https://devfeed.tech/topics/cloud.md>), [Raft](<https://devfeed.tech/topics/raft.md>), [Replication](<https://devfeed.tech/topics/replication.md>), [systems](<https://devfeed.tech/topics/systems.md>), [Availability](<https://devfeed.tech/topics/availability.md>), [Databases](<https://devfeed.tech/topics/databases.md>), [Transactions](<https://devfeed.tech/topics/transactions.md>)

Tags: [availability](<https://devfeed.tech/tags/availability.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [data](<https://devfeed.tech/tags/data.md>), [databases](<https://devfeed.tech/tags/databases.md>), [distributed-systems](<https://devfeed.tech/tags/distributed-systems.md>), [raft](<https://devfeed.tech/tags/raft.md>), [replication](<https://devfeed.tech/tags/replication.md>), [systems](<https://devfeed.tech/tags/systems.md>), [transactions](<https://devfeed.tech/tags/transactions.md>)

## AI overview

The article argues that coordination avoidance, rather than replication and consensus protocols such as Paxos or Raft, is the fundamental mechanism for scaling distributed systems. It presents a spectrum of designs based on how much coordination they require, including coordinated systems, data-dependent coordination, and leveraged coordination.

## Source excerpt

The Fundamental Mechanism of Scaling It's not Paxos, unfortunately. A common misconception among people picking up distributed systems is that replication and consensus protocols--Paxos, Raft, and friends--are the tools used to build the largest and most scalable systems. It's obviously true that these protocols are important building blocks. They're used to build systems that offer more availability, better durability, and stronger integrity than a single machine. At the most basic level, though, they don't make systems scale. Instead, the fundamental approach used to scale distributed systems is avoiding co-ordination. Finding ways to make progress on work that doesn't require messages to pass between machines, between clusters of machines, between datacenters and so on. The fundamental tool of cloud scaling is coordination avoidance. A Spectrum of Systems With this in mind, we can build a kind of spectrum of the amount of coordination required in different system designs: Coordinated These are the kind that use paxos, raft, chain replication or some other protocol to make a group of nodes work closely together. The amount of work done by the system generally scales with the offered work (W) and the number of nodes (N), something like O(N * W) (or, potentially, worse under some kinds of failures). Data-dependent Coordination These systems break their workload up into uncoordinated pieces (like shards), but offer ways to coordinate across shards where needed. Probably the most common type of system in this category is sharded databases, which break data up into independent pieces, but then use some kind of coordination protocol (such as two-phase commit) to offer cross-shard transactions or queries. Work done can vary between O(W) and O(N * W) depending on access patterns, customer behavior and so on. Leveraged Coordination These systems take a coordinated system and build a layer on top of it that can do many requests per unit of coordination. Generally, coordinatio