# least connections

A load-balancing policy that routes incoming requests to the backend server with the fewest active connections.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Tail latency amplification: why p99 of one service becomes p50 of the aggregate

DevFeed: [Tail latency amplification: why p99 of one service becomes p50 of the aggregate](<https://devfeed.tech/articles/tail-latency-amplification-why-p99-of-one-service-becomes-p50-of-the-aggregate-131023.md>)

Original publisher: [Read original article](<https://ankit-rana.com/logs/74-tail-latency-amplification/>)

Author: hello@ankit-rana.com

Published: 2026-11-12T00:00:00Z

Content type: article

Language: en

Sources: [Ankit Rana | Mechanical Sympathy](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md>)

Topics: [Latency](<https://devfeed.tech/topics/latency.md>), [least connections](<https://devfeed.tech/topics/least-connections.md>)

Tags: [aggregate](<https://devfeed.tech/tags/aggregate.md>), [application](<https://devfeed.tech/tags/application.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [arithmetic](<https://devfeed.tech/tags/arithmetic.md>), [backend](<https://devfeed.tech/tags/backend.md>), [backends](<https://devfeed.tech/tags/backends.md>), [budget](<https://devfeed.tech/tags/budget.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [churn](<https://devfeed.tech/tags/churn.md>), [code](<https://devfeed.tech/tags/code.md>), [component](<https://devfeed.tech/tags/component.md>), [delay](<https://devfeed.tech/tags/delay.md>), [distributed-systems](<https://devfeed.tech/tags/distributed-systems.md>), [fan-out](<https://devfeed.tech/tags/fan-out.md>), [latency](<https://devfeed.tech/tags/latency.md>), [load](<https://devfeed.tech/tags/load.md>), [observability](<https://devfeed.tech/tags/observability.md>), [p99](<https://devfeed.tech/tags/p99.md>), [parallel](<https://devfeed.tech/tags/parallel.md>), [performance](<https://devfeed.tech/tags/performance.md>), [probability](<https://devfeed.tech/tags/probability.md>), [request](<https://devfeed.tech/tags/request.md>), [service](<https://devfeed.tech/tags/service.md>), [tail-latency](<https://devfeed.tech/tags/tail-latency.md>)

### AI overview

In a fan-out request that waits for every backend, aggregate latency is set by the slowest response: with 60 independent calls each slow 1% of the time, about 45% of aggregate requests encounter a slow call. The excerpt recommends tracking per-request maximum latency and identifying the slowest backend, since averages can hide the delay users experience. Mitigations include reducing fan-out, hedging to another replica after a delay (with bounded extra load, safe-to-duplicate requests, and enough capacity), returning partial results where correctness allows, propagating deadlines, and routing away from slow replicas. Correlated slowdowns cluster in time; hedging helps when the delay is local to one replica.

### Source excerpt

A request that waits on N backends completes at the speed of the slowest one, so the probability of hitting a slow response is one minus the probability of avoiding it N times. At a hundred parallel calls to a service whose p99 is 100ms, roughly 63 percent of requests contain at least one 100ms response, which turns a rare event at the component level into the common case at the aggregate. Sending a duplicate request after a short delay collapses that tail for a few percent extra load.

## System Design in 6 Points, Explained with a Chai Stall That Went Viral

DevFeed: [System Design in 6 Points, Explained with a Chai Stall That Went Viral](<https://devfeed.tech/articles/system-design-in-6-points-explained-with-a-chai-stall-that-went-viral-104372.md>)

Original publisher: [Read original article](<https://ihatereading.in/blogs/system-design-in-6-points-explained-with-a-chai-stall>)

Author: Shrey Vijayvargiya

Published: 2026-10-28T09:00:00Z

Content type: tutorial

Language: en

Sources: [iHateReading](<https://devfeed.tech/sources/ihatereading.md>)

Topics: [system-design-project](<https://devfeed.tech/topics/system-design-project.md>), [Cache](<https://devfeed.tech/topics/cache.md>), [nginx](<https://devfeed.tech/topics/nginx.md>), [least connections](<https://devfeed.tech/topics/least-connections.md>), [Network](<https://devfeed.tech/topics/network.md>)

Tags: [apache-2-0](<https://devfeed.tech/tags/apache-2-0.md>), [api](<https://devfeed.tech/tags/api.md>), [asynchronous](<https://devfeed.tech/tags/asynchronous.md>), [aws](<https://devfeed.tech/tags/aws.md>), [azure](<https://devfeed.tech/tags/azure.md>), [backblaze-b2](<https://devfeed.tech/tags/backblaze-b2.md>), [backend](<https://devfeed.tech/tags/backend.md>), [backups](<https://devfeed.tech/tags/backups.md>), [batching](<https://devfeed.tech/tags/batching.md>), [blockchain](<https://devfeed.tech/tags/blockchain.md>), [bsd](<https://devfeed.tech/tags/bsd.md>), [cache](<https://devfeed.tech/tags/cache.md>), [cache-control](<https://devfeed.tech/tags/cache-control.md>), [cache-vs-queue](<https://devfeed.tech/tags/cache-vs-queue.md>), [caching](<https://devfeed.tech/tags/caching.md>), [cdn](<https://devfeed.tech/tags/cdn.md>), [coding](<https://devfeed.tech/tags/coding.md>), [development](<https://devfeed.tech/tags/development.md>), [frontend](<https://devfeed.tech/tags/frontend.md>), [javascript](<https://devfeed.tech/tags/javascript.md>), [load-balancer](<https://devfeed.tech/tags/load-balancer.md>), [load-balancer-explained](<https://devfeed.tech/tags/load-balancer-explained.md>), [logs](<https://devfeed.tech/tags/logs.md>), [node-js](<https://devfeed.tech/tags/node-js.md>), [object-storage](<https://devfeed.tech/tags/object-storage.md>), [product](<https://devfeed.tech/tags/product.md>), [programming](<https://devfeed.tech/tags/programming.md>), [queue](<https://devfeed.tech/tags/queue.md>), [react](<https://devfeed.tech/tags/react.md>), [read-replicas](<https://devfeed.tech/tags/read-replicas.md>), [read-replicas-explained](<https://devfeed.tech/tags/read-replicas-explained.md>), [retries](<https://devfeed.tech/tags/retries.md>), [retries-with-exponential-backoff-and-jitter](<https://devfeed.tech/tags/retries-with-exponential-backoff-and-jitter.md>), [system-design](<https://devfeed.tech/tags/system-design.md>), [system-design-basics](<https://devfeed.tech/tags/system-design-basics.md>), [system-design-cheat-sheet](<https://devfeed.tech/tags/system-design-cheat-sheet.md>), [system-design-for-beginners](<https://devfeed.tech/tags/system-design-for-beginners.md>), [typescript](<https://devfeed.tech/tags/typescript.md>), [web-development](<https://devfeed.tech/tags/web-development.md>)

### AI overview

The article explains six system design patterns through a viral chai stall: cache repeated reads, queue slow background work, balance traffic across healthy app servers, use read replicas for read scaling, store files in object storage behind a CDN, and add timeouts, bounded jittered retries, and structured logs around network calls. It describes key limits: caches can serve stale data and need shared storage across servers; queues defer work and require durable jobs, idempotency, and monitoring; load balancers need shared sessions and can expose database connection limits; asynchronous replicas lag and do not reduce write load; file storage has provider-specific egress costs and private files need signed URLs; retries can duplicate unsafe operations or multiply across layers. Suggested order: logs and timeouts, object storage/CDN, then cache, queue, load balancer, and replicas after query and index fixes.

### Source excerpt

System design basics in six points: cache, queue, load balancer, read replicas, object storage + CDN, and retries with timeouts and logs. One chai stall story,...

## NGINX Explained: 8 Use Cases Every Developer Should Know

DevFeed: [NGINX Explained: 8 Use Cases Every Developer Should Know](<https://devfeed.tech/articles/nginx-explained-8-use-cases-every-developer-should-know-85194.md>)

Original publisher: [Read original article](<https://blog.levelupcoding.com/p/nginx-explained-top-8-use-cases>)

Author: Nikki Siapno

Published: 2026-10-12T11:46:46Z

Content type: tutorial

Language: en

Sources: [Level Up Coding System Design Newsletter](<https://devfeed.tech/sources/level-up-coding-system-design-newsletter.md>)

Topics: [nginx](<https://devfeed.tech/topics/nginx.md>), [least connections](<https://devfeed.tech/topics/least-connections.md>), [Deployment Strategies](<https://devfeed.tech/topics/deployment-strategies.md>), [WebSocket](<https://devfeed.tech/topics/websocket.md>), [Back end](<https://devfeed.tech/topics/backend.md>)

Tags: [allows](<https://devfeed.tech/tags/allows.md>), [api](<https://devfeed.tech/tags/api.md>), [apis](<https://devfeed.tech/tags/apis.md>), [application](<https://devfeed.tech/tags/application.md>), [application-servers](<https://devfeed.tech/tags/application-servers.md>), [browser](<https://devfeed.tech/tags/browser.md>), [business](<https://devfeed.tech/tags/business.md>), [cache](<https://devfeed.tech/tags/cache.md>), [cache-stampede](<https://devfeed.tech/tags/cache-stampede.md>), [caching](<https://devfeed.tech/tags/caching.md>), [certificate](<https://devfeed.tech/tags/certificate.md>), [certificate-management](<https://devfeed.tech/tags/certificate-management.md>), [concurrent](<https://devfeed.tech/tags/concurrent.md>), [configuration](<https://devfeed.tech/tags/configuration.md>), [nginx](<https://devfeed.tech/tags/nginx.md>), [server](<https://devfeed.tech/tags/server.md>), [web](<https://devfeed.tech/tags/web.md>), [web-server](<https://devfeed.tech/tags/web-server.md>)

### AI overview

A practical overview of eight NGINX roles at the edge: reverse proxying, load balancing, static file serving, TLS termination, caching, rate limiting, request routing, and WebSocket proxying. It explains how these functions affect application capacity and where traffic configuration should remain separate from application business logic.

### Source excerpt

Calling NGINX a web server isn't wrong. It just describes one of its jobs...

## Balls Into Bins In Distributed Systems

DevFeed: [Balls Into Bins In Distributed Systems](<https://devfeed.tech/articles/balls-into-bins-in-distributed-systems-12481.md>)

Original publisher: [Read original article](<http://brooker.co.za/blog/2018/01/01/balls-into-bins.html>)

Author: Marc Brooker

Published: 2018-01-01T00:00:00Z

Content type: article

Language: en

Sources: [Marc Brooker's](<https://devfeed.tech/sources/marc-brooker-s-blog.md>)

Topics: [Load Balancing](<https://devfeed.tech/topics/load-balancing.md>), [distributed algorithm](<https://devfeed.tech/topics/distributed-algorithm.md>), [Algorithms](<https://devfeed.tech/topics/algorithms.md>), [least connections](<https://devfeed.tech/topics/least-connections.md>), [Back end](<https://devfeed.tech/topics/backend.md>), [consistent hashing](<https://devfeed.tech/topics/consistent-hashing.md>), [Apache Cassandra](<https://devfeed.tech/topics/cassandra.md>)

Tags: [consistent-hashing](<https://devfeed.tech/tags/consistent-hashing.md>), [distributed-systems](<https://devfeed.tech/tags/distributed-systems.md>), [hash-table](<https://devfeed.tech/tags/hash-table.md>), [least-connections](<https://devfeed.tech/tags/least-connections.md>), [load-balancer](<https://devfeed.tech/tags/load-balancer.md>), [load-balancing](<https://devfeed.tech/tags/load-balancing.md>), [random](<https://devfeed.tech/tags/random.md>), [sharding](<https://devfeed.tech/tags/sharding.md>), [stateful](<https://devfeed.tech/tags/stateful.md>)

### AI overview

Random assignment can leave backends badly imbalanced when the number of requests is small relative to the number of backends, so load-sensitive balancing remains useful. The article models requests as balls and backends as bins, connecting the problem to hash tables, load balancing, and hash-based work allocation. It reviews theoretical bounds but favors simple repeated simulations in Python or R for practical planning: simulations are easier to adapt to variable request sizes, arrivals, completions, bursts, and backend scaling, while theoretical results can check that the simulation is plausible. In the reported examples, with 100 requests and 100 backends the busiest backend averages about 4.2 requests; with 1,000 requests and 100 backends, it averages 18.8 versus a mean of 10. The imbalance shrinks as requests increase, but the article concludes that random load balancing, shuffle sharding, and consistent hashing can distribute load poorly at small scale; stateful or stateless load-sensitive methods such as best-of-two remain relevant.

### Source excerpt

Balls Into Bins In Distributed Systems Throwing things can be fun. If you've come across the Balls Into Bins problem, you probably heard about in context of hash tables. When you hash things into a hash table (especially with separate chaining) it's really useful to be able to ask "If I throw 𝑀 balls into 𝑁 bins, what is the distribution of balls in bins?" You can see how this is fundamental to hash tables: the amortized complexity argument for hash tables depends on their being some load factor (i.e. 𝑀/𝑁) for which most bins contain a small number of items. Once this stops being true, lookup and insertion time on hash tables starts to get ugly. So from that perspective it's already clearly an important problem. Load Balancing and Work Allocation Hash tables aren't the only place that the Balls Into Bins problem is interesting. It comes up often in distributed systems, too. For one example, think about a load balancer (in this case a distributor of independent requests) sending load to some number of backends. Requests (𝑀) are balls, and the backends are bins (𝑁) and typically there are multiple requests going to each backend (𝑀 > 𝑁). If we know how to solve for the number of balls in each bin, we can understand the limits of random load balancing, or whether we need a stateful load balancing algorithm like least connections. This is an important question to ask, because sharing consistent state limits scalability, and sharing eventually-consistent state can even make load balancing decisions worse. Load balancing is much easier if it can be done statelessly. A related problem is push-based work allocation. Here, there is some co-ordinator handing out work items to a fleet of workers, and trying to have those workers do approximately equal amounts of work. One way that systems end up with this pattern is if they are using shuffle sharding or consistent hashing to distribute work items (or records). These hashing-based methods can be great for scaling, and so are wid

## Context aware MySQL pools via HAProxy

DevFeed: [Context aware MySQL pools via HAProxy](<https://devfeed.tech/articles/context-aware-mysql-pools-via-haproxy-69619.md>)

Original publisher: [Read original article](<https://github.blog/engineering/infrastructure/context-aware-mysql-pools-via-haproxy/>)

Author: Shlomi Noach

Published: 2016-08-17T07:00:00Z

Content type: tutorial

Language: en

Sources: [GitHub Blog](<https://devfeed.tech/sources/github-engineering.md>), [GitHub Engineering](<https://devfeed.tech/sources/github.md>)

Topics: [MySQL](<https://devfeed.tech/topics/mysql.md>), [connection draining](<https://devfeed.tech/topics/connection-draining.md>), [least connections](<https://devfeed.tech/topics/least-connections.md>)

Tags: [availability](<https://devfeed.tech/tags/availability.md>), [cluster](<https://devfeed.tech/tags/cluster.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [haproxy](<https://devfeed.tech/tags/haproxy.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [metadata](<https://devfeed.tech/tags/metadata.md>), [mysql](<https://devfeed.tech/tags/mysql.md>)

### AI overview

The article describes GitHub's context-aware MySQL read pools behind HAProxy. Replica checks evaluate replication lag, server role, and maintenance state; when too few healthy replicas remain, HAProxy can route reads to a backup pool that accepts lagging replicas. The article also covers manual controls, monitoring, testing, and the shell-based check service used to make pool membership decisions.

### Source excerpt

At GitHub we use MySQL as our main datastore. While repository data lies in git, metadata is stored in MySQL. This includes Issues, Pull Requests, Comments etc. We also auth...

## Yes, Redis cares about your load balancing strategy

DevFeed: [Yes, Redis cares about your load balancing strategy](<https://devfeed.tech/articles/yes-redis-cares-about-your-load-balancing-strategy-15967.md>)

Original publisher: [Read original article](<https://developer.squareup.com/blog/yes-redis-cares-about-your-load-balancing-strategy>)

Author: Square Engineering

Published: 2011-07-06T16:00:00Z

Content type: article

Language: en

Sources: [Square Corner](<https://devfeed.tech/sources/square-corner-blog-medium.md>)

Topics: [Load Balancing](<https://devfeed.tech/topics/load-balancing.md>), [least connections](<https://devfeed.tech/topics/least-connections.md>), [Redis](<https://devfeed.tech/topics/redis.md>), [retry](<https://devfeed.tech/topics/retry.md>)

Tags: [cluster](<https://devfeed.tech/tags/cluster.md>), [distributed-system](<https://devfeed.tech/tags/distributed-system.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [load-balancer](<https://devfeed.tech/tags/load-balancer.md>), [load-balancing](<https://devfeed.tech/tags/load-balancing.md>), [redis](<https://devfeed.tech/tags/redis.md>), [retry](<https://devfeed.tech/tags/retry.md>), [ruby](<https://devfeed.tech/tags/ruby.md>), [web-servers](<https://devfeed.tech/tags/web-servers.md>)

### AI overview

The Redis socket error in the example results from least-connection load balancing combined with aggressive Redis timeouts: during low traffic, some machines receive no work long enough for Redis to close their idle sockets, so later requests encounter Errno::EAGAIN. The excerpt contrasts round-robin, which spreads requests evenly but can overload a node when request times vary, with least-connection, which can handle heterogeneous workloads better under higher traffic but leaves some nodes idle during quiet periods. The error signals that retrying may work. The article frames the failure as an interaction among systems behaving as configured.

### Source excerpt

Innocuous details like load balancing web servers can produce downstream results in hard to predict ways.