# thundering herd Published articles for thundering herd. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## How Fixed Intervals Synchronize Distributed Clients and How Jitter Helps DevFeed: [How Fixed Intervals Synchronize Distributed Clients and How Jitter Helps]() Original publisher: [Read original article]() Author: hello@ankit-rana.com Published: 2026-09-07T00:00:00Z Content type: opinion Language: en Sources: [Ankit Rana | Mechanical Sympathy]() Topics: [reliability](), [distributed-systems](), [systems](), [Network](), [Cache](), [Redis]() Tags: [aws](), [cache](), [code](), [distributed-systems](), [jitter](), [network](), [reliability](), [resilience](), [retries](), [thundering-herd]() ### AI overview Fixed retry intervals, TTLs, restarts, reconnects, and other recurring events can cause independent clients in distributed systems to synchronize, producing repeated load spikes. Adding randomness, or jitter, helps break that alignment; the article cites AWS measurements indicating that full jitter improves total work and completion time under contention. ### Source excerpt Distributed systems synchronise themselves without being told to: every client that retries on a fixed interval, restarts after a deploy, or reconnects after a network blip ends up firing at the same moment as every other client. The resulting spike is a self-inflicted denial of service that no single component is responsible for. Adding randomness to any repeating interval breaks the alignment, and it is usually one line of code against a failure mode that otherwise requires capacity you never needed. ## Cache stampede: how one expired key takes down the database DevFeed: [Cache stampede: how one expired key takes down the database]() Original publisher: [Read original article]() Author: hello@ankit-rana.com Published: 2026-09-05T00:00:00Z Content type: tutorial Language: en Sources: [Ankit Rana | Mechanical Sympathy]() Topics: [Cache](), [Caching](), [Database](), [consistent hashing](), [Redis]() Tags: [cache-stampede](), [caching](), [connection-pool](), [consistent-hashing](), [distributed-systems](), [jitter](), [query](), [redis](), [reliability](), [thundering-herd](), [traffic](), [ttl]() ### AI overview This article explains how a cache stampede occurs when a hot key expires and many requests simultaneously recompute the same value against the database. It recommends TTL jitter to prevent synchronized expirations and request coalescing so only one caller recomputes while others wait or serve stale data. It also discusses cache warming and consistent hashing for broader cache-failure scenarios. ### Source excerpt A cache TTL is a scheduled simultaneous failure: every request being served from one key misses at the same instant and goes to the origin together. If the recompute takes two seconds, every request arriving during those two seconds also misses, so the pileup grows faster than it drains. Jitter on the TTL stops keys expiring in lockstep, and request coalescing so only one caller recomputes while the rest wait or serve stale is what stops a single expensive key from saturating the database. ## Redesigning Kubernetes Workqueues and Resyncs to Reduce Thundering Herd Effects DevFeed: [Redesigning Kubernetes Workqueues and Resyncs to Reduce Thundering Herd Effects]() Original publisher: [Read original article]() Published: 2026-09-02T00:00:00Z Content type: article Language: en Sources: [Chainguard: Unchained]() Topics: [Kubernetes](), [systems](), [Concurrent Programming](), [Latency](), [Maintainers]() Tags: [architecture](), [concurrent](), [defense-in-depth](), [failover](), [goroutines](), [kubernetes](), [latency](), [maintainers](), [reconciliation](), [thundering-herd]() ### AI overview This article explains how Kubernetes controllers can experience thundering herd effects when restarts, leader failovers, or periodic resyncs cause many keys to enter an in-memory workqueue at once. It describes the queue and informer behavior behind the problem and introduces Chainguard's redesign of workqueues and resyncs to improve reconciliation under these conditions. ### Source excerpt The thundering herd is an architecture problem. See how Chainguard redesigned workqueues and resyncs to keep reconciliation running smoothly. ## Understanding Load Balancing for Beginners DevFeed: [Understanding Load Balancing for Beginners]() Original publisher: [Read original article]() Author: Pramodh Kumar M Published: 2026-08-17T17:00:34Z Content type: tutorial Language: en Sources: [Kubernetes - KodeKloud Blog | DevOps, Cloud, Kubernetes, AI Tutorials & More]() Topics: [backends](), [networking](), [Algorithms]() Tags: [backend](), [beginners](), [cloud](), [connection-draining](), [devops](), [failover](), [health-checks](), [high-availability](), [kubernetes](), [layer-4-vs-layer-7](), [least-connections](), [load-balancer](), [load-balancer-for-beginners](), [load-balancing](), [load-balancing-algorithms](), [networking](), [nginx](), [round-robin](), [session-affinity](), [sre](), [sticky-sessions](), [thundering-herd](), [traffic-management]() ### AI overview This beginner-oriented article explains load balancing as both traffic distribution and backend failure detection. It compares round robin and least-connections algorithms, distinguishes Layer 4 from Layer 7 routing, and discusses health checks, failover, draining, sticky sessions, and the risks of overly aggressive checks. ### Source excerpt Spreading traffic is the easy half of load balancing. The half that decides whether an outage lasts ten seconds or ten minutes is noticing that a backend has stopped working, and most health checks cannot. ## The Thundering Herd Problem in Agentic AI: Why Traditional Fixes Fall Short DevFeed: [The Thundering Herd Problem in Agentic AI: Why Traditional Fixes Fall Short]() Original publisher: [Read original article]() Author: Quentin Packard Published: 2026-06-19T00:00:00Z Content type: article Language: en Sources: [Cockroach Labs]() Topics: [Agentic AI Architecture](), [agent orchestration](), [AI Agent](), [Artificial Intelligence](), [Concurrency]() Tags: [agent-orchestration](), [agentic-ai](), [ai-architecture](), [load-testing](), [parallel](), [thundering-herd]() ### AI overview This article examines how agentic AI can create a thundering herd through intentional fan-out and parallel execution. It argues that traditional mitigations only partly transfer because agent-generated synchronization can produce a sharp saturation point that staging load tests may not reveal. ### Source excerpt The thundering herd of the past was externally triggered. ## Scaling Nextdoor's Datastores: Part 3 DevFeed: [Scaling Nextdoor's Datastores: Part 3]() Original publisher: [Read original article]() Author: Ronak Shah Published: 2025-03-19T15:08:43Z Content type: article Language: en Sources: [Nextdoor]() Topics: [Caching](), [Databases](), [schema-evolution](), [Object-relational mapping](), [Django](), [Python](), [Redis](), [valkey]() Tags: [blog](), [cache](), [cache-invalidation](), [caching](), [compatibility](), [database](), [django](), [lua](), [python](), [redis](), [schema](), [serialization-format](), [thundering-herd](), [valkey]() ### AI overview Part 3 of Nextdoor's datastore-scaling series explains how applications serialize database objects for Redis or Valkey look-aside caches. It describes compatibility problems caused by runtime, package, and schema changes, including cache misses and thundering-herd effects during migrations. ### Source excerpt In this part of the Scaling Nextdoor's Datastores blog series, we'll explore how the Core-Services team at Nextdoor serializes database data for caching while ensuring forward and backward compatibility between the cache and application code. In part 1 of this series we discussed how ORMs, object-relational mapping frameworks, help abstract away database specific schemas and queries from application code. Developers simply utilize objects in their application's language to access database data. Here's a simple example of using Python's Django ORM to define a model: from django.db import models class Users(models.Model): first_name = models.CharField(max_length=30) last_name = models.CharField(max_length=30) The associated SQL create table would look like: CREATE TABLE users ( "id" bigint NOT NULL PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY, "first_name" varchar(30) NOT NULL, "last_name" varchar(30) NOT NULL ); Developers would then access database data like this: user_id = 123 user = User.objects.get(id=user_id) print(user.first_name)Object Byte Serialization for Caching An issue arises when adding a look-aside cache such as Redis/Valkey to an application: How do you store what you got from the database in the cache? A common solution to caching complex objects, such as those from ORMs, is object byte serialization. This process converts language objects into bytes before storing them in the cache. When reading from the cache the process is done in reverse where the byte data is turned into language objects. For instance in Python this is often done with the pickle package. The interaction between the application, database, and the cache looks like this: Look-Aside Cacheimport pickle # Try getting from cache ('None' if not in cache) user_bytes = cache.get("user_123") if user_bytes is not None: # Read bytes using pickle user = pickle.loads(user_bytes) else: # Fetch from database user = User.objects.get(id=123) # Convert to bytes user_bytes = pickle.dumps(user) # Sto ## The Thundering Herd Problem DevFeed: [The Thundering Herd Problem]() Original publisher: [Read original article]() Author: Matt Boyle Published: 2024-01-30T00:00:00Z Content type: article Language: en Sources: [Encore Updates]() Topics: [distributed-systems](), [incident](), [Latency](), [Database]() Tags: [database](), [distributed-systems](), [incident](), [latency](), [scale](), [thundering-herd]() ### AI overview An account of a 2018 incident in which a feature launch sent notifications to customers and enabled a waitlist, triggering a thundering herd of requests. The resulting service crashes, high latency, and high CPU usage illustrate how retries and unreliable networks can overload an API and its dependencies. ### Source excerpt Distributed Systems Horror Stories: Part Two ## Epoll's design limitations in multithreaded and load-balancing scenarios DevFeed: [Epoll's design limitations in multithreaded and load-balancing scenarios]() Original publisher: [Read original article]() Author: Marek Published: 2017-02-19T23:00:00Z Content type: opinion Language: en Sources: [Marek Majkowski]() Topics: [IO](), [Linux](), [Concurrency](), [Concurrent Programming](), [Programming](), [Kernel](), [Load Balancing]() Tags: [epoll](), [kernel](), [linux](), [load-balancing](), [threading](), [thundering-herd]() ### AI overview This article examines design limitations in Linux epoll, focusing on event delivery when file descriptors are shared across worker threads and on epoll's association with underlying kernel objects rather than file descriptors. It recommends defensive programming and avoiding several sharing, forking, and load-balancing patterns. ### Source excerpt Epoll is fundamentally broken 1/2 I/O multiplexing part #3 In previous articles we talked about: This time we'll focus on Linux's successor - the I/O multiplexing syscall. Epoll is relatively young. It was created by Davide Libenzi in 2002. For comparison: Windows did IOCP in 1994 and FreeBSD's kqueue was introduced in July 2000. Unfortunately, even though epoll is the youngest in the advanced IO multiplexing family, it's the worse in the bunch. ## Building NSQ Client Libraries DevFeed: [Building NSQ Client Libraries]() Original publisher: [Read original article]() Author: Wordbitly Published: 2013-05-09T18:58:23Z Content type: tutorial Language: en Sources: [Bitly]() Topics: [Messaging](), [client](), [configuration](), [servers](), [HTTP]() Tags: [asynchronous](), [client-library](), [configuration](), [developers](), [distributed](), [guide](), [infrastructure](), [message-queue](), [messaging](), [nsq](), [tcp](), [thundering-herd]() ### AI overview A guide to the responsibilities and design expectations of NSQ client libraries, focusing on consumers. It covers configuration, discovery, TCP connections, message handling, pipelining, asynchronous processing, and techniques for maintaining cluster robustness and performance. ### Source excerpt Brace yourself, this is a long one. The following guide was originally intended for client library developers to describe in detail all the important features and functionality we expected in an NSQ client library. While writing it we began to realize that it had value beyond client library developers. It incorporates a comprehensive analysis of most of the capabilities of NSQ (both client and server) and is therefore interesting and useful for end-users as well (or anyone using or interested in infrastructure messaging platforms). If you need some background on NSQ please see our original blog post or its follow up, spray some NSQ on it. Intro NSQ's design pushes a lot of responsibility onto client libraries in order to maintain overall cluster robustness and performance. This guide attempts to outline the various responsibilities well-behaved client libraries need to fulfill. Because publishing to nsqd is trivial (just an HTTP POST to the /put endpoint), this document focuses on consumers. By setting these expectations we hope to provide a foundation for achieving consistency across languages for NSQ users. Overview Configuration Discovery (optional) Connection Handling Feature Negotiation Data Flow / Heartbeats Message Handling RDY State Backoff Configuration At a high level, our philosophy with respect to configuration is to design the system to have the flexibility to support different workloads, use sane defaults that run well "out of the box", and minimize the number of dials. A client subscribes to a topic on a channel over a TCP connection to nsqd instance(s). You can only subscribe to one topic per connection so multiple topic consumption needs to be structured accordingly. Using nsqlookupd for discovery is optional so client libraries should support a configuration where a client connects directly to one or more nsqd instances or where it is configured to poll one or more nsqlookupd instances. When a client is configured to poll nsqlookupd the polling int