# Fixing retries with token buckets and circuit breakers

DevFeed: [Fixing retries with token buckets and circuit breakers](<https://devfeed.tech/articles/fixing-retries-with-token-buckets-and-circuit-breakers-12517.md>)

Original publisher: [Read original article](<http://brooker.co.za/blog/2022/02/28/retries.html>)

Author: Marc Brooker

Published: 2022-02-28T00:00:00Z

Content type: article

Language: en

Sources: [Marc Brooker's Blog](<https://devfeed.tech/sources/marc-brooker-s-blog.md>), [Marc Brooker's Blog](<https://devfeed.tech/sources/marc-brooker-s-blog-2.md>)

Topics: [event driven](<https://devfeed.tech/topics/event-driven.md>), [client](<https://devfeed.tech/topics/client.md>), [Server](<https://devfeed.tech/topics/server.md>)

Tags: [event-driven](<https://devfeed.tech/tags/event-driven.md>), [performance](<https://devfeed.tech/tags/performance.md>), [simulation](<https://devfeed.tech/tags/simulation.md>), [strategy](<https://devfeed.tech/tags/strategy.md>)

## AI overview

The article compares retry strategies for distributed services, focusing on token buckets and circuit breakers. It explains how adaptive retries can limit additional load during failures and describes an event-driven simulation used to evaluate client success rates and server load.

## Source excerpt

Fixing retries with token buckets and circuit breakers Throttle yourself before you DoS yourself. After my last post on circuit breakers, a couple of people reached out to recommend using circuit breakers only to break retries, and still send normal first try traffic no matter the failure rate. That's a nice approach. It provides possible solutions to the core problem with client-side circuit breakers (they may make partial outages worse), and to the retry problem (where retries increase load on already-overloaded downstream services). To see how well that works, we can compare it to my favorite better retries approach: a token bucket. First, let's formally introduce the players: No retries. When a client wants to make a call, it makes that call as normal. If it fails, the client moves on without retrying. N retries. When a client wants to make a call, it makes that call as normal. If it fails, the client makes a maximum of N retries of the call. Adaptive Retries (aka the retry token bucket). When a client wants to make a call, it makes that call as normal. If it succeeds, it drops part of a token into a limited-size token bucket. If the call fails, retry up to N times as long as there are (whole) tokens in the bucket. For example, each success could deposit 0.1 tokens, and each retry could consume 1 token. Retry circuit breaker. When a client wants to make a call, it makes that call as normal. On success or failure, it updates statistics which track the (recent) failure rate. If that failure rate is below a threshold, it retries up to N times. If it's above the threshold, it doesn't retry at all. Think it through First, let's try think through how each of these would perform. No retries is the easiest. If the downstream failure rate is x%, the effective failure rate is x%. N retries is the next easiest. If the downstream failure rate is x%, the effective failure rate is xN, but with significant additional work. At 100% failure rate, the system does 1+N times as muc