# Ankit Rana | Mechanical Sympathy

Engineering logs on distributed systems, database internals, CPU and memory behaviour, and AI infrastructure by Ankit Rana, Senior Software Engineer at Walmart Global Tech.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Ordering without clocks: Lamport timestamps and vector clocks in practice

DevFeed: [Ordering without clocks: Lamport timestamps and vector clocks in practice](<https://devfeed.tech/articles/ordering-without-clocks-lamport-timestamps-and-vector-clocks-in-practice-136262.md>)

Original publisher: [Read original article](<https://ankit-rana.com/logs/75-ordering-without-clocks-logical-time/>)

Author: hello@ankit-rana.com

Published: 2026-11-16T00:00:00Z

Content type: article

Language: en

Sources: [Ankit Rana | Mechanical Sympathy](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md>)

Topics: [synchronization](<https://devfeed.tech/topics/synchronization.md>), [consistency](<https://devfeed.tech/topics/consistency.md>), [ordering](<https://devfeed.tech/topics/ordering.md>)

Tags: [causality](<https://devfeed.tech/tags/causality.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [conflict-resolution](<https://devfeed.tech/tags/conflict-resolution.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [distributed-systems](<https://devfeed.tech/tags/distributed-systems.md>), [lamport-timestamps](<https://devfeed.tech/tags/lamport-timestamps.md>), [logical-clocks](<https://devfeed.tech/tags/logical-clocks.md>), [ordering](<https://devfeed.tech/tags/ordering.md>), [timestamps](<https://devfeed.tech/tags/timestamps.md>), [vector-clocks](<https://devfeed.tech/tags/vector-clocks.md>)

### AI overview

Lamport timestamps provide causal ordering without relying on drifting clocks: increment a counter for local events, attach it to messages, and on receipt advance past the larger counter. A smaller timestamp does not prove that one event caused another, so using Lamport values alone to resolve conflicts still selects an arbitrary winner. Vector clocks track a counter per node and identify concurrent updates, allowing systems such as Riak to return both values for application merging; their size can grow with the number of writers. The excerpt describes dotted version vectors as limiting history to per-key concurrency, and hybrid logical clocks as combining physical time with a logical counter. It also notes that Kafka offsets order records only within a partition and database version columns support per-row optimistic locking.

### Source excerpt

A wall clock cannot order events across machines because clocks drift and NTP corrections can move time backwards. A Lamport timestamp is a counter incremented on every event and carried on every message, which guarantees that if one event caused another the causal one has a smaller number, giving a consistent total order without any clock. It cannot tell you whether two events were concurrent, which is exactly the question you need answered during a conflict, and that is what vector clocks provide at the cost of storing one counter per node.

## Tail latency amplification: why p99 of one service becomes p50 of the aggregate

DevFeed: [Tail latency amplification: why p99 of one service becomes p50 of the aggregate](<https://devfeed.tech/articles/tail-latency-amplification-why-p99-of-one-service-becomes-p50-of-the-aggregate-131023.md>)

Original publisher: [Read original article](<https://ankit-rana.com/logs/74-tail-latency-amplification/>)

Author: hello@ankit-rana.com

Published: 2026-11-12T00:00:00Z

Content type: article

Language: en

Sources: [Ankit Rana | Mechanical Sympathy](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md>)

Topics: [Latency](<https://devfeed.tech/topics/latency.md>), [least connections](<https://devfeed.tech/topics/least-connections.md>)

Tags: [aggregate](<https://devfeed.tech/tags/aggregate.md>), [application](<https://devfeed.tech/tags/application.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [arithmetic](<https://devfeed.tech/tags/arithmetic.md>), [backend](<https://devfeed.tech/tags/backend.md>), [backends](<https://devfeed.tech/tags/backends.md>), [budget](<https://devfeed.tech/tags/budget.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [churn](<https://devfeed.tech/tags/churn.md>), [code](<https://devfeed.tech/tags/code.md>), [component](<https://devfeed.tech/tags/component.md>), [delay](<https://devfeed.tech/tags/delay.md>), [distributed-systems](<https://devfeed.tech/tags/distributed-systems.md>), [fan-out](<https://devfeed.tech/tags/fan-out.md>), [latency](<https://devfeed.tech/tags/latency.md>), [load](<https://devfeed.tech/tags/load.md>), [observability](<https://devfeed.tech/tags/observability.md>), [p99](<https://devfeed.tech/tags/p99.md>), [parallel](<https://devfeed.tech/tags/parallel.md>), [performance](<https://devfeed.tech/tags/performance.md>), [probability](<https://devfeed.tech/tags/probability.md>), [request](<https://devfeed.tech/tags/request.md>), [service](<https://devfeed.tech/tags/service.md>), [tail-latency](<https://devfeed.tech/tags/tail-latency.md>)

### AI overview

In a fan-out request that waits for every backend, aggregate latency is set by the slowest response: with 60 independent calls each slow 1% of the time, about 45% of aggregate requests encounter a slow call. The excerpt recommends tracking per-request maximum latency and identifying the slowest backend, since averages can hide the delay users experience. Mitigations include reducing fan-out, hedging to another replica after a delay (with bounded extra load, safe-to-duplicate requests, and enough capacity), returning partial results where correctness allows, propagating deadlines, and routing away from slow replicas. Correlated slowdowns cluster in time; hedging helps when the delay is local to one replica.

### Source excerpt

A request that waits on N backends completes at the speed of the slowest one, so the probability of hitting a slow response is one minus the probability of avoiding it N times. At a hundred parallel calls to a service whose p99 is 100ms, roughly 63 percent of requests contain at least one 100ms response, which turns a rare event at the component level into the common case at the aggregate. Sending a duplicate request after a short delay collapses that tail for a few percent extra load.

## Service mesh: what it buys and the latency it charges

DevFeed: [Service mesh: what it buys and the latency it charges](<https://devfeed.tech/articles/service-mesh-what-it-buys-and-the-latency-it-charges-124066.md>)

Original publisher: [Read original article](<https://ankit-rana.com/logs/73-service-mesh-what-it-buys-and-charges/>)

Author: hello@ankit-rana.com

Published: 2026-11-09T00:00:00Z

Content type: opinion

Language: en

Sources: [Ankit Rana | Mechanical Sympathy](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md>)

Topics: [service-mesh](<https://devfeed.tech/topics/service-mesh.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [traffic splitting](<https://devfeed.tech/topics/traffic-splitting.md>), [Load Balancing](<https://devfeed.tech/topics/load-balancing.md>)

Tags: [envoy](<https://devfeed.tech/tags/envoy.md>), [istio](<https://devfeed.tech/tags/istio.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [latency](<https://devfeed.tech/tags/latency.md>), [mesh](<https://devfeed.tech/tags/mesh.md>), [platform-engineering](<https://devfeed.tech/tags/platform-engineering.md>), [service-mesh](<https://devfeed.tech/tags/service-mesh.md>), [sidecar](<https://devfeed.tech/tags/sidecar.md>)

### AI overview

A sidecar service mesh adds two proxy crossings to each service call; the article estimates 0.5-2 ms of added latency per call, with worse tail latency, plus roughly 50-150 MB of memory and some CPU per pod. These costs compound along deep call chains and across pod counts. The comparison is not mesh versus no capabilities: without a mesh, mTLS, retries, circuit breaking, load balancing and telemetry must be implemented and upgraded in service libraries, potentially across languages. A mesh is most compelling for many services across several languages, mandatory estate-wide encryption, traffic splitting, or telemetry for components you do not control. It is less attractive for a small, single-language system, latency-critical deep paths, or teams without platform ownership. For an existing mesh, measure injected versus bypassed latency on the same service, configure retries in one layer, place proxy metrics beside application metrics, and consider ambient or sidecarless modes when per-pod memory is the main constraint, accepting less per-pod isolation.

### Source excerpt

A sidecar proxy adds two extra network hops per call, typically half a millisecond to two milliseconds of added latency plus memory and CPU per pod. The comparison people make is against having nothing, which is wrong, because the mTLS, retries, circuit breaking and telemetry it provides were previously implemented as libraries that had their own cost and had to be upgraded across every service. A mesh is worth it when you have enough services in enough languages that library sprawl is the larger problem, and it is overhead when you do not.

## Retry storms: how the resilience layer became the outage

DevFeed: [Retry storms: how the resilience layer became the outage](<https://devfeed.tech/articles/retry-storms-how-the-resilience-layer-became-the-outage-119561.md>)

Original publisher: [Read original article](<https://ankit-rana.com/logs/72-retry-storms-resilience-became-the-outage/>)

Author: hello@ankit-rana.com

Published: 2026-11-05T00:00:00Z

Content type: article

Language: en

Sources: [Ankit Rana | Mechanical Sympathy](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md>)

Topics: [retry](<https://devfeed.tech/topics/retry.md>), [Resilience](<https://devfeed.tech/topics/resilience.md>), [rate-limiting](<https://devfeed.tech/topics/rate-limiting.md>), [service-mesh](<https://devfeed.tech/topics/service-mesh.md>)

Tags: [backpressure](<https://devfeed.tech/tags/backpressure.md>), [budget](<https://devfeed.tech/tags/budget.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [circuit-breaker](<https://devfeed.tech/tags/circuit-breaker.md>), [distributed-systems](<https://devfeed.tech/tags/distributed-systems.md>), [outage](<https://devfeed.tech/tags/outage.md>), [reliability](<https://devfeed.tech/tags/reliability.md>), [resilience](<https://devfeed.tech/tags/resilience.md>), [retries](<https://devfeed.tech/tags/retries.md>), [retry](<https://devfeed.tech/tags/retry.md>)

### AI overview

Retries configured independently across service layers multiply: two retries at both a gateway and service can send nine attempts to a dependency per original request, just as degradation leaves it least able to respond. That extra load can persist after the trigger is fixed and prevent recovery. The article recommends limiting retry ownership, using circuit breakers to fail fast and probe cautiously, and ensuring each inner layer's total retry time fits within its caller's timeout; propagated deadlines help enforce this. Per-request attempt limits do not bound aggregate load, so a retry budget tied to total traffic can cap amplification. Retry only plausibly transient failures, honor 429 responses and `Retry-After`, and use idempotency keys for writes such as charges. Add jitter to backoff. The text says Envoy and the gRPC ecosystem provide retry budgets, and gRPC natively propagates deadlines.

### Source excerpt

Retries configured independently at each hop multiply, so three layers each retrying three times means nine requests reaching the bottom for every one at the top. When a dependency degrades, that multiplier arrives exactly when it has least capacity, and the retries alone can keep it down after the original cause is gone. Per-call retry counts cannot bound this because no layer knows what the others are doing, which is why the useful control is a retry budget expressed as a fraction of total traffic.

## R plus W greater than N is the arithmetic that decides your consistency

DevFeed: [R plus W greater than N is the arithmetic that decides your consistency](<https://devfeed.tech/articles/r-plus-w-greater-than-n-is-the-arithmetic-that-decides-your-consistency-112595.md>)

Original publisher: [Read original article](<https://ankit-rana.com/logs/71-quorum-arithmetic-r-plus-w/>)

Author: hello@ankit-rana.com

Published: 2026-11-02T00:00:00Z

Content type: article

Language: en

Sources: [Ankit Rana | Mechanical Sympathy](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md>)

Topics: [consistency](<https://devfeed.tech/topics/consistency.md>), [database-consistency](<https://devfeed.tech/topics/database-consistency.md>)

Tags: [3](<https://devfeed.tech/tags/3.md>), [availability](<https://devfeed.tech/tags/availability.md>), [balance](<https://devfeed.tech/tags/balance.md>), [cassandra](<https://devfeed.tech/tags/cassandra.md>), [concurrently](<https://devfeed.tech/tags/concurrently.md>), [conflict-resolution](<https://devfeed.tech/tags/conflict-resolution.md>), [consensus](<https://devfeed.tech/tags/consensus.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [cost](<https://devfeed.tech/tags/cost.md>), [dashboard](<https://devfeed.tech/tags/dashboard.md>), [distributed-systems](<https://devfeed.tech/tags/distributed-systems.md>), [durability](<https://devfeed.tech/tags/durability.md>), [dynamo](<https://devfeed.tech/tags/dynamo.md>), [inequality](<https://devfeed.tech/tags/inequality.md>), [latency](<https://devfeed.tech/tags/latency.md>), [overlap](<https://devfeed.tech/tags/overlap.md>), [quorum](<https://devfeed.tech/tags/quorum.md>), [reading](<https://devfeed.tech/tags/reading.md>), [replica](<https://devfeed.tech/tags/replica.md>), [replication](<https://devfeed.tech/tags/replication.md>), [store](<https://devfeed.tech/tags/store.md>), [write](<https://devfeed.tech/tags/write.md>), [writing](<https://devfeed.tech/tags/writing.md>)

### AI overview

For N replicas, reads of R replicas and writes acknowledged by W replicas guarantee overlap only when R + W > N; when the sum equals N, a read can miss an acknowledged write. Increasing R or W trades latency and availability for stronger recency coverage: the coordinator waits for the Rth or Wth response, and requiring all replicas can block writes when one is unavailable. This overlap guarantee does not resolve concurrent writes, which may be settled by last-write-wins timestamps; conditional writes or consensus provide stronger conflict handling at higher cost. Sloppy quorums can satisfy W on nodes outside the key's replica set, weakening the guarantee until hinted data arrives. Check each store's N, R and W, its concurrent-write behavior, and whether it uses sloppy quorums.

### Source excerpt

In a replicated store with N copies, reading from R replicas and writing to W replicas guarantees a read sees the latest write only when R plus W exceeds N, because that forces the two sets to share at least one replica. The inequality is the entire guarantee, and it says nothing about which value wins when two writes happen concurrently, so overlap gives you recency and not conflict resolution. Tuning R and W is choosing where to spend latency, since the coordinator waits for the slowest replica in whichever set is larger.

## Leader election is easy, failover is where systems die

DevFeed: [Leader election is easy, failover is where systems die](<https://devfeed.tech/articles/leader-election-is-easy-failover-is-where-systems-die-106790.md>)

Original publisher: [Read original article](<https://ankit-rana.com/logs/70-leader-election-failover-fencing/>)

Author: hello@ankit-rana.com

Published: 2026-10-29T00:00:00Z

Content type: article

Language: en

Sources: [Ankit Rana | Mechanical Sympathy](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md>)

Topics: [distributed algorithm](<https://devfeed.tech/topics/distributed-algorithm.md>), [consistency](<https://devfeed.tech/topics/consistency.md>), [Raft](<https://devfeed.tech/topics/raft.md>)

Tags: [algorithms](<https://devfeed.tech/tags/algorithms.md>), [api](<https://devfeed.tech/tags/api.md>), [application](<https://devfeed.tech/tags/application.md>), [asynchronous](<https://devfeed.tech/tags/asynchronous.md>), [clock](<https://devfeed.tech/tags/clock.md>), [cluster](<https://devfeed.tech/tags/cluster.md>), [column](<https://devfeed.tech/tags/column.md>), [conditional](<https://devfeed.tech/tags/conditional.md>), [configuration](<https://devfeed.tech/tags/configuration.md>), [consensus](<https://devfeed.tech/tags/consensus.md>), [database](<https://devfeed.tech/tags/database.md>), [distributed-systems](<https://devfeed.tech/tags/distributed-systems.md>), [election](<https://devfeed.tech/tags/election.md>), [failover](<https://devfeed.tech/tags/failover.md>), [fencing](<https://devfeed.tech/tags/fencing.md>), [layer](<https://devfeed.tech/tags/layer.md>), [leader](<https://devfeed.tech/tags/leader.md>), [leader-election](<https://devfeed.tech/tags/leader-election.md>), [network](<https://devfeed.tech/tags/network.md>), [reliability](<https://devfeed.tech/tags/reliability.md>), [storage](<https://devfeed.tech/tags/storage.md>), [timeout](<https://devfeed.tech/tags/timeout.md>), [token](<https://devfeed.tech/tags/token.md>), [zookeeper](<https://devfeed.tech/tags/zookeeper.md>)

### AI overview

Leader election can succeed while a paused former leader resumes and keeps writing: it cannot know its lease expired until it contacts the coordinator. Checking a lease immediately before a write does not close the race, since a pause can occur between the check and write. The text's remedy is fencing: grants carry monotonically increasing tokens, and storage rejects writes with tokens older than the highest already seen. It cites ZooKeeper zxid, etcd revision, and Kubernetes resourceVersion as token sources, while noting that a third-party API unable to check tokens requires idempotent operations to make duplicates harmless. Failover time also includes detection, state recovery, and warm-up; measuring time to elect alone can understate the outage. The suggested test pauses a leader past its lease TTL, then resumes it, to expose stale-writer behavior.

### Source excerpt

Electing a leader is a solved problem that libraries handle well. The hard part is that a leader which has been declared dead often does not know it, because a garbage collection pause or a network partition is indistinguishable from being alive and idle from the inside. The old leader resumes and keeps writing, so for a window there are two. Only a monotonically increasing fencing token checked at the storage layer stops the stale writer, because every timeout-based scheme assumes a clock nobody controls.

## Your trace has a gap exactly where the bug is

DevFeed: [Your trace has a gap exactly where the bug is](<https://devfeed.tech/articles/your-trace-has-a-gap-exactly-where-the-bug-is-100770.md>)

Original publisher: [Read original article](<https://ankit-rana.com/logs/69-trace-gap-where-the-bug-is/>)

Author: hello@ankit-rana.com

Published: 2026-10-26T00:00:00Z

Content type: article

Language: en

Sources: [Ankit Rana | Mechanical Sympathy](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md>)

Topics: [tracing](<https://devfeed.tech/topics/tracing.md>), [threadpools](<https://devfeed.tech/topics/threadpools.md>), [Publish-subscribe pattern](<https://devfeed.tech/topics/pubsub.md>), [out-of-process](<https://devfeed.tech/topics/out-of-process.md>)

Tags: [asynchronous](<https://devfeed.tech/tags/asynchronous.md>), [bug](<https://devfeed.tech/tags/bug.md>), [callback](<https://devfeed.tech/tags/callback.md>), [client](<https://devfeed.tech/tags/client.md>), [context](<https://devfeed.tech/tags/context.md>), [context-propagation](<https://devfeed.tech/tags/context-propagation.md>), [debug](<https://devfeed.tech/tags/debug.md>), [debugging](<https://devfeed.tech/tags/debugging.md>), [distributed-systems](<https://devfeed.tech/tags/distributed-systems.md>), [distributed-tracing](<https://devfeed.tech/tags/distributed-tracing.md>), [error](<https://devfeed.tech/tags/error.md>), [executor](<https://devfeed.tech/tags/executor.md>), [gap](<https://devfeed.tech/tags/gap.md>), [garbage-collection](<https://devfeed.tech/tags/garbage-collection.md>), [latency](<https://devfeed.tech/tags/latency.md>), [message-queues](<https://devfeed.tech/tags/message-queues.md>), [observability](<https://devfeed.tech/tags/observability.md>), [opentelemetry](<https://devfeed.tech/tags/opentelemetry.md>), [queues](<https://devfeed.tech/tags/queues.md>), [retries](<https://devfeed.tech/tags/retries.md>), [thread](<https://devfeed.tech/tags/thread.md>), [thread-pool](<https://devfeed.tech/tags/thread-pool.md>), [trace](<https://devfeed.tech/tags/trace.md>), [tracing](<https://devfeed.tech/tags/tracing.md>)

### AI overview

A long parent span with much shorter child spans can mean work lost its trace context at an execution boundary, leaving missing work invisible rather than producing an error. The excerpt explains that thread-local context does not cross executor threads unless captured and restored (for example, by wrapping the executor); queue context must be injected into and extracted from message headers, with links recommended for asynchronous causality; and callbacks or reactive schedulers need context bridging. Library retries can hide multiple attempts inside one span, so instrumenting attempts separately can reveal failures and retry counts. Similar gaps can come from stop-the-world GC pauses or connection-pool waits, which can be distinguished by correlating pauses or instrumenting acquisition. To investigate, compare child durations with the parent and inspect what ran during the unaccounted interval. Head sampling decides at trace start and can miss incidents; tail sampling can retain slow or failed traces but requires buffering and routing all trace spans to one collector.

### Source excerpt

Distributed tracing works by passing a context object along the call path, and any boundary that does not carry it forward silently ends the trace. Thread pool handoffs, message queues, callbacks and library-managed retries are the usual places, which is unfortunate because those are also where the latency and the bugs live. A gap does not appear as an error, it appears as a span that seems to take four seconds while its children are simply absent.

## The saga pattern is compensating transactions with better marketing

DevFeed: [The saga pattern is compensating transactions with better marketing](<https://devfeed.tech/articles/the-saga-pattern-is-compensating-transactions-with-better-marketing-94471.md>)

Original publisher: [Read original article](<https://ankit-rana.com/logs/68-sagas-are-compensating-transactions/>)

Author: hello@ankit-rana.com

Published: 2026-10-22T00:00:00Z

Content type: article

Language: en

Sources: [Ankit Rana | Mechanical Sympathy](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md>)

Topics: [Transactions](<https://devfeed.tech/topics/transactions.md>), [acid](<https://devfeed.tech/topics/acid.md>), [Publish-subscribe pattern](<https://devfeed.tech/topics/pubsub.md>)

Tags: [architecture](<https://devfeed.tech/tags/architecture.md>), [atomic](<https://devfeed.tech/tags/atomic.md>), [availability](<https://devfeed.tech/tags/availability.md>), [compensating-transactions](<https://devfeed.tech/tags/compensating-transactions.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [distributed-transactions](<https://devfeed.tech/tags/distributed-transactions.md>), [email](<https://devfeed.tech/tags/email.md>), [isolation](<https://devfeed.tech/tags/isolation.md>), [microservices](<https://devfeed.tech/tags/microservices.md>), [observable](<https://devfeed.tech/tags/observable.md>), [payment](<https://devfeed.tech/tags/payment.md>), [reliability](<https://devfeed.tech/tags/reliability.md>), [rollback](<https://devfeed.tech/tags/rollback.md>), [saga](<https://devfeed.tech/tags/saga.md>), [transactions](<https://devfeed.tech/tags/transactions.md>)

### AI overview

A saga trades a single transaction's isolation for independently committed service steps: intermediate states are visible, and failures require new compensating actions rather than rollback. Those actions can fail too, so the article calls for idempotent steps and compensations, retries, timeouts, durable and queryable saga state, and human escalation when compensation cannot succeed. It recommends ordering reversible work first and irreversible actions, such as sending confirmation, last; where possible, reserve resources before finalizing them. Choreography avoids a coordinator but leaves workflow and progress scattered across services. Orchestration centralizes sequencing, state tracking, recovery, and compensation, at the cost of a coordination component. The pattern adds substantial failure-path complexity; the author advises using it when service boundaries make a local transaction impossible, and notes that visible intermediate states must be acceptable to the product.

### Source excerpt

A saga replaces one atomic transaction with a sequence of local transactions, each of which commits and becomes visible immediately, so there is no isolation and intermediate states are observable. When a later step fails, earlier ones are undone by running compensating actions, which are forward operations with their own failure modes rather than a rollback. The difficulty is not the mechanism but the steps that cannot be compensated, such as an email already sent or a payment already captured, which have to be designed around rather than reversed.

## Two-phase commit provides atomicity but can leave participants blocked and reduce transaction availability

DevFeed: [Two-phase commit provides atomicity but can leave participants blocked and reduce transaction availability](<https://devfeed.tech/articles/two-phase-commit-is-not-dead-it-is-just-expensive-and-you-should-know-the-price-89928.md>)

Original publisher: [Read original article](<https://ankit-rana.com/logs/67-two-phase-commit-is-expensive-not-dead/>)

Author: hello@ankit-rana.com

Published: 2026-10-19T00:00:00Z

Content type: article

Language: en

Sources: [Ankit Rana | Mechanical Sympathy](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md>)

Topics: [scalable-architecture](<https://devfeed.tech/topics/scalable-architecture.md>), [self-hosted databases vs managed services](<https://devfeed.tech/topics/self-hosted-databases-vs-managed-services.md>), [CAP theorem](<https://devfeed.tech/topics/cap-theorem.md>), [Availability](<https://devfeed.tech/topics/availability.md>), [synchronization](<https://devfeed.tech/topics/synchronization.md>)

Tags: [architecture](<https://devfeed.tech/tags/architecture.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [database](<https://devfeed.tech/tags/database.md>), [databases](<https://devfeed.tech/tags/databases.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [distributed-systems](<https://devfeed.tech/tags/distributed-systems.md>), [distributed-transaction](<https://devfeed.tech/tags/distributed-transaction.md>), [distributed-transactions](<https://devfeed.tech/tags/distributed-transactions.md>), [fsync](<https://devfeed.tech/tags/fsync.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [latency](<https://devfeed.tech/tags/latency.md>), [locks](<https://devfeed.tech/tags/locks.md>), [two-phase-commit](<https://devfeed.tech/tags/two-phase-commit.md>)

### AI overview

Two-phase commit (2PC) provides atomicity across resource managers by requiring participants to hold locks after voting until a coordinator decides. If the coordinator fails during that interval, participants can remain blocked. The article explains how adding participants lowers transaction availability, contrasts 2PC with the transactional outbox pattern, and identifies cases where 2PC may still be appropriate.

### Source excerpt

Two-phase commit gives real atomicity across independent resource managers, and it pays for it by making every participant hold locks from the moment it votes until the coordinator tells it what to do. If the coordinator dies in that window, participants are stuck holding locks with no legal way to proceed, because they promised they could commit and cannot take it back. Availability is the product of every participant's availability, so adding services makes the transaction less likely to complete rather than more.

## Virtual threads changed the calculus for blocking I/O and not for CPU work

DevFeed: [Virtual threads changed the calculus for blocking I/O and not for CPU work](<https://devfeed.tech/articles/virtual-threads-changed-the-calculus-for-blocking-i-o-and-not-for-cpu-work-87574.md>)

Original publisher: [Read original article](<https://ankit-rana.com/logs/66-virtual-threads-blocking-io-not-cpu/>)

Author: hello@ankit-rana.com

Published: 2026-10-15T00:00:00Z

Content type: article

Language: en

Sources: [Ankit Rana | Mechanical Sympathy](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md>)

Topics: [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>), [threadpools](<https://devfeed.tech/topics/threadpools.md>), [Deadlock](<https://devfeed.tech/topics/deadlock.md>), [performance-optimization](<https://devfeed.tech/topics/performance-optimization.md>)

Tags: [backpressure](<https://devfeed.tech/tags/backpressure.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [database](<https://devfeed.tech/tags/database.md>), [deadlock](<https://devfeed.tech/tags/deadlock.md>), [java](<https://devfeed.tech/tags/java.md>), [jvm](<https://devfeed.tech/tags/jvm.md>), [loom](<https://devfeed.tech/tags/loom.md>), [performance](<https://devfeed.tech/tags/performance.md>), [virtual-threads](<https://devfeed.tech/tags/virtual-threads.md>)

### AI overview

The article explains how Java virtual threads improve concurrency for services that spend much of their time waiting on blocking I/O, while providing no benefit for CPU-bound work. It discusses carrier pinning, avoiding virtual-thread pools, and placing concurrency limits at constrained downstream resources.

### Source excerpt

A virtual thread that blocks on I/O unmounts from its carrier and costs nothing while it waits, which makes thread-per-request viable at concurrency levels that would have needed reactive code. It does nothing for CPU bound work, because computation still needs a real core and the carrier pool is sized to the core count. Pooling virtual threads is an anti-pattern since they are cheap to create, and blocking inside a synchronized block pins the carrier, which can deadlock a workload that would have been fine on platform threads.

## Idempotency keys: the API contract that makes retries safe

DevFeed: [Idempotency keys: the API contract that makes retries safe](<https://devfeed.tech/articles/idempotency-keys-the-api-contract-that-makes-retries-safe-85160.md>)

Original publisher: [Read original article](<https://ankit-rana.com/logs/65-idempotency-keys-api-contract/>)

Author: hello@ankit-rana.com

Published: 2026-10-12T00:00:00Z

Content type: tutorial

Language: en

Sources: [Ankit Rana | Mechanical Sympathy](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md>)

Topics: [API](<https://devfeed.tech/topics/api.md>), [REST API](<https://devfeed.tech/topics/rest-api.md>), [database-consistency](<https://devfeed.tech/topics/database-consistency.md>), [retry](<https://devfeed.tech/topics/retry.md>), [Publish-subscribe pattern](<https://devfeed.tech/topics/pubsub.md>), [event-sourcing](<https://devfeed.tech/topics/event-sourcing.md>), [CAP theorem](<https://devfeed.tech/topics/cap-theorem.md>)

Tags: [api](<https://devfeed.tech/tags/api.md>), [api-design](<https://devfeed.tech/tags/api-design.md>), [application](<https://devfeed.tech/tags/application.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [concurrent](<https://devfeed.tech/tags/concurrent.md>), [database](<https://devfeed.tech/tags/database.md>), [distributed-systems](<https://devfeed.tech/tags/distributed-systems.md>), [exactly-once](<https://devfeed.tech/tags/exactly-once.md>), [idempotency](<https://devfeed.tech/tags/idempotency.md>), [keys](<https://devfeed.tech/tags/keys.md>), [postgresql](<https://devfeed.tech/tags/postgresql.md>), [reliability](<https://devfeed.tech/tags/reliability.md>), [retries](<https://devfeed.tech/tags/retries.md>), [transaction](<https://devfeed.tech/tags/transaction.md>)

### AI overview

The article explains how API idempotency keys make retries safe when a client cannot tell whether a timed-out request succeeded. It covers reserving keys atomically, validating repeated requests, replaying stored responses, coordinating external side effects with providers, and the limits imposed by key expiry.

### Source excerpt

A client that times out cannot distinguish a request that failed from one that succeeded with a lost response, so it must retry, and without an idempotency key that retry duplicates the write. The key has to be reserved before the work begins, in the same transaction, so two concurrent copies of the same request cannot both pass the check. The stored response must then be replayed verbatim, because recomputing it can return a different answer than the one the first caller received.

## Immutability is a concurrency strategy, not a style preference

DevFeed: [Immutability is a concurrency strategy, not a style preference](<https://devfeed.tech/articles/immutability-is-a-concurrency-strategy-not-a-style-preference-77003.md>)

Original publisher: [Read original article](<https://ankit-rana.com/logs/64-immutability-is-a-concurrency-strategy/>)

Author: hello@ankit-rana.com

Published: 2026-10-08T00:00:00Z

Content type: article

Language: en

Sources: [Ankit Rana | Mechanical Sympathy](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md>)

Topics: [thread safe data sharing](<https://devfeed.tech/topics/thread-safe-data-sharing.md>), [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>)

Tags: [concurrency](<https://devfeed.tech/tags/concurrency.md>), [design](<https://devfeed.tech/tags/design.md>), [immutability](<https://devfeed.tech/tags/immutability.md>), [java](<https://devfeed.tech/tags/java.md>), [memory-model](<https://devfeed.tech/tags/memory-model.md>), [thread-safety](<https://devfeed.tech/tags/thread-safety.md>), [threads](<https://devfeed.tech/tags/threads.md>)

### AI overview

The article explains how immutable objects and snapshot replacement can prevent readers from observing partially updated shared state. It covers safe publication, defensive copies, allocation tradeoffs, and cases where local mutation or lifecycle-driven mutability remain appropriate.

### Source excerpt

An object that cannot change after construction is safe to share across threads with no lock, no volatile, and no reasoning about happens-before, because there is no write for a reader to miss. That removes an entire category of bug rather than mitigating it. The usual objection is allocation cost, and it is generally the wrong trade to optimise: short lived objects are cheap to collect, while a data race is invisible in testing and reproduces once a quarter in production.

## Thread pool starvation from nested async calls

DevFeed: [Thread pool starvation from nested async calls](<https://devfeed.tech/articles/thread-pool-starvation-from-nested-async-calls-64743.md>)

Original publisher: [Read original article](<https://ankit-rana.com/logs/63-thread-pool-starvation-nested-async/>)

Author: hello@ankit-rana.com

Published: 2026-10-05T00:00:00Z

Content type: tutorial

Language: en

Sources: [Ankit Rana | Mechanical Sympathy](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md>)

Topics: [Deadlock](<https://devfeed.tech/topics/deadlock.md>), [threadpools](<https://devfeed.tech/topics/threadpools.md>), [queueing theory](<https://devfeed.tech/topics/queueing-theory.md>), [synchronization](<https://devfeed.tech/topics/synchronization.md>)

Tags: [async](<https://devfeed.tech/tags/async.md>), [completablefuture](<https://devfeed.tech/tags/completablefuture.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [deadlock](<https://devfeed.tech/tags/deadlock.md>), [executor](<https://devfeed.tech/tags/executor.md>), [executorservice](<https://devfeed.tech/tags/executorservice.md>), [java](<https://devfeed.tech/tags/java.md>), [reliability](<https://devfeed.tech/tags/reliability.md>), [thread-pool](<https://devfeed.tech/tags/thread-pool.md>)

### AI overview

Nested asynchronous work in a bounded Java thread pool can starve the pool when parent tasks block while waiting for child tasks submitted to that same pool. The article explains why increasing pool size only delays the failure and presents alternatives: compose futures without blocking, isolate unavoidable blocking work in a separate bounded pool, use timeouts, and capture thread dumps and queue metrics to diagnose the problem. It also describes the limits of virtual threads for this failure mode.

### Source excerpt

A task that blocks waiting for another task submitted to the same bounded pool can deadlock, because the thread it needs is the thread it is occupying. With N threads and N waiting parents, every thread is blocked on work that has no thread left to run it. Nothing detects this, there is no error and no timeout by default, so the service simply stops responding while looking healthy. Raising the pool size delays it rather than fixing it, since the failure needs only enough concurrent parents to fill the pool.

## Optimistic vs pessimistic locking: choose by contention, not preference

DevFeed: [Optimistic vs pessimistic locking: choose by contention, not preference](<https://devfeed.tech/articles/optimistic-vs-pessimistic-locking-choose-by-contention-not-preference-64322.md>)

Original publisher: [Read original article](<https://ankit-rana.com/logs/62-optimistic-vs-pessimistic-locking/>)

Author: hello@ankit-rana.com

Published: 2026-10-03T00:00:00Z

Content type: article

Language: en

Sources: [Ankit Rana | Mechanical Sympathy](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md>)

Topics: [locking](<https://devfeed.tech/topics/locking.md>), [Concurrency](<https://devfeed.tech/topics/concurrency.md>), [Transactions](<https://devfeed.tech/topics/transactions.md>)

Tags: [concurrency](<https://devfeed.tech/tags/concurrency.md>), [database](<https://devfeed.tech/tags/database.md>), [jpa](<https://devfeed.tech/tags/jpa.md>), [locking](<https://devfeed.tech/tags/locking.md>), [optimistic-locking](<https://devfeed.tech/tags/optimistic-locking.md>), [postgresql](<https://devfeed.tech/tags/postgresql.md>), [transactions](<https://devfeed.tech/tags/transactions.md>)

### AI overview

The article compares optimistic and pessimistic database locking, explaining how contention, transaction duration, and retry behavior affect their tradeoffs. It also discusses atomic updates, sharding, and queueing as alternatives for highly contended operations, and recommends keeping transactions short around external calls.

### Source excerpt

Optimistic locking detects conflict at write time using a version column, so readers never block and the cost of a conflict is a wasted transaction plus a retry. Pessimistic locking prevents conflict by taking a lock up front, so work is never wasted and contending writers queue. The crossover is contention: below a few percent conflict rate optimistic wins clearly, above roughly ten percent the retry storm costs more than the queue would have. Both are wrong for a human editing a form, where the correct answer is to show the conflict rather than resolve it.

## Backpressure is not rate limiting, and treating them the same fills your queues

DevFeed: [Backpressure is not rate limiting, and treating them the same fills your queues](<https://devfeed.tech/articles/backpressure-is-not-rate-limiting-and-treating-them-the-same-fills-your-queues-63064.md>)

Original publisher: [Read original article](<https://ankit-rana.com/logs/61-backpressure-is-not-rate-limiting/>)

Author: hello@ankit-rana.com

Published: 2026-10-01T00:00:00Z

Content type: article

Language: en

Sources: [Ankit Rana | Mechanical Sympathy](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md>)

Topics: [backpressure](<https://devfeed.tech/topics/backpressure.md>), [rate-limiting](<https://devfeed.tech/topics/rate-limiting.md>), [Reactive Streams](<https://devfeed.tech/topics/reactive-streams.md>)

Tags: [backpressure](<https://devfeed.tech/tags/backpressure.md>), [distributed-systems](<https://devfeed.tech/tags/distributed-systems.md>), [latency](<https://devfeed.tech/tags/latency.md>), [memory](<https://devfeed.tech/tags/memory.md>), [queues](<https://devfeed.tech/tags/queues.md>), [rate-limiting](<https://devfeed.tech/tags/rate-limiting.md>), [reliability](<https://devfeed.tech/tags/reliability.md>), [resilience](<https://devfeed.tech/tags/resilience.md>)

### AI overview

The article distinguishes rate limiting, which applies a fixed policy ceiling, from backpressure, which communicates current capacity constraints upstream. It explains how unbounded queues turn overload into latency and memory failures, and discusses bounded queues, rejection policies, HTTP status codes, flow control, load shedding, and deadlines as ways to manage overload.

### Source excerpt

Rate limiting enforces a fixed ceiling chosen in advance, which protects you from a producer that sends too much and does nothing when your own capacity drops. Backpressure is a signal travelling backwards from the component that is actually struggling, so the allowed rate tracks real capacity instead of a guess. The thing that defeats both is an unbounded queue, because it absorbs the overload silently and converts a fast rejection into unbounded latency and eventually a memory failure.

## Heap looks fine, pod still OOM killed

DevFeed: [Heap looks fine, pod still OOM killed](<https://devfeed.tech/articles/heap-looks-fine-pod-still-oom-killed-61580.md>)

Original publisher: [Read original article](<https://ankit-rana.com/logs/60-heap-fine-pod-oom-killed/>)

Author: hello@ankit-rana.com

Published: 2026-09-29T00:00:00Z

Content type: tutorial

Language: en

Sources: [Ankit Rana | Mechanical Sympathy](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md>)

Topics: [heap overflow](<https://devfeed.tech/topics/heap-overflow.md>), [Kernel](<https://devfeed.tech/topics/kernel.md>)

Tags: [container](<https://devfeed.tech/tags/container.md>), [containers](<https://devfeed.tech/tags/containers.md>), [heap](<https://devfeed.tech/tags/heap.md>), [java](<https://devfeed.tech/tags/java.md>), [jit](<https://devfeed.tech/tags/jit.md>), [jvm](<https://devfeed.tech/tags/jvm.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [memory](<https://devfeed.tech/tags/memory.md>), [oom](<https://devfeed.tech/tags/oom.md>), [socket](<https://devfeed.tech/tags/socket.md>)

### AI overview

A Java service can be killed by the kernel for exceeding its container memory limit even when its heap graph appears healthy. The article explains that memory used by metaspace, thread stacks, the JIT code cache, direct byte buffers and native allocations also counts toward the limit, and that a kernel OOM kill produces no Java exception or stack trace.

### Source excerpt

A container limit applies to the whole process, and the Java heap is only one part of what a JVM allocates. Thread stacks, metaspace, the code cache, direct byte buffers and native allocations inside libraries all live outside the heap and outside every heap graph, so a service can be killed by the kernel while its heap chart shows plenty of headroom. The kill produces no Java exception and no stack trace, because SIGKILL gives the process no opportunity to report anything.

## Diagnosing high p99 latency caused by Java garbage collection pauses

DevFeed: [Diagnosing high p99 latency caused by Java garbage collection pauses](<https://devfeed.tech/articles/your-p99-is-a-garbage-collector-setting-60525.md>)

Original publisher: [Read original article](<https://ankit-rana.com/logs/59-p99-is-a-garbage-collector-setting/>)

Author: hello@ankit-rana.com

Published: 2026-09-27T00:00:00Z

Content type: article

Language: en

Sources: [Ankit Rana | Mechanical Sympathy](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md>)

Topics: [Go Language](<https://devfeed.tech/topics/go-language.md>), [Running Median](<https://devfeed.tech/topics/running-median.md>), [proguard](<https://devfeed.tech/topics/proguard.md>)

Tags: [cost](<https://devfeed.tech/tags/cost.md>), [garbage-collection](<https://devfeed.tech/tags/garbage-collection.md>), [java](<https://devfeed.tech/tags/java.md>), [jvm](<https://devfeed.tech/tags/jvm.md>), [latency](<https://devfeed.tech/tags/latency.md>), [logs](<https://devfeed.tech/tags/logs.md>), [median](<https://devfeed.tech/tags/median.md>), [observability](<https://devfeed.tech/tags/observability.md>), [performance](<https://devfeed.tech/tags/performance.md>)

### AI overview

The article explains how stop-the-world garbage collection can cause p99 latency to rise even when application code paths are fast. It recommends comparing GC pause logs with request timestamps and explains how allocation rate, object retention, and heap size affect collection frequency and pause duration.

### Source excerpt

A p99 that is many times the median with no matching slow code path is usually a stop the world garbage collection pause, which stops every application thread regardless of what it was doing. The lever is allocation rate rather than heap size, because most collectors scale their work with the number of surviving objects and the frequency of collection follows directly from how fast you fill the young generation. Enlarging the heap reduces collection frequency and can lengthen each pause, which trades a common small delay for a rare large one.

## The GIL is not why your Python is slow

DevFeed: [The GIL is not why your Python is slow](<https://devfeed.tech/articles/the-gil-is-not-why-your-python-is-slow-59761.md>)

Original publisher: [Read original article](<https://ankit-rana.com/logs/58-the-gil-is-not-why-your-python-is-slow/>)

Author: hello@ankit-rana.com

Published: 2026-09-25T00:00:00Z

Content type: article

Language: en

Sources: [Ankit Rana | Mechanical Sympathy](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md>)

Topics: [Python](<https://devfeed.tech/topics/python.md>), [Concurrency](<https://devfeed.tech/topics/concurrency.md>), [Database](<https://devfeed.tech/topics/database.md>), [Object-relational mapping](<https://devfeed.tech/topics/orm.md>), [JSON](<https://devfeed.tech/topics/json.md>), [HTTP](<https://devfeed.tech/topics/http.md>)

Tags: [bytecode](<https://devfeed.tech/tags/bytecode.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [database](<https://devfeed.tech/tags/database.md>), [gil](<https://devfeed.tech/tags/gil.md>), [http](<https://devfeed.tech/tags/http.md>), [json](<https://devfeed.tech/tags/json.md>), [multiprocessing](<https://devfeed.tech/tags/multiprocessing.md>), [orm](<https://devfeed.tech/tags/orm.md>), [performance](<https://devfeed.tech/tags/performance.md>), [profiling](<https://devfeed.tech/tags/profiling.md>), [python](<https://devfeed.tech/tags/python.md>), [query](<https://devfeed.tech/tags/query.md>), [threads](<https://devfeed.tech/tags/threads.md>)

### AI overview

The article argues that Python's global interpreter lock is often blamed incorrectly for slow services. It uses profiling to show that database round trips, N+1 queries, unbatched external calls, and JSON serialization usually dominate request latency, while I/O releases the lock.

### Source excerpt

The global interpreter lock only prevents two threads from executing Python bytecode simultaneously, and it is released during file and socket I/O, during most C extension work including NumPy and compression, and periodically during long computation. A typical service spends its time waiting on the network or serialising JSON, neither of which the lock affects. Before blaming it, profile: the answer is usually N+1 queries, unbatched round trips or serialisation, and none of those get faster by removing the lock.

## Async does not mean parallel, and confusing them costs you a thread pool

DevFeed: [Async does not mean parallel, and confusing them costs you a thread pool](<https://devfeed.tech/articles/async-does-not-mean-parallel-and-confusing-them-costs-you-a-thread-pool-58423.md>)

Original publisher: [Read original article](<https://ankit-rana.com/logs/57-async-does-not-mean-parallel/>)

Author: hello@ankit-rana.com

Published: 2026-09-23T00:00:00Z

Content type: article

Language: en

Sources: [Ankit Rana | Mechanical Sympathy](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md>)

Topics: [async](<https://devfeed.tech/topics/async.md>), [Concurrency](<https://devfeed.tech/topics/concurrency.md>), [Parallelism](<https://devfeed.tech/topics/parallelism.md>), [Python](<https://devfeed.tech/topics/python.md>), [await](<https://devfeed.tech/topics/await.md>), [Process](<https://devfeed.tech/topics/process.md>), [cpu](<https://devfeed.tech/topics/cpu.md>)

Tags: [async](<https://devfeed.tech/tags/async.md>), [await](<https://devfeed.tech/tags/await.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [event-loop](<https://devfeed.tech/tags/event-loop.md>), [nodejs](<https://devfeed.tech/tags/nodejs.md>), [parallel](<https://devfeed.tech/tags/parallel.md>), [performance](<https://devfeed.tech/tags/performance.md>), [process](<https://devfeed.tech/tags/process.md>), [python](<https://devfeed.tech/tags/python.md>), [threads](<https://devfeed.tech/tags/threads.md>)

### AI overview

This article explains that asynchronous programming enables concurrency by interleaving tasks during waits, but does not make CPU-bound work parallel. In a single-threaded Python event loop, a computation-heavy handler can block every other request and increase latency. The article recommends moving CPU work to another process when genuine parallel execution is needed, while noting the cost of copying arguments and results.

### Source excerpt

Async is about not blocking while waiting for something else, and parallel is about doing two things at the same instant on different cores. An event loop achieves enormous concurrency with one thread because most request time is spent waiting on I/O, and that same design collapses the moment a handler does real computation, because nothing else runs until it finishes. Marking CPU bound work async does not make it faster and usually makes latency worse by moving it onto a thread other work depends on.

## Deadlocks are a lock ordering bug, and lock ordering is a design decision

DevFeed: [Deadlocks are a lock ordering bug, and lock ordering is a design decision](<https://devfeed.tech/articles/deadlocks-are-a-lock-ordering-bug-and-lock-ordering-is-a-design-decision-55691.md>)

Original publisher: [Read original article](<https://ankit-rana.com/logs/56-deadlocks-are-a-lock-ordering-bug/>)

Author: hello@ankit-rana.com

Published: 2026-09-21T00:00:00Z

Content type: article

Language: en

Sources: [Ankit Rana | Mechanical Sympathy](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md>)

Topics: [Deadlock](<https://devfeed.tech/topics/deadlock.md>), [ordering](<https://devfeed.tech/topics/ordering.md>), [Concurrency](<https://devfeed.tech/topics/concurrency.md>), [Database](<https://devfeed.tech/topics/database.md>), [locking](<https://devfeed.tech/topics/locking.md>), [Sorting](<https://devfeed.tech/topics/sorting.md>), [Transactions](<https://devfeed.tech/topics/transactions.md>)

Tags: [concurrency](<https://devfeed.tech/tags/concurrency.md>), [database](<https://devfeed.tech/tags/database.md>), [deadlock](<https://devfeed.tech/tags/deadlock.md>), [deadlocks](<https://devfeed.tech/tags/deadlocks.md>), [java](<https://devfeed.tech/tags/java.md>), [locking](<https://devfeed.tech/tags/locking.md>), [ordering](<https://devfeed.tech/tags/ordering.md>), [postgresql](<https://devfeed.tech/tags/postgresql.md>), [sorting](<https://devfeed.tech/tags/sorting.md>), [transactions](<https://devfeed.tech/tags/transactions.md>)

### AI overview

The article explains how database deadlocks arise when concurrent transactions acquire the same locks in opposite orders. It recommends imposing a consistent global lock order, typically by sorting identifiers, to prevent circular wait in transfers and batch operations.

### Source excerpt

A deadlock needs four conditions and the only one you control is circular wait, which is why every deadlock reduces to two transactions acquiring the same locks in opposite orders. Imposing a global order on lock acquisition, usually by sorting the identifiers before touching any row, makes the cycle impossible to form. A database deadlock is the survivable version because the engine detects it and kills a victim, and an application level deadlock across two mutexes has no detector and simply hangs.

## Hot keys defeat consistent hashing

DevFeed: [Hot keys defeat consistent hashing](<https://devfeed.tech/articles/hot-keys-defeat-consistent-hashing-55282.md>)

Original publisher: [Read original article](<https://ankit-rana.com/logs/55-hot-keys-defeat-consistent-hashing/>)

Author: hello@ankit-rana.com

Published: 2026-09-19T00:00:00Z

Content type: article

Language: en

Sources: [Ankit Rana | Mechanical Sympathy](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md>)

Topics: [consistent hashing](<https://devfeed.tech/topics/consistent-hashing.md>), [Cache](<https://devfeed.tech/topics/cache.md>), [Redis](<https://devfeed.tech/topics/redis.md>)

Tags: [cache](<https://devfeed.tech/tags/cache.md>), [caching](<https://devfeed.tech/tags/caching.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [cluster](<https://devfeed.tech/tags/cluster.md>), [consistent-hashing](<https://devfeed.tech/tags/consistent-hashing.md>), [hot-keys](<https://devfeed.tech/tags/hot-keys.md>), [node](<https://devfeed.tech/tags/node.md>), [redis](<https://devfeed.tech/tags/redis.md>), [scalability](<https://devfeed.tech/tags/scalability.md>), [sharding](<https://devfeed.tech/tags/sharding.md>), [traffic](<https://devfeed.tech/tags/traffic.md>)

### AI overview

Consistent hashing can distribute keys evenly across cache nodes without distributing request traffic evenly. The article explains how hot keys overload individual Redis nodes and outlines remedies such as key replication, value sharding, and local caching.

### Source excerpt

Consistent hashing distributes keys evenly across nodes, which is a different problem from distributing traffic evenly. One key that receives forty percent of requests lands on exactly one node no matter how many virtual nodes you configure, and adding capacity does nothing because the bottleneck is a single key rather than the cluster. Fixing it requires changing the key: replicating it under several suffixes, splitting the value into shards, or putting a small local cache in front of the handful of keys that are actually hot.

## Local and distributed cache coherence: stale data caused by missed invalidation messages

DevFeed: [Local and distributed cache coherence: stale data caused by missed invalidation messages](<https://devfeed.tech/articles/local-cache-plus-distributed-cache-the-coherence-bill-nobody-budgets-for-39606.md>)

Original publisher: [Read original article](<https://ankit-rana.com/logs/54-local-and-distributed-cache-coherence/>)

Author: hello@ankit-rana.com

Published: 2026-09-17T00:00:00Z

Content type: opinion

Language: en

Sources: [Ankit Rana | Mechanical Sympathy](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md>)

Topics: [Cache](<https://devfeed.tech/topics/cache.md>), [Redis](<https://devfeed.tech/topics/redis.md>), [data](<https://devfeed.tech/topics/data.md>), [Network](<https://devfeed.tech/topics/network.md>)

Tags: [cache](<https://devfeed.tech/tags/cache.md>), [cache-coherence](<https://devfeed.tech/tags/cache-coherence.md>), [caching](<https://devfeed.tech/tags/caching.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [data](<https://devfeed.tech/tags/data.md>), [distributed-cache](<https://devfeed.tech/tags/distributed-cache.md>), [distributed-systems](<https://devfeed.tech/tags/distributed-systems.md>), [jvm](<https://devfeed.tech/tags/jvm.md>), [network](<https://devfeed.tech/tags/network.md>), [redis](<https://devfeed.tech/tags/redis.md>), [ttl](<https://devfeed.tech/tags/ttl.md>)

### AI overview

The article explains that adding an in-process cache in front of a distributed cache can improve latency but creates independently stale copies across service instances. It focuses on Redis pub/sub invalidation, which provides no persistence, acknowledgement, retry, or replay, allowing instances to miss invalidation messages and serve stale data until their TTL expires.

### Source excerpt

An in-process cache in front of a distributed cache removes a network hop and adds one independent copy of the data per instance, each of which can be stale on its own schedule. Invalidation is normally broadcast over pub/sub, which is fire and forget, so any instance that is restarting, garbage collecting or briefly disconnected simply misses the message and serves stale data until its TTL expires. That TTL is not a performance setting, it is the maximum duration of incorrectness.

## Why your Redis is slow: it is single threaded and you sent it KEYS

DevFeed: [Why your Redis is slow: it is single threaded and you sent it KEYS](<https://devfeed.tech/articles/why-your-redis-is-slow-it-is-single-threaded-and-you-sent-it-keys-39605.md>)

Original publisher: [Read original article](<https://ankit-rana.com/logs/53-redis-single-threaded-keys-command/>)

Author: hello@ankit-rana.com

Published: 2026-09-15T00:00:00Z

Content type: tutorial

Language: en

Sources: [Ankit Rana | Mechanical Sympathy](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md>)

Topics: [Redis](<https://devfeed.tech/topics/redis.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [Concurrency](<https://devfeed.tech/topics/concurrency.md>)

Tags: [blocking](<https://devfeed.tech/tags/blocking.md>), [caching](<https://devfeed.tech/tags/caching.md>), [command](<https://devfeed.tech/tags/command.md>), [latency](<https://devfeed.tech/tags/latency.md>), [observability](<https://devfeed.tech/tags/observability.md>), [performance](<https://devfeed.tech/tags/performance.md>), [redis](<https://devfeed.tech/tags/redis.md>), [single-threaded](<https://devfeed.tech/tags/single-threaded.md>)

### AI overview

The article explains that Redis executes commands one at a time on a single thread, so expensive commands can block every client and cause latency spikes across all operations. It identifies KEYS as a key offender and also discusses large DEL operations, unbounded LRANGE, SMEMBERS on large sets, and FLUSHALL. Redis 6 threaded I/O helps with socket handling but does not change single-threaded command execution.

### Source excerpt

Redis executes commands one at a time on a single thread, so any command that takes 200ms makes every other client wait 200ms regardless of how trivial their request was. KEYS scans the entire keyspace and is the obvious offender, but DEL on a large collection, unbounded LRANGE, SMEMBERS on a big set and FLUSHALL are all linear in the size of the data and block identically. The tell is a latency spike across all operations at once while CPU and network look normal.

## Cache invalidation is a distributed systems problem

DevFeed: [Cache invalidation is a distributed systems problem](<https://devfeed.tech/articles/cache-invalidation-is-a-distributed-systems-problem-in-a-convenience-costume-39604.md>)

Original publisher: [Read original article](<https://ankit-rana.com/logs/52-cache-invalidation-distributed-problem/>)

Author: hello@ankit-rana.com

Published: 2026-09-13T00:00:00Z

Content type: tutorial

Language: en

Sources: [Ankit Rana | Mechanical Sympathy](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md>)

Topics: [Cache](<https://devfeed.tech/topics/cache.md>), [distributed-systems](<https://devfeed.tech/topics/distributed-systems.md>)

Tags: [cache](<https://devfeed.tech/tags/cache.md>), [cache-invalidation](<https://devfeed.tech/tags/cache-invalidation.md>), [caching](<https://devfeed.tech/tags/caching.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [distributed-systems](<https://devfeed.tech/tags/distributed-systems.md>), [ordering](<https://devfeed.tech/tags/ordering.md>), [race-conditions](<https://devfeed.tech/tags/race-conditions.md>), [redis](<https://devfeed.tech/tags/redis.md>), [ttl](<https://devfeed.tech/tags/ttl.md>)

### AI overview

Cache invalidation coordinates updates between a database and a cache without a shared transaction. The article explains how operation ordering can let a concurrent reader repopulate stale data and recommends writing to the database before invalidating the cache, followed by a second invalidation to close the remaining race.

### Source excerpt

A cache and a database are two stores that must agree, which makes every invalidation a distributed transaction without a coordinator. The ordering matters more than the mechanism: invalidating before the database write leaves a window where a concurrent reader repopulates the cache with the old value and it stays wrong until the TTL expires. Deleting the key rather than writing the new value removes a whole class of ordering bug, because two concurrent deletes commute and two concurrent writes do not.

[Next page](<https://devfeed.tech/sources/ankit-rana-mechanical-sympathy.md?cursor=WyIyMDI2LTA5LTEzVDAwOjAwOjAwKzAwOjAwIiwgImE0Y2E2ZDgxLWI3YTEtNDRiOC1hN2JlLTdlNzc1ZGI3OTU2YiJd>)