# Using Clojure channels to increase throughput

DevFeed: [Using Clojure channels to increase throughput](<https://devfeed.tech/articles/using-clojure-channels-to-increase-throughput-30521.md>)

Original publisher: [Read original article](<https://medium.com/helpshift-engineering/using-clojure-channels-to-increase-throughput-c051cc7f9893?source=rss----3229f31ca4f4---4>)

Author: Abhinav Dubey

Published: 2025-05-28T10:07:12Z

Content type: tutorial

Language: en

Sources: [Helpshift](<https://devfeed.tech/sources/helpshift.md>)

Topics: [Clojure](<https://devfeed.tech/topics/clojure.md>), [Concurrency](<https://devfeed.tech/topics/concurrency.md>), [Kafka](<https://devfeed.tech/topics/kafka.md>), [benchmarking](<https://devfeed.tech/topics/benchmarking.md>), [async](<https://devfeed.tech/topics/async.md>), [Grafana](<https://devfeed.tech/topics/grafana.md>)

Tags: [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [channel](<https://devfeed.tech/tags/channel.md>), [clojure](<https://devfeed.tech/tags/clojure.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [core-async](<https://devfeed.tech/tags/core-async.md>), [grafana](<https://devfeed.tech/tags/grafana.md>), [kafka](<https://devfeed.tech/tags/kafka.md>), [multithreading](<https://devfeed.tech/tags/multithreading.md>), [parallelism](<https://devfeed.tech/tags/parallelism.md>)

## AI overview

This tutorial explains how to increase throughput in a Clojure-based Kafka consumer by using core.async channels and multiple worker threads. It compares a single-threaded baseline with configurations using three and ten workers, reporting shorter processing times for 1,000 events in the described benchmark.

## Source excerpt

When building systems that process large volumes of messages synchronously, performance bottlenecks can quickly become a challenge specially with single-threaded designs. In this post, we'll look at how leveraging worker threads in a Clojure-based Kafka consumer can significantly boost throughput & reduce total processing time. Using simple concurrency primitives, it's possible to achieve parallelism & scale gracefully, all while keeping the codebase clean & maintainable. We'll start with a baseline, introduce worker threads using Clojure's core.async & measure the impact. Setup & Context Kafka & Zookeeper For observability: Grafana Kafka producer: A simple script that sends messages to a Kafka topic at a configurable rate (messages per minute) for a fixed duration. After each event is pushed, a counter metric is emitted Kafka consumer: A simple script that listens to a topic & consumes messages & simulates processing time finding square-root of a number (henceforth, assume that it takes ~1 second to find the square root) . A counter metric is emitted after processing each message The Baseline: Single-Threaded Consumer If each message takes t seconds to process & there are n messages, total processing time becomes n x t seconds. This provides a clean baseline to evaluate the impact of using channel moving forward. Adding workers with core.asyncValues are conveyed on queue-like channels. By default channels require producer and consumer to rendezvous for the transfer of a value through the channel https://clojuredocs.org/clojure.core.async To improve throughput, we introduce parallelism using Clojure's core.async channels. Messages from Kafka are fed into a channel, & multiple worker threads read from this channel to process messages concurrently Here, we used >!! (blocking put) & <!! (blocking take) to communicate via channels & future to execute the business-logic on a separate thread Who gets blocked & when : The thread putting message into the channel will get bl