# rush: Scalable Asynchronous Distributed Computing via Shared State in R

DevFeed: [rush: Scalable Asynchronous Distributed Computing via Shared State in R](<https://devfeed.tech/articles/rush-scalable-asynchronous-distributed-computing-via-shared-state-in-r-101581.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.21430>)

Author: Marc Becker, Bernd Bischl

Published: 2026-10-27T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [distributed algorithm](<https://devfeed.tech/topics/distributed-algorithm.md>), [Hyperparameter optimization](<https://devfeed.tech/topics/hyperparameter-optimization.md>), [shared mutable state](<https://devfeed.tech/topics/shared-mutable-state.md>)

Tags: [algorithms](<https://devfeed.tech/tags/algorithms.md>), [api](<https://devfeed.tech/tags/api.md>), [asynchronous](<https://devfeed.tech/tags/asynchronous.md>), [caching](<https://devfeed.tech/tags/caching.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [distributed-computing](<https://devfeed.tech/tags/distributed-computing.md>), [error-handling](<https://devfeed.tech/tags/error-handling.md>), [hyperparameter-optimization](<https://devfeed.tech/tags/hyperparameter-optimization.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [parallel-computing](<https://devfeed.tech/tags/parallel-computing.md>), [r](<https://devfeed.tech/tags/r.md>), [redis](<https://devfeed.tech/tags/redis.md>), [state](<https://devfeed.tech/tags/state.md>)

## AI overview

rush provides R with shared-state coordination for asynchronous, decentralized iterative algorithms. Workers use a Redis key-value store to read and write task data while running independent loops, avoiding a central controller-worker pattern. Its API manages task lifecycles and is described as offering sub-millisecond per-task overhead, automatic detection of lost workers, error handling, and caching. Optional integration with mlr3 supports asynchronous optimization through bbotk and mlr3tuning. The authors demonstrate asynchronous decentralized Bayesian optimization for LightGBM hyperparameter tuning across four datasets with 448 workers; the supplied abstract does not report benchmark outcomes.

## Source excerpt

arXiv:2606.21430v4 Announce Type: replace Abstract: Many algorithms in statistics and machine learning can be parallelized in an asynchronous manner where workers need to communicate through shared state rather than execute independent tasks dispatched by a central controller. Especially in modern hyperparameter optimization and parallel black-box optimization with expensive objectives, this decentralized approach has become widespread, and several Python frameworks adopt it (e.g., Optuna, DeepHyper, and Hyperopt). However, all popular R packages for parallel computing follow a centralized controller-worker architecture that does not support this pattern. We present rush, an R package that provides a shared-state coordination layer for asynchronously parallelized iterative algorithms. rush uses a Redis database as a shared key-value store: workers read and write task data through the database and independently execute their own loops. The package provides a high-level API for managing tasks and their lifecycle, featuring sub-millisecond per-task overhead, robust error handling with automatic detection of lost workers, and efficient caching. rush optionally integrates with the mlr3 ecosystem, powering asynchronous optimization in the bbotk and mlr3tuning packages. We demonstrate the practical utility of rush by implementing asynchronous decentralized Bayesian optimization (ADBO) and benchmarking it on hyperparameter optimization of LightGBM across four datasets using 448 workers.