# Rain: A key-value store for Strava's scale

DevFeed: [Rain: A key-value store for Strava's scale](<https://devfeed.tech/articles/rain-a-key-value-store-for-strava-s-scale-26575.md>)

Original publisher: [Read original article](<https://medium.com/strava-engineering/rain-a-key-value-store-for-stravas-scale-7f580f5b4848?source=rss----89d4108ce2a3---4>)

Author: Derick Yang

Published: 2025-01-24T18:02:11Z

Content type: article

Language: en

Sources: [Strava Engineering](<https://devfeed.tech/sources/strava-engineering.md>)

Topics: [data](<https://devfeed.tech/topics/data.md>), [datasets](<https://devfeed.tech/topics/datasets.md>), [Amazon S3](<https://devfeed.tech/topics/amazon-s3.md>), [Apache Cassandra](<https://devfeed.tech/topics/cassandra.md>), [Deployment](<https://devfeed.tech/topics/deployment.md>), [Databases](<https://devfeed.tech/topics/databases.md>)

Tags: [caching](<https://devfeed.tech/tags/caching.md>), [cassandra](<https://devfeed.tech/tags/cassandra.md>), [data](<https://devfeed.tech/tags/data.md>), [data-pipeline](<https://devfeed.tech/tags/data-pipeline.md>), [databases](<https://devfeed.tech/tags/databases.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [key-value-store](<https://devfeed.tech/tags/key-value-store.md>), [maps](<https://devfeed.tech/tags/maps.md>), [memory](<https://devfeed.tech/tags/memory.md>), [performance](<https://devfeed.tech/tags/performance.md>), [production](<https://devfeed.tech/tags/production.md>), [s3](<https://devfeed.tech/tags/s3.md>), [scale](<https://devfeed.tech/tags/scale.md>), [spark](<https://devfeed.tech/tags/spark.md>)

## AI overview

Strava's Geo and Metro teams work with large, immutable datasets derived from map, GPS, and third-party data. The article describes the challenges of serving these datasets and the limitations of using PalDB and Cassandra, including slow deployments, duplicated memory usage, cost constraints, and schema inflexibility.

## Source excerpt

Much of our heatmaps are built on batch data outputs stored in Rain At Strava, we love maps -- some of our most loved features are nestled on map surfaces. My team, the Geo team, is focused on building and improving these products. On the Geo and Metro teams, we tend to work with large datasets: aggregations of open source map data via OpenStreetMaps, GPS data points from uploaded activities, third-party datasets for properties like elevation, and beyond. This aggregated dataset eventually turns into Geo features we know and love, like the global heatmap, Strava Metro, the routing product, route suggestions, elevation profiles, and points of interest. We perform these data aggregations in a rather hefty data pipeline, run on a regular cadence to ensure we're serving up-to-date geo data. One of the Geo team's key challenges is efficiently serving large, immutable (write-once, read-many) datasets produced by our pipeline. This is particularly hard for compute-intensive use cases like routing, where: Write-Optimized vs. Read-Optimized Conflict: Traditional read-optimized data stores struggle with large batch writes without impacting read performance or introducing significant operational complexity. Cost Constraints: Storing rarely accessed datasets in production databases can be prohibitively expensive -- especially for projects like Strava Metro, which are accessed sporadically. Schema Complexity: Defining schemas externally from the service that uses them can be costly and inflexible for developers. Previously Our previous solution to large writes used a combination of datastores: PalDB and Cassandra. PalDB is a binary data format ideal for small datasets. The README states that it is optimized for "side data", relatively small datasets you read "in passing" on your service. PalDB is unideal, however, for larger datasets. In our case, since each service deployment required downloading the key-value file from S3, our deployments were taking upwards of twenty minutes. S