# Redis streams as a pure data structure

DevFeed: [Redis streams as a pure data structure](<https://devfeed.tech/articles/redis-streams-as-a-pure-data-structure-20619.md>)

Original publisher: [Read original article](<http://antirez.com/news/128>)

Published: 2019-03-22T15:10:15Z

Content type: article

Language: en

Sources: [Antirez](<https://devfeed.tech/sources/antirez.md>)

Topics: [Redis](<https://devfeed.tech/topics/redis.md>), [Data structures](<https://devfeed.tech/topics/data-structures.md>), [Streaming](<https://devfeed.tech/topics/streaming.md>), [CSV](<https://devfeed.tech/topics/csv.md>)

Tags: [data-structures](<https://devfeed.tech/tags/data-structures.md>), [databases](<https://devfeed.tech/tags/databases.md>), [files](<https://devfeed.tech/tags/files.md>), [kafka](<https://devfeed.tech/tags/kafka.md>), [redis](<https://devfeed.tech/tags/redis.md>), [streaming](<https://devfeed.tech/tags/streaming.md>), [streams](<https://devfeed.tech/tags/streams.md>)

## AI overview

This article presents Redis Streams as a general-purpose data structure rather than only a solution for Kafka-like messaging use cases. It introduces the idea of comparing Streams with append-only CSV files for logging structured data and discusses limitations such as inefficient range queries and redundant fields.

## Source excerpt

The new Redis data structure introduced in Redis 5 under the name of "Streams" generated quite some interest in the community. Soon or later I want to run a community survey, talking with users having production use cases, and blogging about it. Today I want to address another issue: I'm starting to suspect that many users are only thinking at Streams as a way to solve Kafka(TM)-alike use cases. Actually the data structure was designed to *also* work in the context of messaging with producers and consumers, but to think that Redis Streams are just good for that is incredibly reductive. Streaming is a terrific pattern and "mental model" that can be applied when designing systems with great success, but Redis Streams, like most Redis data structures, are more general, and can be used to model dozen of different unrelated problems. So in this blog post I'll focus on Streams as a pure data structure, completely ignoring its blocking operations, consumer groups, and all the messaging parts. ## Streams are CSV files on steroids If you want to log a series of structured data items and decided that databases are overrated after all, you may say something like: let's just open a file in append only mode, and log every row as a CSV (Comma Separated Value) item: (open data.csv in append only) time=1553096724033,cpu_temp=23.4,load=2.3 time=1553096725029,cpu_temp=23.2,load=2.1 Looks simple and people did this for ages and still do: it's a solid pattern if you know what you are doing. But what is the in-memory equivalent of that? Memory is more powerful than an append only file and can automagically remove the limitations of a CSV file like that: 1. It's hard (inefficient) to do range queries here. 2. There is too much redundant information: the time is almost the same in every entry and the fields are duplicated. At the same time removing it will make the format less flexible, if I want to switch to a different set of fields. 3. Item offsets are just the byte offset in the file: