# Streams: a new general purpose data structure in Redis.

DevFeed: [Streams: a new general purpose data structure in Redis.](<https://devfeed.tech/articles/streams-a-new-general-purpose-data-structure-in-redis-20605.md>)

Original publisher: [Read original article](<http://antirez.com/news/114>)

Published: 2017-10-02T15:12:35Z

Content type: article

Language: en

Sources: [Antirez](<https://devfeed.tech/sources/antirez.md>)

Topics: [Redis](<https://devfeed.tech/topics/redis.md>), [Data structures](<https://devfeed.tech/topics/data-structures.md>), [Messaging](<https://devfeed.tech/topics/messaging.md>), [Temporal data](<https://devfeed.tech/topics/temporal-data.md>), [Kafka](<https://devfeed.tech/topics/kafka.md>)

Tags: [data-structure](<https://devfeed.tech/tags/data-structure.md>), [data-structures](<https://devfeed.tech/tags/data-structures.md>), [messaging](<https://devfeed.tech/tags/messaging.md>), [redis](<https://devfeed.tech/tags/redis.md>), [streams](<https://devfeed.tech/tags/streams.md>), [time-series](<https://devfeed.tech/tags/time-series.md>)

## AI overview

The article explains why Redis needed a stream data structure to fill gaps between lists, sorted sets, and Pub/Sub. It discusses tradeoffs involving message history, blocking reads, fan-out, element identifiers, and range queries, including time-series use cases.

## Source excerpt

Until a few months ago, for me streams were no more than an interesting and relatively straightforward concept in the context of messaging. After Kafka popularized the concept, I mostly investigated their usefulness in the case of Disque, a message queue that is now headed to be translated into a Redis 4.2 module. Later I decided that Disque was all about AP messaging, which is, fault tolerance and guarantees of delivery without much efforts from the client, so I decided that the concept of streams was not a good match in that case. However, at the same time, there was a problem in Redis, that was not taking me relaxed about the data structures exported by default. There is some kind of gap between Redis lists, sorted sets, and Pub/Sub capabilities. You can kindly use all these tools in order to model a sequence of messages or events, but with different tradeoffs. Sorted sets are memory hungry, can't model naturally the same message delivered again and again, clients can't block for new messages. Because a sorted set is not a sequential data structure, it's a set where elements can be moved around changing their scores: no wonder if it was not a good match for things like time series. Lists have different problems creating similar applicability issues in certain use cases: you cannot explore what is in the middle of a list because the access time in that case is linear. Moreover no fan-out is possible, blocking operations on list serve a single element to a single client. Nor there was a fixed element identifier in lists, in order to say: given me things starting from that element. For one-to-many workloads there is Pub/Sub, which is great in many cases, but for certain things you do not want fire-and-forget: to retain a history is important, not just to refetch messages after a disconnection, also because certain list of messages, like time series, are very important to explore with range queries: what were my temperature readings in this 10 seconds range? The way