# Why we don't have benchmarks comparing Redis with other DBs

DevFeed: [Why we don't have benchmarks comparing Redis with other DBs](<https://devfeed.tech/articles/why-we-don-t-have-benchmarks-comparing-redis-with-other-dbs-20676.md>)

Original publisher: [Read original article](<http://antirez.com/news/85>)

Published: 2015-01-29T09:21:41Z

Content type: opinion

Language: en

Sources: [Antirez](<https://devfeed.tech/sources/antirez.md>)

Topics: [Redis](<https://devfeed.tech/topics/redis.md>), [Databases](<https://devfeed.tech/topics/databases.md>), [benchmarking](<https://devfeed.tech/topics/benchmarking.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>), [Amazon EC2](<https://devfeed.tech/topics/amazon-ec2.md>)

Tags: [aws](<https://devfeed.tech/tags/aws.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [compare](<https://devfeed.tech/tags/compare.md>), [database](<https://devfeed.tech/tags/database.md>), [ec2](<https://devfeed.tech/tags/ec2.md>), [redis](<https://devfeed.tech/tags/redis.md>)

## AI overview

The article explains why Redis does not publish benchmarks comparing it with other databases. It argues that fair comparisons require matching data models, durability guarantees, replication safety, availability, protocols, and client libraries. It uses a Redis and AerospikeDB benchmark to show how hardware and test conditions can produce misleading results.

## Source excerpt

Redis speed could be one selling point for new users, so following the trend of comparative "advertising" it should be logical to have a few comparisons at Redis.io. However there are two problems with this. One is of goals: I don't want to convince developers to adopt Redis, we just do our best in order to provide a suitable product, and we are happy if people can get work done with it, that's where my marketing wishes end. There is more: it is almost always impossible to compare different systems in a fair way. When you compare two databases, to get fair numbers, they need to share *a lot*: data model, exact durability guarantees, data replication safety, availability during partitions, and so forth: often a system will score in a lower way than another system since it sacrifices speed to provide less "hey look at me" qualities but that are very important nonetheless. Moreover the testing suite is a complex matter as well unless different database systems talk the same exact protocol: differences in the client library alone can contribute for large differences. However there are people that beg to differ, and believe comparing different database systems for speed is a good idea anyway. For example, yesterday a benchmark of Redis and AerospikeDB was published here: http://lynnlangit.com/2015/01/28/lessons-learned-benchmarking-nosql-on-the-aws-cloud-aerospikedb-and-redis/. I'll use this benchmark to show my point about how benchmarks are misleading beasts. In the benchmark huge EC2 instances are used, for some strange reason, since the instances are equipped with 244 GB of RAM (!). Those are R3.8xlarge instances. For my tests I'll use a more real world m3.medium instance. Using such a beast of an instance Redis scored, in the single node case, able to provide 128k ops per second. My EC2 instance is much more limited, testing from another EC2 instance with Redis benchmark, not using pipelining, and with the same 100 bytes data size, I get 32k ops/sec, so my instance