开发者

Will Redis's sorted sets scale?

开发者 https://www.devze.com 2023-03-26 04:35 出处:网络
This may be more of a theoretical question but I\'m looking for a pragmatic answer. I plan to use Redis\'s Sorted Sets to store the ranking of a model in my database based on a calculated value. Curr

This may be more of a theoretical question but I'm looking for a pragmatic answer.

I plan to use Redis's Sorted Sets to store the ranking of a model in my database based on a calculated value. Currently my data set is small (250 members in the set). I'm wondering if the sorted sets would scale to say, 5,000 members or larger. Redis claims a 1GB maximum value and my values are the ID of my model so I'm not really concerned about the scalability of the value of the sorted set.

ZRANGE has a time complexity of O(log(N)+M). If I'm most frequently trying to get the top 5 ranked items from the set, log(N) of N set items might开发者_如何学Go be a concern.

I also plan to use ZINTERSTORE which has a time complexity of O(N*K)+O(M*log(M)). I plan to use ZINTERSTORE frequently and retrieve the results using ZRANGE 0 -1

I guess my question is two fold.

  1. Will Redis sorted sets scale to 5,000 members without issues? 10,000? 50,000?
  2. Will ZRANGE and ZINTERSTORE (in conjunction with ZRANGE) begin to show performance issues when applied to a large set?


I have had no issues with hundreds of thousands of keys in sorted sets. Sure getting the entire set will take a while the larger the set is, but that is expected - even from just an I/O Standpoint.

One such instance was on a sever with several DBs in use and several sorted sets with 50k to >150k keys in them. High writes were the norm as these use a lot of zincrby commands coming by way of realtime webserver log analysis peaking at over 150M records per day. And I'd store a week at a time.

Given my experience, I'd say go for it and see; it will likely be fine unless your server hardware is really low end.


In Redis, sorted sets having scaling limitations. A sorted set cannot be partitioned. As a result, if the size of a sorted set exceeds the size of the partition, there is nothing you can do (without modifying Redis).

Quote from article:

The partitioning granularity is the key, so it is not possible to shard a dataset with a single huge key like a very big sorted set[1].

Reference:

[1] http://redis.io/topics/partitioning

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号