Redis vs MySQL vs Tokyo Tyrant (on EC2)


Update (28th April 2009) – Added Tokyo Tyrant

What is Redis?

Briefly, Redis is a key-value store that stores the values to disk periodically instead of after every set.

What is Tokyo Tyrant?

Tokyo Tyrant is similar to Redis – a key-value store.

Aim

This is a brief comparison of the performance of Redis and MySQL. It is not very thorough and is only to be used as an indication as to whether Redis may be worth your further investigation.

After a brief search I failed to find any performance comparisons of Redis and MySQL to determine if it was even worth my time looking at Redis. I appreciate they are very different beasts but in our database we have a lot of data that would fit very nicely into a key-value store.

So, the question I am trying to answer is “roughly how much performance can we gain from using Redis for storing simple data instead of MySQL?”. Then it is up to you to look at all the differences and decide whether it is worth pursuing a switch to Redis.

Note: I will be looking at other key-value stores in the near future :-)

The Test

The two tests performed are:

  • Using 2 threads: set X items with the keys/values being consecutive numbers
  • Using 2 threads: randomly get these items Y times

Setup

  1. EC2 small instance.
  2. Default install of MySQL with a single InnoDB table for storing keys and values.
  3. Default make of Redis.
  4. Small Java benchmark programs sitting on the same box

Source Code

The numbers in these files are not the exact numbers I used (apart from using two client threads in both tests)

  1. Redis benchmark: http://pastebin.com/f5adb41b0
  2. MySQL benchmark: http://pastebin.com/m52fd35d1

Results

Mysql

3,030 sets/second.

4,670 gets/second.

Redis (In-memory only)

11,200 sets/second. (3.7x MySQL)

9,840 gets/second. (2.1x MySQL)

Tokyo Tyrant (Writing to disk)

9,030 sets/second. (3.0x MySQL)

9,250 gets/second. (2.0x MySQL)

Thoughts

Key-value stores are generally going to be faster than MySQL. They (generally) provide less functionality in the way of querying arbitrary columns and ACID compliance. This translates into faster performance on gets/sets.

Once again, it is worth pointing out that this is a very simple benchmark. My suspicion is that when you have lots of cross updates going on then Redis/Tyrant will have a greater performance boost over MySQL – but at the cost of transactionality.

My suspicion is that as the technology matures we are going to start seeing a big shift towards using key-value stores as the first port of call for data storage for websites. I expect it won’t be long before we start seeing hosts offering LAKP (Linux, Apache, a key-value store, PHP) as well as the more traditional LAMP offerings.

I would love to hear from anyone with real world experience and numbers of switching over to Redis… until I have some of my own to share anyway :)

About these ads

24 Responses to Redis vs MySQL vs Tokyo Tyrant (on EC2)

  1. Julien says:

    “My suspicion is that when you have lots of cross updates going on then Redis/Tyrant will have a greater performance boost over MySQL – but at the cost of transactionality”

    Tokyo Tyrant having native support for Lua procedures, you don’t need transactions anymore.

    Interesting benchmark! on-disk TT being as fast as in-memory Redis is awesome, and I’m now made my choice. Thanks!

  2. Have you looked at http://randomfoo.net/2009/04/20/some-notes-on-distributed-key-stores and the comments about MySQL Cluster in particular? I have a feeling you are using “vanilla MySQL” and comparing it to Redis, when you should really be comparing Redis to MySQL Cluster (uses Ndb Cluster)

    Do you think you could repeat the test with MySQL Cluster?

    • colinhowe says:

      I am indeed using Vanilla MySQL – I hadn’t heard of MySQL Cluster until your comment! Are you aware of any benchmarks that compare MySQL Cluster to Vanilla MySQL? I’ve had a look around but can’t find any.

      When I get some more time I’ll have a look at comparing the two.. MySQL Cluster looks interesting! :-)

    • ActsAsFlinn says:

      True comparing Redis to NDB would be more accurate since they are both in-memory, that’s not the case for Tyrant.

      • From what I’ve read to far, yes, Redis needs to have all data in memory. But I think that is not the case with MySQL Cluster at all!

      • ActsAsFlinn says:

        Otis NDB are similar to Redis in that they put the DB in memory and do checkpoint disk synchronization. Any amazing benchmark Oracle/Sun/MySQL are touting are based on in-memory configurations. Like the comments note in the article you link to previous to 5.1 the entire db was required to be in-memory.

      • But they all load/cache some data into memory – they have to. So are you saying MySQL requires *all* data to be memory resident at all times?

        Checkpointing is fine, too, for lots of people/applications. I think the comment in that other blog post said checkpointing happens every 2 seconds, which means you could lose 2 seconds-worth of data, which is no big deal for some applications.

      • ActsAsFlinn says:

        At this point NDB doesn’t require the entire database in memory, Redis does (at least right now, I have a feeling it will change).

        I think the point is MySQL Cluster is significantly more complex a configuration compared to a vanilla Redis or Tyrant install. Out of the box without tuning both of those solutions are faster in terms of performance and easier to setup.

        All that said, all three of those solutions occupy entirely different segments. MySQL is RDBMS while Redis and Tokyo Tyrant are networked hash dbs. Tokyo Cabinet is still yet different as an embeddable hash storage.

        All of these speed comparisons are nice because they get people to think about other solutions but they are all comparing apple to oranges.

  3. I’m not aware of any such comparisons. If you remember, it would be great if you could leave a comment here when you do publish the MySQL vanilla vs. cluster benchmarks, so anyone following this through comments can find out about it.
    Thanks!

  4. Krut says:

    What makes Redis special, and in certain use cases, makes it an obvious choice, is it’s support for lists and sets. If you want to benchmark mysql getting it’s but handed to it (nbd or not), try to intersect a few million row tables, vs a few million member Redis sets…

    • This is irrelevant in a concurrent environment – try your test with 50+ concurrent users and you’ll see that intersecting millions of members in redis sets wil only lead to slow serialization since Redis is single-thread – MySQL is not.

  5. Tim says:

    FYI, I’ve also made a performance test for Redis, Tokyo Tyrant and MemcacheDB.

    http://timyang.net/data/mcdb-tt-redis/

  6. [...] you like to saw other benchmark for other Key-Value systems check this links: – Redis vs MySQL vs Tokyo Tyrant (on EC2) – Redis Benchmark Share and [...]

  7. [...] Redis vs MySQL vs Tokyo Tyrant (on EC2) « Colin Howe’s Blog [...]

  8. [...] Redis vs MySQL vs Tokyo Tyrant (on EC2) [...]

  9. Will Radford says:

    Looking at the latest version of TT (1.1.34), you should be able to start it as an ‘on-memory’ hash or tree database:

    $ ttserver ‘*’ # hash
    $ ttserver ‘+’ # tree

    I’d be interested to see how these stacked up against Redis.

  10. [...] Redis vs MySQL vs Tokyo Tyrant (on EC2) « Colin Howe’s Blog [...]

  11. hc says:

    FLKC = FreeBSD + lighttpd + key value store + (ANSI) C
    :-)

  12. [...] to use Redis as the data store, after hearing lot of good about it and further seeing some impressive [...]

  13. Lee says:

    Was she/he well-behaved?

  14. fopriasia says:

    I recently found this website that I want to share with everyopne as they were a remarkable help when I was
    working on a recent project. I needed to know how to do some stuff in PHP and they were able to provide
    a tutorial sepcifically tailored to my needs. It was awesome.

    TeamTutorials

    I recommend you hit them up and check them out if you need assistance.

  15. Appears it is pretty energetic right here. Quite a few of these messages on this post appear to be junk, You should filter them out.

  16. This is irrelevant in a concurrent environment – try your test with 50+ concurrent users and you’ll see that intersecting millions of members in redis sets wil only lead to slow serialization since Redis is single-thread – MySQL is not.

  17. Sorry previous comment was a reply to Krut

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: