The numbers involved in web scale computing at turns excite, amaze, dazzle, and frighten me. 3,500 queries a second times four servers. Crikey. I know at least one system that cost a million dollars which doesn't see 3,500 queries a week.
And then you look over at Facebook or Google and a system munching through five digits of writes in a second looks like a quaint little toy.
So it's actually each of the server that sees this traffic :)
And yes, we're nothing (yet!) compared to what the big guys just. However, most of the tim they used their very own software to achieve that; I'm just using out-of-the-box redis, which makes this available to _anyone_.
I get a real kid-in-a-candy-store feel from OSS these days, because it really, really is bringing down barriers for making wonderful things. Facebook solves some of the hardest engineering challenges the software industry has ever known. I sell bingo cards to sixty year-old women. We both use the exact same Memcached. How cool is that? (OK, so they might have about four terabytes more RAM attached to it than I do, give or take, but that's just an implementation detail. :) )
Ok I revise my earlier opinion [1]. This is definitely one of the best posts I've read for the last few weeks. Engaging, front line, full of content (i.e. links to their code and other bits) and pleasantly written. More of the same please :)
EDIT: this is the first major posting I have noticed that pushes redis as a big big technnology. has anyone see other people using it? I ask because at the time Antirez first posted it my feeling was redis has the scope to be a really big, important tech: and this looks to be a step in that direction.
Thanks man, I'm honored. I wish I had more time to write on more of our infrastructure stuff!
I think many people use redis, but don't necessarily advertise it, which is also one of the reasons of our "coming out" :)
Here's a question: if you have a barebones join table like
create table feed_items (item_id integer, feed_id integer, created_at datetime);
create index idx1 on feed_items (feed_id, datetime, item_id);
how much slower is that covering index for your query of
select item_id from feed_items where feed_id=42 order by datetime desc limit 10
than the mysql/memcached/redis trifecta, and why not upgrade your database machine to a 68GB EC2 instance? Writing's definitely going to be slower, but you can batch transactions every 10 seconds or so, and 9x the RAM will be a big help.
The first bottleneck on the MySQL set-up is the reading, not the writing. Even with the indices stored entirely in RAM (as the author pointed out in his blog post), the MySQL box can't keep up. (Consider the workflow of a software system that needs to parse hundreds of thousands of feeds on a continual basis. Each feed you download will have dozens of entries, and most of the time you'll have already processed the entry, so it can be ignored -- but the only way to know is to do a look-up.)
Regardless, A 68GB EC2 instance costs ~$1700/month. Four 2GB slices on Slicehost are priced at $520/month, with the cheaper set-up performing significantly better.
I think the only way to is add RAM when your indices grow.. that's what we did up until now. There is absolutely now way a data store to keep this kind of performance as soon as you involve a disk, so, yes, the secret of whatever datastore is to _always_ keep everything in memory.
And then you look over at Facebook or Google and a system munching through five digits of writes in a second looks like a quaint little toy.