+972 52-548-6969
That No SQL Thing – Key / Value stores – Usages
Now that we have a fairly good understanding of what Key / Value stores are, what are they good for? Well, they are really good if you need to lookup things by key :-)
Storing the user’s session is a common example, or the user’s shopping cart. In fact, per user’s data is probably the most common usage for a key value store. The reasons for that are fairly obvious, I would imagine, we have the user’s id already, and we need a well known data. That make it very cheap to access, modify and store at the end of a request.
Other usages include storing already processed data for quick retrieval, consider things like: Favorite Products, Best Of …, etc. The idea here is to have the application access the key / value store using well known keys for those data items, while background processes will update those periodically. It isn’t quite a cache, because there isn’t really a scenario where the value doesn’t exists, but it is a near thing.
One interesting use case that I run into was to store the product catalog in the key/value store, that allowed pulling a particular product’s details in a single call. The key/value store was also used to track how many units were physically in stock, so it was able to tell if the item was available for purchase or not.
Just remember, if you need to do things more complex than just access a bucket of bits using a key, you probably need to look at something else, and the logical next step in the chain in the Document Database.
Comments
Interesting ideas.
Could I use a combination of Key / Value store and for example document db, where you store user session data, shopping cart etc in the K/V store, and your "domain model" in a document db?
Henning,
Yes, that is possible, but the fun part about most document dbs is that they are already key/value stores :-)
Ahh, awesome.
I'm starting to look into MongoDB now. Looks very promising, although it requires a bit of a brainshift in the way I think about storage.
I would say that only useful for accessing 'a bucket of bits using a key' is only true for primitive key value stores like Memcached.
There are new NoSQL databases (like Redis and MongoDB) with a lot more advanced features opening up a new suite of use-cases:
At mflow (apart from maintaining intelligent in-memory data views) we are using Redis to store rolling error logs (important for maintaining a fast, combined chronological ordered list of error logs for our load-balanced SOA web services) and as a fast in-memory Message Queue for persisting and load-balancing async web service requests.
There are a number of other use-cases for where it is the best solution, like being the optimal data store for high-performance 'web streaming' real-time notification comet servers, that when used in conjunction with a technology like nodejs.org solves the 10k connection problem.
A good example of advanced software that utilizes Redis as a backend that I just noticed on the twitter-sphere today is:
http://www.ohmstudio.com/
Who provide software allowing you to collaboratively compose music online. Their demo video is impressive.
When I read about how people mention that they actually used a non-RDMS in production, the story usually starts off hopeful and then descends into regret and despair.
@Mark Rogers
[Citation Needed]
@Mark Rogers I think like any technology there is a risk to over use it. Devs might think hey here's this shiny new tech that's x times faster than our RDBMS and will solve all of our performance and scalability issues only to realize half-implemented way down the line that they have a suite of new problems and actually need that lovely query language in-built into their RDBMS.
Personally I don't view NoSQL data stores as a replacement technology. Rather I believe they make a good complementary technology and are great when they have in-built support for your Use-Case. As Oren has pointed out most key value stores are a good fit when you want to 'just access a bucket of bits using a key'.
Personally we've had great success at mflow.com and have provided a much better and more responsive user experience as a result of utilizing Redis whenever it made sense to. Most of our web services end up hitting optimized 'in-memory data views' which mirrors data that ultimately resides in our sharded RDBMS databases. If you're interested in checking out the user experience said technology and you live in the UK, you can access our closed-beta and as a bonus 'download any free track of your choice' with my limited invite codes attached to this url (we're kinda like social music service that's kinda like itunes+twitter+spotify):
http://www.mflow.com/download/index/mythz007
Hmmm, if I put two and two together, I'd say that a Key-Value store combined with something like Lucene is a really good option for a lot of things. I'm probably wrong here, but as far as I understand, it works a bit like this right?
Lucene walks your data, generating indexes (by key), and allows very quick searches by using these indexes to find the key.
A key-value store uses keys, obviously.
So to search for something, you use Lucene to find the keys, and fetch them from your KV store.
So searching shouldn't be a problem like that, right? You're probably going to prove me wrong with the next one in the series about document databases showing a far easier way though ;-)
@Erik yeah that would be golden, and it looks like Oren's heading down that path with his Divan Db.
The problem is that if Lucene walks your data in a background task than you could potentially have stale data in the Search Index. The ideal solution would be something along the lines of Oren's solution where the Search Index is updated in real-time (or close to real-time i think he quoted 25ms) preferably using queue technology.
Erik,
Yes and no.
You would need some way to update the lucene index, and if you need to do multi key access, that is likely to be expensive.
For things like that, I would rather use a doc db, yes.
Demis,
Raven's does that, yep. It provide two things, immediate background indexing and explicit notification about whatever the indexed results are stale or not.
Rob,
I am going to respond to that post in a separate post
Comment preview