With the RDBMS Ailing … What's Next?
Over the past couple of years we’ve come across application after application where the biggest block to being able to utilize a cloud (public or private) has been the relational database. This usually goes hand in hand with an inability to scale.
In a very (Monty) Python-esque manner RDBMS proponents proclaim “I’m not dead yet”, and the thing is that they’re absolutely right … but they are beginning to ail a bit.
There are many reasons for this, but mostly the cracks in the RDBMS monolith have a very simple explanation – the relational database is only one – albeit one powerful and long-lived – storage abstraction., and is not the best choice for many common, everyday problems.
We Can Do Better
In other words, there are many common, everyday problems in which the data can be effectively stored, managed, and retrieved in other abstractions …
… perhaps abstractions that fit the problems themselves much better, and may well be far more “cloud friendly”.
If you’ll grant me that for now (and we will get back to this question in future posts), then the next question is inevitable:
for those applications where the RDBMS is not the best choice, what else should I consider?
A Great Question
As life would have it, I think for anyone thinking about cloud-friendly storage abstractions this is the real question. Several friends (including @KentLangley and my colleague @msgroner) pointed out an excellent overview post by last.fm founder Richard Jones. From the intro:
Perhaps you’re considering using a dedicated key-value or document store instead of a traditional relational database. Reasons for this might include:
1. You’re suffering from Cloud-computing Mania.
2. You need an excuse to ‘get your Erlang on’
3. You heard CouchDB was cool.
4. You hate MySQL, and although PostgreSQL is much better, it still doesn’t have decent replication. There’s no chance you’re buying Oracle licenses.
5. Your data is stored and retrieved mainly by primary key, without complex joins.
6. You have a non-trivial amount of data, and the thought of managing lots of RDBMS shards and replication failure scenarios gives you the fear.
At any rate the post by Jones is a good place to get started on non-relational basic name-value stores.
This whole business of storing data in cloud-scale apps has become one of my favorite area for discussion – I can promise you more posts to come!
Leave a Reply
Available Now!
"Possibilities. Realities. Action"
now this whole computing thing starts becoming rather significant ...
WE HAVE THE ALTERNATIVE!!!!
I am glad, at long last, the real limitations of RDBMS are being exposed in full day light by ‘cloud computing’. It was obvious, even earlier, to all web service developers. RDBMS, of whatever make, just dragged its feet and slowed down transactions in any kind of online computing or web service beyond a point.
Yet, in the recent past attention was diverted more to the lack of object orientation (at the cost of real solutions for real problems). Object oriented database vendors emerged. Existing RDBMS vendors came up with object relational offerings. Yet database back ended web services left a lot to be desired.
Many sought to overcome the sluggishness of RDBMS by adding processing power and cache memory. Everyone admitted that scalability is a barrier. Key store is another attempt to relieve web services of the limitations of RDBMS.
As experience has shown OODBMS, ORDBMS, and Key stores have not managed to provide effective solutions. This is what the cloudsters have to overcome
Scalability was the main issue, when I set out to maintain online service front end for a business information database in 1993. The database which was performing alright up to 500,000 records began flailing badly just few thousand records thereafter. The funny part of the RDBMSscalability constraint is that the speed would fall exponentially once you hit the max.
This is what made me to look for alternatives. As usual RDBMS vendors would announce cheekily that they never promised things what their software cannot do. They can show you how they have not mentioned scalability as a performance guarantee.
It was out of necessity I had to test out alternative ways of handling data within the RDBMS. After 15 years of research, development and testing I do have now an alternative way of handling data inside an RDBMS which relieves it of the scalability crunch. To prove its viability, I have also developed and implemented a few knowledge base applications having KM and BI components. Hopefully, we would get more projects and thus show to the world that scalability is not an issue in RDBMS, provided one uses my architecture.
The point I want to highlight here is that one needs to do a thousand things to make RDBMS overcome its scalability limitation. Literally yes. There are at least 5 or 6 major fundamental changes that one would need. One of the main things to do is to bid good bye to primary and foreign key architectures and move away from imposing uniqueness constraint and indexing. You might ask what is left in RDBMS, if one is not using these basic features. You can simply move away and yet achieve far better efficiency, scalability and most of all incisive querying.
We would be happy to develop prototypes for anyone stuck with scalability issues. In fact, our brochure clearly says: ” Please ignore this brochure, if you are happy with your current RDBMS implementations and vendors. Perhaps, the same holds good for this comment on this blog too.
Nice summary – another candidate for leveraging existing RDBMS’s is through the use of in-memory data grids. I touch on this in my post about the NoSQL movement in the context of legacy systems:
http://bigdatamatters.com/bigdatamatters/2009/07/nosql-vs-rdbms.html