In my everyday work, I'm often thinking about this question. I have a theory regarding the current system design which we use in nearly 90% of all of our systems. Most of the codes are now using the Object Oriented programming model. If we utilize an RDBMS, to any kind of system we always construct and flatten objects. My personal opinion is that this kind of activity consumes more than the used computation task.
With the evolution of the Object Based storages, we can store and retrieve the objects with a single call, we don't need to do an unnecessarily needed conversion. I think that for these kind of tasks the CEPH initiative is a good start. With CEPH we can provide the redundancy, and replication at the storage level.
The problematic question is always how we can query the objects itself based on the chosen attributes. For this purpose, we can build in-memory indexes. The in-memory indexes can be maintained based on the code which runs on top of this database.
If we think about the MapD database: http://www.mapd.com it shows that we can utilize the GPU for special tasks. With this effort the index scan rate can be scaled to 3-4 TB/s rates per node, with high grid architecture to hundreds of TBs.
Of course, if we can reduce the layers of the code to the data storage it helps a lot. Depending on the volume size, we can balance latency vs. Throughput. For high throughput I prefer GPU, for low latency I prefer NV-DIMM technology.
With the above considered, is there any future for RDBMS databases, or it is time to move forward to the Object Database Era.
If I need to bet on technology I see much more of a future in Object Storage technology. I know this is a bad news for Software Vendors, but for Innovative Companies a good start to open for the future...
I would be interested if anyone has a good comparison of performance difference on the two technologies.
