One of the deepest challenges in running an enterprise stack of software is deciding when to stick with the old and when to switch to the new. Every line of code in the stack is getting older by the minute, and while you might not think so, the reality is that software manages to find a way to go bad, little by little.
The old code really does stop working. Partners start offering slightly different services and sometimes stop supporting features altogether. Twitter, for instance, locked out people who used its old API when the company started insisting on using the OAuth API. These stories are repeated again and again.
The trouble is that replacing the old with the new can be expensive. Programmers of the new are usually forced to maintain compatibility with old code, a challenge that often requires writing two programs: one filled with the old bugs and one filled with new ones that haven't been discovered yet.
To make matters worse, the new code is often held to higher standards. I've seen new fancy AJAX masterpieces that run much slower than old green-screen mainframe code all because they have fancy buttons and tons of images that push the video card. The look is slicker, but the feel is slower.
There is no easy answer to this dilemma. The old code still works. We like it. It's just that it's not compatible with the new version of the operating system or a new multicore chip. The new code costs money. We can usually fix a number of glaring problems with the old code, but who knows what new problems might appear?
There is one big challenge for the database administrators of the world: stick with tried-and-true SQL or switch to trendy NoSQL where everything is bigger and ready for endless streams of data.
The new NoSQL databases sound attractive. They can be much faster than older databases, and they often force users to avoid many of the problems that caused so much trouble in the first place. JOINs, for instance, can slow down a database if the schema gets too complicated. NoSQL tosses them out the window along with many parts of the schema. You can store any key-value pair you like, and the NoSQL database will come up with the answer.
But if you look closely, the NoSQL databases aren't always so wonderful. First, they often don't offer guarantees that the data will be recorded. It probably will be OK, but not if something happens to a hard drive or a computer in the cluster. Some of the newer NoSQL options from companies like Oracle allow you to ask for a transaction confirmation, but your code will need to twiddle its thumbs and wait just like the code that uses a SQL database.
There are deeper issues. Many of the speed problems came about because programmers didn't think about the subtle effects of SQL. The way you structure your tables and queries can make a big difference in performance. Linking together multiple tables and forcing the database to JOIN the information slows things down.
But if you try to accomplish the same thing with a NoSQL database, you'll often be writing data in and out of multiple places and hoping it will all stay consistent. You get to do all of the work of JOINing disparate sections of the database, and that probably means you'll pay the cost in speed. If you are aware of this and are able to think through the trade-offs when designing code, you'll be OK. But if you're not, you may find that your code is even slower and buggier. The database won't enforce the transactions, and you'll need to do it yourself.
This dilemma has a simple answer: Applications that need better consistency should rely upon the transaction guarantees of older SQL machinery. Applications that need speed and can handle a few scrambled records can choose the newer NoSQL datastores. But if you need speed and consistency, you might as well start pulling out your hair.