The data lake is becoming the new data warehouse

Platforms like AWS Lake Formation and Delta Lake point toward a central hub for decision support and AI-driven decision automation

Are data warehouses relevant again, or are they a dying breed?

You’re forgiven if you’re a bit confused on this issue. On the one hand, data warehousing certainly seems to be on a hot streak. As a longtime industry observer, I’ve seen the industry surge in successive waves of innovation and startup activity.

This trend essentially began when the appliance form factor entered the data warehousing mainstream a decade ago, and then gained new momentum several years ago as the market shifted toward the new generation of cloud data warehouses. In the past few years, one cloud data warehouse vendor—Snowflake—has gained an inordinate amount of traction in the marketplace.

The eclipse of the data warehouse

On the other hand, data warehousing keeps getting eclipsed by new industry paradigms, such as big data, machine learning, and artificial intelligence. This trend has fostered the impression that data warehousing is declining as an enterprise IT priority, but in fact most organizations now have at least one and often multiple data warehouses serving various downstream applications.

The persistence of data warehousing as a core enterprise workload is why, several years ago, I felt I had to contribute my thoughts on why the data warehouse is far from dead. It also probably explains why other observers felt they had to redefine the concept of the data warehouse to keep it relevant in the era of the data lakes and cloud computing.

To continue reading this article register now