Databricks has announced the availability of Delta Engine and the acquisition of Redash. These new capabilities make it faster and easier for data teams to use its Unified Data Analytics platform for data science, machine learning, and a broad range of data analytics use cases. Delta Engine is a high-performance query engine on cloud data lakes and Redash is an open source dashboarding and visualisation service for data scientists and analysts to do data exploration.
Delta Engine is tailored for use with Delta Lake, the popular open-source structured transaction layer that brings quality and reliability to data lakes. Organisations can now build curated data lakes that include structured and semi-structured data and run all their analytics on high quality, fresh data in the cloud. Databricks acquired Redash, the company behind the successful Redash open source project, to provide easy-to-use dashboarding and visualisation capabilities on these curated data lakes. With Redash, data scientists and SQL analysts can eliminate the complexity of moving data into other systems for analysis.
Together, these enhancements enable organisations to adopt a single, simplified cloud architecture for data management, helping them significantly reduce costs and complexity and accelerate data team productivity. They are also a response to the emerging “Lakehouse” design pattern, which many enterprise organisations have adopted to bring structured transactions, quality and performance to their cloud data lakes. The announcements were made today at Spark + AI Summit, which takes place this week virtually with over 60,000 members of the data community, from over 100 countries.
“Most organisations who want to do data science and data warehousing are using multiple architectures. Data is stuck in organisational silos, defined by closed and proprietary systems that slow organisations down and make it harder to arrive at high quality decisions because information is fragmented and out of date,” said Ali Ghodsi, cofounder and CEO at Databricks. “Curated cloud data lakes provide organisations a way to run any kind of analytics, including data science and machine learning, on all their most recent data. Our introduction of Delta Engine and the acquisition of Redash are significant steps forward in helping organisations build these high quality, curated data lakes that some call ‘Lakehouses’.”
