Databricks Unveils Agentic Upgrades to Lakeflow Data Engineering Platform

Databricks announced the next major evolution of Databricks Lakeflow at the Data + AI Summit. [1] Lakeflow is a unified platform for data engineering from ingestion, to transformation and orchestration. [1] All Lakeflow capabilities are fully integrated and centrally governed by Unity Catalog. [1] This unified architecture enables agents not only to build but also to operate data pipelines. [1]

Users can use Genie Code to create ingestion connectors, build pipelines in Python and SQL and develop jobs with tasks, triggers and dependencies. [1] Now generally available, Lakeflow Designer empowers teams to develop pipelines using a drag-and-drop canvas and natural language prompts. [1] Business analysts and non-technical users can build production-ready ETL pipelines without writing code. [1] Every visual Flow built in Designer natively runs on a production-ready Spark Declarative Pipeline. [1]

The launch of Lakeflow Designer targets a persistent challenge in enterprise data management by enabling non-technical users to build ETL pipelines natively on Spark without writing code. By integrating agentic capabilities directly into ingestion and orchestration workflows, Databricks aims to simplify complex, fragmented data stacks that have historically required extensive manual maintenance across a range of use cases and user personas. The native translation to Spark Declarative Pipelines also bridges the gap between business analysts and data engineers, allowing the latter to review generated code directly in place without switching context. What the sources don't address: How the pricing and consumption structure will change for existing customers adopting these newly integrated agentic automation features.