Databricks Details Autoscaling Architecture for Lakebase Postgres

Lakebase Postgres omits the database sizing experience altogether thanks to autoscaling. [1] Instead of requiring developers to manually provision specific hardware for peak workloads, the platform dynamically adjusts resources on the fly. [1] The system's autoscaling responsiveness comes from in-place VM resizing and an algorithm that tracks CPU, memory, and the database’s working set. [1]

Traditional Postgres runs as a stateful process tied to a machine and its disks. [1] The Lakebase Postgres architecture separates those responsibilities by dividing the system into distinct compute and storage layers. [1] The compute layer runs Postgres and executes queries using RAM and local NVMe, but owns no durable state. [1] The separate storage layer owns durability and history, utilizing safekeepers, pageservers, and object storage. [1] Because of this architectural foundation, a compute node can start, stop, move, or change size without moving the database underneath it. [1]

The separation of compute and storage in Lakebase Postgres aligns with modern cloud-native database design, moving away from monolithic, stateful architectures. By making the compute layer stateless and reliant on local NVMe for low-latency query execution, Databricks enables seamless vertical scaling without the downtime typically associated with migrating databases to larger instances. This directly addresses the compute waste inherent in provisioning for peak database workloads upfront. What the sources don't address: How the autoscaling algorithm handles sudden, extreme traffic spikes before the in-place VM resizing can complete.