Right this moment marks an vital milestone for open requirements and the broader information neighborhood. We’re working with Microsoft to combine the information managed each in Unity Catalog and OneLake utilizing open requirements. The Unity Catalog Open API specification offers an interface to trade information each methods, zero copy, to open up entry to the complete lakehouse ecosystem for engines in both platform.
For practically a decade, Databricks and Microsoft have partnered to assist organizations unlock the worth of their information by the openness and suppleness of the lakehouse. Right this moment, that partnership takes one other step ahead, giving clients extra alternative in how they entry and govern information throughout platforms.
The Path to Open Interoperability
As open desk codecs like Delta Lake and Iceberg gained adoption, new silos emerged on the catalog-level. To handle this, Databricks launched Unity Catalog Open APIs, establishing an open commonplace for unified governance throughout information and AI, whereas additionally absolutely supporting the Iceberg REST catalog API for tabular information. This dual-API strategy empowers organizations to undertake best-in-class options and keep away from vendor lock-in whereas seamlessly sharing information throughout  many information platforms and SaaS Functions.  We additionally launched catalog federation to offer a single ruled view throughout all information platforms. This mannequin, open APIs powering catalog federation, has grow to be the trade commonplace for interoperability.
Right this moment, we’re excited to see this open imaginative and prescient validated by our companions at Microsoft.
Microsoft provides help for Unity Catalog Open APIs
As we introduced with our companions at Microsoft at the moment, our collaboration is deepening.
Microsoft has already added help for studying from Azure Databricks with OneLake Mirroring, permitting Cloth engines to entry the entire information managed in Unity Catalog seamlessly. Â Not like different OneLake integrations with proprietary platforms, this makes all information in Unity Catalog – together with the very best efficiency tables – obtainable in OneLake with out copying, altering code, or extra prices.
Right this moment, Microsoft and Databricks are extending their openness by integrating much more natively. With Microsoft OneLake including help for the Unity Catalog Open API commonplace, Azure Databricks customers will be capable to seamlessly learn information from OneLake.
With OneLake including help for open catalog APIs together with Iceberg REST and Unity Catalog Open APIs, Microsoft is making OneLake structured and unstructured information from providers like Dynamics, Dataverse, Blob retailer, and so forth. natively accessible to the complete open lakehouse ecosystem, together with Databricks.
Databricks Builds on the Open Basis
Leveraging this expanded functionality, Databricks is including help to make all OneLake information seamlessly obtainable in Unity Catalog.
This allows zero-copy studying of OneLake information from inside Databricks making certain seamless interoperability and entry throughout each platforms, giving Databricks clients entry to all their information within the Microsoft ecosystem to energy their most superior AI, machine studying, and large-scale information engineering workloads.
Catalog Integration
- Studying from Unity Catalog in Cloth – Usually Accessible Now
Microsoft has launched Azure Databricks mirroring. Prospects can mirror Azure Databricks information into OneLake, by Unity Catalog making certain that ALL information managed by Azure Databricks is immediately obtainable throughout Microsoft Cloth workloads. Each platforms can work over the identical copy of information with no information motion. - Studying from OneLake in Azure Databricks – EOY 2025Â
This new functionality, permits Azure Databricks to learn information from OneLake by way of their Unity Catalog Open APIs and permits all your OneLake information to be leveraged seamlessly in Databricks. Prospects can create Unity Catalog Connections to OneLake primarily based on Managed Identities from EntraID, and may seamlessly mount information objects from any Cloth workspace, straight as a catalog in Unity.
Connecting to OneLake information in Azure Databricks
The Future is Open
This milestone displays how far the trade has come towards an open, interoperable information ecosystem. Prospects not want to decide on between innovation and interoperability—they’ll have each.
With Microsoft now embracing the open catalog commonplace, Â we’re excited to proceed constructing the way forward for information and AI on this basis. Â Keep tuned for extra bulletins and roadmap updates, together with help for Azure Databricks writing information straight in OneLake, as we deepen our integration.
