(Ociacia/Shutterstock)
On the subject of utilizing AI to speed up BI, practitioners are discovering out that semantics actually issues. In actual fact, once you’re utilizing AI to generate SQL to question a database desk, figuring out precisely what information a selected desk holds is completely a requirement. So-called semantic layers operate as a form of map that enables the pure language question to work. It’s not stunning, then, {that a} battle is breaking out over management of this semantic layer.
Earlier than the age of AI, few folks apprehensive about semantic layers moreover these answerable for implementing BI instruments and modeling information in information warehouses. In case your group was adopting Tableau or Looker atop an analytic database from Oracle or Teradata, you’d in all probability use the semantic instruments related to these BI merchandise to outline precisely how the “working income” database view was developed, for instance.
The semantic layer defines and enforces guidelines about information, corresponding to what information sources can be utilized, what transformations are required, what checks should be carried out. Your CFO want to know that the “working income” entry on her dashboard was calculated appropriately–not only one time however each single time–and the semantic layer is a key piece of the analytics plumbing that ensured that occurred.
Constructing a semantic layer historically has been an information modeling train (Tee11/Shutterstock)
Since ChatGPT ignited the AI revolution again in 2022, the semantic layer has skilled a renaissance. That’s because of the drawback alluded to above: Letting language fashions lose on a database with no map of what the info means is a path to poor information, frustration, and enterprise failure. With out a semantic layer to information the language mannequin’s entry to information, it’s nearly assured to make errors that may upset your CFO. That’s not one thing that you just wish to get within the behavior of doing if job safety is vital to you.
The emergence of the common-or-garden semantic layer as a keystone within the AI-BI panorama has been occurring for the previous couple of years. Distributors that developed semantic layers that exist impartial of BI instruments and permit organizations to standardize how their analysts work together with underlying databases, corresponding to AtScale and Dice, have stepped up their improvement and advertising and marketing efforts to capitalize on the brand new demand. The oldsters at dbt Labs, developer of what’s arguably the world’s hottest information transformation device, additionally launched their very own semantic layer again in 2023.
Now the massive canines are beginning to eat on the semantic buffet. Snowflake launched its semantic views function throughout its latest Snowflake Summit 2025. To not be outdone, Databricks launched its personal tackle semantic views, Databricks Unity Catalog metric views, at its Databricks AI & Knowledge Summit 2025.
Snowflake Semantic Views
In its weblog saying semantic views, Snowflake engineers describe semantic views as “a brand new schema-level object that shops all semantic mannequin info natively within the database.” It replaces the present Cortex Analyst YAML file, and gives an agreed-upon metadata normal for a spread of Snowflake experiences, together with AI-powered analytics, BI purchasers, Streamlit purposes, Workspaces, Notebooks, and customized purposes.
Snowflake’s semantic views operate is in beta
Snowflake gives analysts with a number of methods to create semantic views, together with through the use of the Cortex Analyst UI, the database object explorer in Snowsight, or by writing a customized DDL assertion. Snowflake says it has adopted a mannequin definition language that lets customers outline the core attributes of a semantic view, which incorporates bodily mannequin objects (tables or views; SQL queries to be added sooner or later); relationships between these bodily objects; dimensions (or business-friendly attributes that group and filter analytical queries); and metrics (or business-friendly calculations and aggregations that characterize KPIs). Semantic views are at present in beta.
“We developed semantic views in response to a typical buyer want,” the Snowflake engineers wrote. “Clients need AI-powered conversational analytics however fear about ungoverned information entry and inconsistent or inaccurate solutions.”
Databricks Unity Catalog Metric Views
Databricks is doing one thing just like its rival with its new Unity Catalog metric views. Unity Catalog, after all, is the centralized information catalog and information governance providing that helps clients handle entry to information.
“In contrast to metrics outlined solely within the BI layer, which restrict reuse and integration, defining metrics on the information layer makes enterprise semantics reusable throughout all workloads, like dashboards, AI fashions, and information engineering jobs,” Databricks writes in its June 12 weblog put up. “Unity Catalog Metrics are additionally absolutely addressable by way of SQL to make sure that everybody within the group can have the identical view of metrics, regardless of what device they select.”
In response to Databricks’ metric view documentation, metric views specify a set of definitions, together with dimensions (corresponding to product names or buyer varieties) and measures (the appropriate values that summarize a enterprise exercise). Metric views are outlined in YAML and registered in Unity Catalog; customers can create them utilizing SQL or the Catalog Explorer UI.
Databricks Unity Catalog metrics view is anticipated to go GA this summer time
Metric views are ruled and auditable by default, Databricks says. “Licensed metrics include auditing and lineage out of the field, enabling trusted, compliant insights throughout groups,” the corporate says. Customers can create metrics as soon as in Unity Catalog, and use them throughout a number of instruments and experiences in Databricks, together with AI/BI Dashboards, Genie, Notebooks, SQL, and Lakeflow jobs, the corporate says. Sooner or later, Databricks say it would prolong metric views to assist BI instruments like Tableau, Hex, Sigma, ThoughtSpot, and Omni, in addition to observability instruments like Anomalo and Monte Carlo, the corporate writes in its weblog. Metric views are at present in beta, and are anticipated to be out there later this summer time.
Built-in or Impartial?
There’s clearly demand for a semantic layer, as Snowflake’s and Databricks’ strikes into the product class exhibit. With out this vital metadata layer, pure language question of enterprise databases will stay an AI pipedream.
The large query appears to be whether or not there’s sufficient momentum and demand for semantic layers to realize escape velocity and grow to be its personal product class, impartial of the BI merchandise they have been initially tied to, corresponding to Looker and Tableau, or information platforms just like the Snowflake and Databricks. The strikes by Snowflake and Databricks counsel in any other case.
That’s to not say it couldn’t occur. Greater up within the stack, that demand and momentum appeared for an impartial desk format to emerge: Apache Iceberg. Snowflake and Databricks laid down arms and determined to standardize on Iceberg, which was a win for purchasers and information independence and a loss for information lock-in and vendor greed.
Additional down the stack, the semantic layer has equally emerged as vital part for attaining information interoperability and bringing repeatability and reliability to AI-powered BI. Will the massive canines agree on a common layer that advantages everybody, or search to make semantic layers a aggressive benefit? Time will inform.
Associated Gadgets:
Rethinking AI-Prepared Knowledge with Semantic Layers
Is the Common Semantic Layer the Subsequent Massive Knowledge Battleground?
Semantic Layer Belongs in Middleware, and dbt Needs to Ship It
