In half 1 of this sequence, we explored how shifting Backstage’s underlying database to Databricks Lakebase turned dangerous schema migrations into 1-second branch-and-test operations. However a sooner developer cycle solely will get you thus far if Safety and Governance groups are nonetheless treating your operational database like a black field.
In a standard stack, your software database and your knowledge lake stay in two solely totally different safety paradigms. The possession graph on your infrastructure lives in Backstage, backed by an remoted RDS occasion and ruled by advanced IAM roles and Postgres native grants. In the meantime, your warehouse knowledge is ruled by the information staff utilizing Unity Catalog. Unity Catalog is an Open Supply framework created by Databricks that gives a unified governance layer for knowledge, AI, and now operational databases – a single place to handle entry controls, audit trails, lineage, and compliance throughout every little thing on the platform.
To audit a single desk drop on RDS, you’d have to cross-reference CloudTrail for the IAM principal, pg_stat_activity or pgaudit logs for the SQL assertion, and CloudWatch for the timestamp, three providers, three question languages, three entry insurance policies. The operational database turns into a compliance side-channel.
Unity Catalog Absorbs the Operational DB
After we pointed Backstage at Lakebase, we did not simply change the place the information lived; we modified the place the entry coverage lived.
As a result of Lakebase is natively embedded inside Databricks, Unity Catalog extends straight over the operational Postgres database. On this POC, we used Lakehouse Federation to show the Backstage catalog as a international catalog (lakebase_bs) in Unity Catalog. As soon as it is there, customary UC grants management who can see what, no Postgres-level position administration required:
Whereas we did not construct end-to-end Row-Stage Safety insurance policies for Backstage on this POC, architecturally, the very same RLS guidelines that defend delicate billing tables might be utilized straight to those operational tables. The wall between “operational” and “analytical” stops being a bodily boundary, and easily turns into an entry sample.
A Unified Audit Path Out of the Field
Keep in mind the 1-second copy-on-write branching we executed in Half 1? In a standard setup, proving to a safety engineer {that a} developer solely branched the database for an hour after which destroyed it’s a handbook train.
With Lakebase, each control-plane motion in opposition to the operational database is routinely recorded in system.entry.audit. To show this, we queried the audit log for the precise department operations from our Half 1 disaster-recovery experiment:
Outcome:
Each department creation and deletion from our Half 1 experiments is logged. Every occasion is tied to a particular OAuth person id and supply IP, captured routinely, and ruled by the very same Row-Stage Safety controls as each different audit desk in Unity Catalog. No CloudTrail cross-referencing. No RDS log parsing. One SQL question.
Automated Price Attribution by Department
A governance staff does not simply need to know who created a department, they need to know what it value.
In a standard AWS atmosphere, monitoring the price of an ephemeral RDS occasion requires customized CloudWatch tagging methods that always miss short-lived workloads. As a result of Lakebase integrates natively with Unity Catalog’s system billing tables, compute prices break down routinely by project_id, branch_id, and endpoint_id.
On this POC, the manufacturing department was billed at 31.6130 DBU, whereas the dropped check department was independently attributed 0.0107 DBU. The audit path and the fee path are ruled in the very same place.
What This Means for Groups That Department Each Day
Our governance story solutions the compliance query: can we show who did what, when, and what it value? The reply is sure – one SQL question as a substitute of three providers. However there is a second governance query that issues simply as a lot for improvement groups adopting the branching workflow from Half 1: what occurs to governance when your staff creates dozens of branches per dash?
In Half 1, we described a workflow the place each function department and each pull request will get its personal remoted database copy. A staff of six builders operating two-week sprints would possibly create and destroy 30-40 branches in a single dash. That is 30-40 copies of manufacturing knowledge, every one probably containing delicate fields – buyer PII, monetary information, well being knowledge.
That is the place Unity Catalog’s branch-level governance turns into load-bearing, not simply handy. When a Lakebase department is created, Unity Catalog’s attribute-level masking insurance policies propagate routinely to the brand new department. A developer engaged on their function department by no means sees unmasked manufacturing knowledge – not as a result of somebody remembered to configure it, however as a result of the governance layer enforces it at creation time. The CI department that runs your PR assessments is ruled identically to manufacturing. The QA department the place a tester runs damaging eventualities is ruled identically to manufacturing. There is no such thing as a “non-production exception” the place delicate knowledge leaks as a result of somebody forgot to use the coverage.
This issues greater than it might sound. In keeping with Perforce’s 2025 State of Information Compliance report, 60% of organizations have skilled breaches or theft in non-production environments the place delicate knowledge was inadequately anonymized. The normal method – manually masking knowledge when provisioning dev/check environments – does not scale when environments are created and destroyed in seconds. Governance needs to be automated, or it does not occur.
The DBA’s New Alternative
The audit path and value attribution knowledge additionally sign a quieter shift: the DBA’s position is evolving from reactive ticket work to strategic platform structure.
Right this moment, a lot of a DBA’s time goes to operational requests – atmosphere provisioning, schema opinions, knowledge refreshes, entry grants. A six-developer staff can generate 30+ tickets per dash, and the DBA’s calendar turns into a queue. The experience that makes DBAs helpful – understanding knowledge integrity, efficiency, and governance at a deep stage – will get buried beneath repetitive provisioning work.
When branching is self-service and governance is automated, that repetitive work falls away. Builders provision their very own environments in a single second. Schema modifications are reviewed asynchronously in pull requests – the DBA sees a formatted schema diff posted by CI, opinions it on their very own schedule, and approves or requests modifications via the conventional PR workflow. With the time now obtainable, these opinions go deeper: the DBA helps staff members perceive the present knowledge and constructions in manufacturing, works with them to reach at higher options, and conducts thorough opinions that uphold knowledge integrity and governance requirements. Information masking is enforced by coverage, not by handbook intervention. Price attribution is automated, not a month-to-month reconciliation train.
What opens up is the work that truly leverages the DBA’s experience: defining branching insurance policies, designing governance guidelines, architecting promotion workflows, tuning efficiency, and establishing the guardrails that make self-service protected. The DBA shifts from doing the work to designing how the work will get finished – from 30+ operational tickets per dash to fewer than 5 high-value coverage opinions. The audit path demonstrated above is not only a compliance artifact – it is the DBA’s new strategic dashboard, a real-time view of how the platform is getting used and the place to speculate subsequent.
From Function Shift to Tooling
The DBA’s pivot from operational tickets to platform design solely works if the tooling shifts with the position. The platform has to do the routine work by itself, and the DBA wants a spot to design how that work will get finished.
Two open-source instruments, each deployed as Databricks Apps and each ruled by the identical Unity Catalog grants and audit path described above, shut that loop.
LakebaseOps is what the platform does by itself. Three brokers – Provisioning, Efficiency, and Well being – change 51 of the duties a DBA used to file tickets for. Seven of them run as scheduled Databricks Jobs and change the pg_cron crontab a DBA would in any other case hand-maintain. A monitoring UI surfaces stay pg_stat metrics, slow-query regressions, department TTL enforcement, and a 9-KPI adoption dashboard. A migration wizard scores ten supply engines (Aurora, RDS, Cloud SQL, AlloyDB, Cosmos DB, and extra) in opposition to Lakebase, with stay pricing from the AWS and Azure APIs.
Lakebase MCP is what the DBA does on prime of the platform. A Mannequin Context Protocol server exposing 46 instruments to any MCP-capable AI agent (Claude, Copilot, GPT). The DBA stops opening pgAdmin and begins describing intent:
Two design selections maintain this protected. First, dual-layer governance: a SQL-statement guard and a per-tool entry guard, with 4 pre-built profiles (read_only, analyst, developer, admin) that map onto the identical UC entry patterns proven above. A coding assistant runs as read_only and bodily can’t drop a desk.
Second, each question is attributable – the server tags each assertion with the originating instrument:
Mixed with the branch-level value attribution proven earlier, you possibly can reply “which agent on which department generated the 4 AM CPU spike?” in a single SQL question.
LakebaseOps runs for the staff. Lakebase MCP runs with the staff. Each inherit the governance posture you simply noticed.
In Half 3 of this sequence, we are going to have a look at the last word payoff: taking the infrastructure possession knowledge inside Backstage and becoming a member of it on to cloud billing knowledge in a single SQL question.
