Real-time reconciliation of approved models (SOLL) vs live Unity Catalog state (IST).
Column `loyalty_tier` exists in live Unity Catalog but is missing from approved model.
Column `card_number` is plain text in Unity Catalog! Approved model requires dynamic masking.
Table is partitioned by `created_at` (high-cardinality anti-pattern). Approved model specifies CLUSTER BY (customer_id).
Migrating orders from Hive partitioning to Liquid Clustering will save an estimated 65% Serverless SQL DBU scan time.
-- DynaDatum Automated Lakehouse Remediation Script
-- Generated to reconcile IST (Unity Catalog) with SOLL (Approved Architecture)
-- 1. Fix Liquid Clustering on orders table (Eliminate small files & cut DBU costs)
ALTER TABLE main.analytics.orders CLUSTER BY (customer_id, created_at);
-- 2. Enforce Dynamic Column Masking on exposed PII
CREATE OR REPLACE FUNCTION mask_email(val STRING)
RETURN CASE WHEN is_account_group_member('compliance_officers') THEN val ELSE '***@***.com' END;
ALTER TABLE main.analytics.customers ALTER COLUMN email SET MASK mask_email;
-- 3. Tag sensitive columns for Unity Catalog Data Governance
ALTER TABLE main.analytics.customers ALTER COLUMN email SET TAGS ('sensitivity' = 'pii', 'gdpr' = 'true');
ALTER TABLE main.analytics.customers ALTER COLUMN phone_number SET TAGS ('sensitivity' = 'pii');
-- 4. Enable Delta Table Properties for Column Defaults
ALTER TABLE main.analytics.customers SET TBLPROPERTIES ('delta.feature.allowColumnDefaults' = 'supported');