Forum Discussion
Purview Data Quality with Microsoft Fabric
Hi,
Our technical data quality checks are written in SQL inside the Fabric Warehouse. When a check fails, the ETL pipeline stops.
- Can business data quality rules defined in Purview Data Quality work the same way? Can they run inside the Warehouse and stop the ETL pipeline when they fail?
- Could someone also explain how to estimate the cost of running these rules in Purview Data Quality? How exactly is it billed: per compute unit, per compute time, or per number of rows processed?
Thanks!
2 Replies
- OwenLantyBrass Contributor
Purview data quality rules run as Purview jobs; they do not execute inside the Fabric Warehouse or automatically fail a Fabric pipeline. To use them as a gate, trigger the scan from the pipeline, read the rule results, and explicitly fail the pipeline when the score is below your threshold. Cost should be estimated from the current Purview Data Quality pricing meter and the frequency and duration of those scans, not from Warehouse rows alone.
Purview Data Quality is not a drop-in replacement for your Warehouse SQL checks that stop ETL. Microsoft documents its scans as a separate Spark-based service, not rules executed inside the Warehouse transaction. Its current Fabric data-quality source list names Lakehouse, shortcuts and mirrored data; Warehouse metadata appearing in Data Map does not establish equivalent data-quality execution support. Keep your SQL checks as the pipeline’s blocking gate unless you explicitly build and validate orchestration around a supported Purview result. Purview can provide business-rule assessment and monitoring alongside that gate. Billing uses Data Governance Processing Units consumed per run, rather than a flat per-row charge. A DGPU represents 60 minutes of managed compute; consumption varies with rule complexity, data volume, source and performance tier. Pilot representative rules, inspect the completed job’s PU consumed value, multiply by the applicable SKU rate and planned frequency, and include the separate governed-asset meter in your estimate.