Forum Discussion
Data Quality/Data Profiling Scan issue in Purview
Hi Everyone,
Hope you are doing great !!
I tried to run Data Quality/Data profiling scan in Purview, Scan got failed with an error "DataQualityInternalError Internal service error occurred. Please retry or contact Microsoft support"
Could you please help me how can I resolve it.
Want to ensure did I have provided right access to Purview : Currently I am using Purview MSI as an authentication option to integrate (Fabric and Purview) and provided Purview MSI with "Contributor" role access to our required Fabric Workspace. Also I am having "Contributor" role access to the required Workspace.
Kindly help me to understand:
- if we want to run data quality scan on Fabric Workspace, User (Myself) should assign with which level of access Member or Contributor in Fabric
- To run Data quality/Data profiling scan on Fabric Lakehouse delta tables in Purview , Which authentication should I have use (MSI or Service Principal)
Could you please help me to resolve it, It will help us to do Data Quality and Profiling.
Thanks in advance.
Regards,
Banumathi
2 Replies
ShafiqMannanMicrosoft
Purview MSI need contributor access to Fabric Lakehouse to connect and profile Fabric Lakehouse data. For datamap you needs to use SPN but for Data Quality Scan you need to use MSI. Follow these documents to configure the scan successfully:
Data Quality for Fabric Lakehouse in Unified Catalog | Microsoft Learn
milgoThank you for reaching out. Typically, the error you have indicated above occurs as a result of the reasons mentioned in the documentation here - https://learn.microsoft.com/en-us/purview/data-quality-troubleshooting#my-dq-scan-job-failed-i-see-an-error-message-internal-service-error-occurred-please-retry-or-contact-microsoft-support-what-should-i-do-to-troubleshoot
The reader is not sufficient as you will need to assign contributor access. As listed in the common reasons for the error you are seeing- If the contributor access for the Microsoft Purview MSI hasn't been provided to the Microsoft Fabric workspace, then the data quality scan fails.
The supported authentication mechanism to run data quality scans is using Managed Identity as described in the documentation here - https://learn.microsoft.com/en-us/purview/how-to-configure-and-run-data-quality-scan#supported-authentication-methods.