Azure Purview at Spring Ignite 2021

[Gaurav Malhotra]

Last December, we launched Azure Purview, a unified data governance service that enables customers to maximize the business value of data. Azure Purview enables customers to manage and govern their on-premises, multicloud, and SaaS data by creating a holistic data map powered by automated data discovery, AI-powered data classification and end-to-end lineage. This trustworthy, valuable data can then be discovered effortlessly by data consumers via the Purview Data Catalog.

The reception to Azure Purview since launch has been tremendous! We are thrilled to announce that over 14.5 Billion data assets were discovered by customers across their hybrid environments! And today, we are happy to announce that we have some great new features to help our customers do more with Azure Purview. Below are the highlights. Stay tuned for deep dive technical blogs that delve into each awesome feature over the span of Ignite (2-4). 

1. Azure Purview now scans and classifies AWS Simple Storage Service (S3) data, in public preview - Azure Purview can now scan and classify data stored in AWS Simple Cloud Storage (S3) in public preview. Chief Data Officers, Security and Compliance managers can now understand what data resides across their hybrid environment including multicloud sources viz. AWS S3, Azure and gain a bird's-eye view of their data landscape.
2. Azure Purview connectors for more on-premises sources - Azure Purview extends the Purview Data Map's ability to automatically scan data to more on-premises sources. With this feature, you can now automatically scan and view lineage between assets for the following sources:
   1. Azure Purview now supports ERP sources like SAP S/4 HANA and SAP ECC.
   2. Azure Purview also supports Oracle DB as a data source
3. Scan Azure Synapse Dedicated and Serverless Pools - You can now use Azure Purview to scan Azure Synapse workspaces across serverless and dedicated SQL pools. Learn more here.
4. Scan Azure resource groups & subscriptions with Purview - You can now govern multiple Azure sources within Azure Purview by registering an entire Azure subscription or a resource group. This allows you to set up scans using scan rule sets, triggers and enable discovery of assets along with their metadata, schema and classification at an azure subscription or resource group level in Azure Purview Catalog. Learn more here. 
5. Azure Purview powered search in Azure Synapse workspaces - Azure Synapse Analytics workspaces can now turn on an Azure Purview powered search. This integration allows customers to discover Azure Purview managed data assets and interact with them within the native Synapse Analytics workspace. Learn more on connecting an Azure Purview Account to Synapse and enabling effortless data discovery within the Synapse workspace!
6. Support for hierarchical business glossary within Azure Purview Data Catalog-  As an Azure Purview customer, you can now implement a hierarchical business glossary in public preview. With this feature, customers can better represent real-life business vocabulary in their Azure Purview Data Catalog. They can now organize business terms in hierarchies, create same term name in different parents and allow right set of users to manage the right glossary terms. Learn more here and read the blog.
7. Azure Purview now supports private endpoint -  Azure private link for Azure Purview is now available in public preview to connect securely via private endpoints. You can now set up scans on your on-premises and Azure data sources with ingestion private endpoints and a self—hosted integration runtime to ensure network isolation, and metadata traversing only via private networks end-to-end and block all access to your Purview instance from public internet.

Join us at Ignite to learn more!

To learn about all these updates and more, please join us at Ignite:

1. Go Limitless: with Azure Data & AI (FS174 & FS174-R1): Link, Link
2. Manage risk and secure information across your environment (FS196, FS196-R1): Link, Link
3. Enable unified data governance with Azure Purview (OD337): Link
4. Build systems of intelligence with Power BI and Azure Data (OD352): Link
5. Take charge of your data governance across your data landscape (OD358): Link

We also want to hear more from you and we have an opportunity for you to join the product team at 2 Azure Purview roundtables where you can learn about exciting new updates in Azure Purview and share your perspective and feedback on what we should be doing next. Once registered for Microsoft Ignite, simply click on one of the below links to join us:

1. Azure Purview Product Roundtable on 3/2/2021, 3-4 PM 

https://myignite.microsoft.com/auth-redirect?dest=%2Fsessions/0b688a54-1a08-4924-a6fe-4d4a97e6c64a

2. Azure Purview Product Roundtable on 3/3/2021, 12PM- 1 AM

https://myignite.microsoft.com/auth-redirect?dest=%2Fsessions/c2714de1-bddf-41a5-b433-b726e3fc664e

We can't wait to meet you at Ignite. Leave us feedback via the comments! 

ICYMI, last week we announced that we are extending the Azure Purview offer to provision 4 Capacity Units of the Data Map for free till May 31, 2021! Charging will only start on June 1, 2021.

Get started now with Azure Purview, learn more here.

[blandry]

I'm assuming it is still considered in Preview and not GA. When is it expected to get to GA?

[Vijay1711]

Hi,

We have an ability to register the Power BI tenant. However, there is no ability to select only required workspace/Data flows/Datasets for scans. Would be great if user gets an ability to select individual  workspace/ Data flows for scans. 

[NavinLadda]

I see in the mechanics video Hive Metastore as one of the Sources you can register in Purview, but in my Purview Preview, I do not see such a source.

I was able to setup Purview and install the Starter Kit scripts and see the data sources it creates, assets under them and lineage.

But what I am really interested is in its scanning capabilities for a Cloudera Hadoop running under Azure cloud. I want to learn how to configure the Atlas REST APIs that will scan the HiveQL code which is my ETL tool and see the lineage information it extracts out.

I have seen some posts such as this one on this : Azure Purview — Cataloging Hive Assets using Apache Atlas API | by Prosenjit Chakraborty | Medium

But need some help in trying this out in my trial Azure Purview instance.

[NewPurviewUser]

What is the Product roadmap for Azure Purview? I see it has limited connectors and it doesn't cover all major sources . 

[cvillegas]

I have tried AZ Purview and was really excited and showed my team; however, it doesn't appear to function properly even when we had an Azure engineer that lead one of the demos at Ignite on the call. We really just want to use it as a better modelling platform for our Power BI capacities (Arun and Ajit mentioned that it should be used as an ancillary tool to compliment Power BI to do better dataflow flow lineage since Power BI can't currently display the lineage of computed dataflows across workspaces) but it doesn't appear to clean out legacy entries on new scans and somehow is also pulling in data sources that we haven't even granted the product access to within the scan. I'm unable to find a good support method since when I select submit support request I get an access denied error so if anyone has any information that can assist that'd be great. 

[Keith Davies]

@cvillegas 

We are thinking of using AZ Purview.

I was wondering if any of your issues have been resolved.

Would be good to know.

Many thanks :thumbs_up:

[cvillegas]

Hi Keith, Not really. I can actually do quite a bit with the API, but not much from the front-end. I was even able to see stuff on the tabular end that could work with the API. Unfortunately without every data source in the product we couldn't do much so we abandoned it for now.

[Keith Davies]

@cvillegas 

Thanks for letting me know :thumbs_up:

"...so we abandoned it for now"

Do you see yourself using it at some point in the future?

Do you think the product is mature enough to invest in at the moment?