DataBot (aka DataConcierge) with Microsoft Purview

Microsoft Purview is a PaaS service that helps organizations manage and govern their data in hybrid and multi-cloud environments. By using the enterprise data catalog, organizations can easily track and catalog their data assets and sources. With Microsoft Teams and Power Automate, it is possible to create a simple bot that can access the catalog's REST API and perform various operations within the Teams interface. This can make it easier for teams to access and manage their data without having to switch between different tools.

 

The additional objective is to enable seamless integration and collaboration within the enterprise business users and modern digital solutions.

 

 

The Architecture is simple, as Microsoft Teams will be a primary interface and connect to Microsoft Purview via REST API using Power Virtual Agent and Power Automate. It also integrates with Azure Active Directory for authentication and authorization. You can take out Power Virtual Agent if you want to optimize the cost by going in a less conversational way.

 

An interactive conversational agent can be embedded using Power Virtual Agent or traditional bot framework with Azure Solutions and this can be published as an app and consumed via different channels such as Microsoft Teams, direct Bot interface, Web Application, Mobile App etc.,

 

 

I have built a POC that demonstrates the basic functionalities (search data asset & glossary) of the Data Concierge, and this can be expanded based on the use cases and I have primarily used Microsoft teams with a private chat and teams channel for better collaboration.

 

The Microsoft Teams integration enables better collaboration with enterprise teams/business units. All teams can interact with the same Data Bot just by typing @Data Concierge<topic keywords/response> with new/existing conversations.

 

 

The Microsoft Purview documentation references for REST API.

How to use REST APIs for Microsoft Purview Data Planes - Microsoft Purview | Microsoft Docs

Discovery - Query - REST API (Microsoft Purview) | Microsoft Docs

 

Quick Links:

• Microsoft Purview: https://web.purview.azure.com/
• Azure Active Directory App or Service Principal: https://portal.azure.com/#blade/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/RegisteredApps or  https://aad.portal.azure.com/#blade/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/RegisteredApps
• Microsoft Teams: https://teams.microsoft.com/
• Power Virtual Agent: https://web.powerva.microsoft.com/
• Power Automate: https://flow.microsoft.com/
• Azure Portal: https://portal.azure.com/

 

Conclusion:

Microsoft Purview REST API can be integrated with any custom applications.

Some relevant use cases for Microsoft Purview REST API usage:

• Data Bot for an enterprise user with all data sources/assets
• Virtual Data Engineer for Data Analyst and Data Scientist
• Custom app with Data Lineage and Data Map
• Custom data assets registration and data enrichment
• Workflow automation and metadata policy management with REST API
• Glossary search and data assets scanning
• Enterprise Data Lake adoption through custom bot interface

 

You may reachout to Microsoft team if you're already using Microsoft Azure Cloud & Microsoft Teams in your environment.