People of Purview: Elie El Karkafi
How long have you been working with Microsoft products? I am a Microsoft MVP in Security and a results-driven and dedicated Senior Solutions Architect with more than 14 years of professional experience in Microsoft Technologies. I have in-depth experience in Cloud, Security, Enterprise Mobility, Messaging & Collaboration, and IT Infrastructure. I am dedicated to customer satisfaction with focused delivery of technical solutions. I'm a proven leader in directing operations, maintenance, and support of complex systems. Highly adept in request for proposal development, technology needs assessment and staff training. How (and when) did you get involved in the Microsoft Community? (MCCP, MVP) Tell us about your journey! I became a member of the Microsoft Customer Connection Program (MCCP) in 2022 and was honored with the Microsoft MVP award in Security in 2023. Throughout my time in the MCCP, I’ve consistently contributed valuable insights and feedback to help enhance Microsoft Security products. Over the past years, I’ve been recognized as one of the top contributors across several MCCP focus areas—ranking second in Security, and first in Identity, Management, and Purview in the last two years. My contributions have earned me multiple accolades, including the Community Rockstar Award and Community Leader Awards in Security, Management, Entra, and Purview, along with several Partner of the Month honors. My engagement spans more than 150 private previews, 200 surveys, 50 focus groups, and 50 one-on-one feedback sessions with Microsoft product teams. I’ve been actively involved in shaping the future of AI and Microsoft Security Copilot, including identifying and resolving bugs in Copilot for Entra and Purview. As a Security Design Partner, I’ve collaborated directly with Microsoft designers on key features such as XDR Case Management and Microsoft Entra Conditional Access Optimization Agent. The MVP recognition not only validates my expertise but also enhances my professional credibility, supporting both client engagements and career growth. Being an MVP provides early access to Microsoft products and services, allowing me to deliver cutting-edge solutions. It also opens direct communication channels with Microsoft product teams, enabling me to influence product development and stay informed about upcoming features and roadmaps. What do you find most rewarding about being a community member? Being part of Microsoft Customer Connection Program (MCCP) significantly enhanced and diversified my skill sets in many ways: Product Innovation: Contributed to the enhancement of Microsoft products—specifically Defender, Sentinel, Entra, Purview, and Intune—by supporting the development of new features and improved functionalities. Community Empowerment: Actively supported peers in engaging with Microsoft Customer Connection Programs (MCCP), helping them amplify their impact through increased contributions and facilitating more direct interactions with product teams. Program Advocacy: Championed participation in Microsoft initiatives by providing strategic feedback and encouraging broader community involvement to drive continuous improvement. Collaborative Development: Partnered closely with Microsoft Product Groups and UX Designers, delivering in-depth user feedback to influence product direction and usability. Program Shaping: Leveraged personal experience within Microsoft programs to help co-develop and refine these initiatives in collaboration with internal teams. Tool Creation: Designed and launched a security calculator—originally built for internal use—which is now widely adopted by CCP members to assess and compare Microsoft security solutions and simulate associated costs. What advice do you have for others who would like to get involved in their Microsoft Community? To all Folks, look for opportunities to join programs like the Microsoft Customer Connection Program (MCCP), MVP program, or private previews. These give you direct access to product teams and a chance to influence development through feedback. Don’t be afraid to ask questions, share your experiences, or offer help. Even small contributions like answering a question or sharing a tip can make a big impact and help you build credibility Community involvement is a journey. Stay curious, keep learning, and show up regularly. Over time, your presence and contributions will be recognized!!! And the most important thing is to stay humble!!! Do you have anything you’d like to promote or recommend? (your blog or podcast, an article you recommend, a book everyone should read, etc) Security Calculator that I Built: Login - Microsoft Security Calculator Personal website where my blogs are posted : Elie El Karkafi - MVP - Personal Blog LinkedIn Page: (1) Elie El Karkafi | LinkedIn Credly Profile: https://www.credly.com/users/eliekarkafy MVP Profile: MVP Communities --- Elie is based in Dallas, Texas and is a Senior Solutions Architect at ampiO Solutions. (Listen.Develop.Deliver - ampiO Solutions) Banner photos (above) are from Elie's trip to Spain visiting Santiago Bernabeu stadium as he is a fan of Real Madrid football Team. ____________________________________________________________________________________ Stay tuned to meet more People of Purview! If you would like to get involved with the Microsoft Security Community, here are some quick actions you can take: Log in (here, on Tech Community!) and follow: The Purview Community - post questions, respond to community members The all-up Microsoft Security Blog Join the Security Community mailing list Join the Customer Connection Program Check out this Community Choice article for a comprehensive list of Microsoft Security Community offerings.
Safeguard data on third-party collaboration platforms
I am exploring options to safeguard sensitive data in third-party collaboration platforms like GitHub and Confluence. Does Microsoft Purview provide any native integration for these platforms? Do I need to rely on third-party connectors/integrations to extend Purview’s capabilities into these environments?
Column-Level Lineage Visualization Issue for Custom Entities and Processes in Azure Purview
I’m trying to implement column-level lineage for data assets and custom transformation processes in Azure Purview using the Atlas API. I have defined custom typedefs for tables (yellowbrick_table), columns (column), and a process type (custom_data_transformation_process), and I’m uploading entities with composition relationships and detailed column mappings. Although the generated JSON and typedefs appear to be correct according to the Apache Atlas documentation, I’m unable to get the column-level lineage to display properly in the Purview UI Specific Issues: 1. Missing 'Schema' tab in custom table entities (yellowbrick_table): When navigating to the detail page of a yellowbrick_table entity, the 'Schema' tab — where the contained columns should be listed — does not appear. 2. Missing 'Switch to column lineage' button in the lineage view of custom processes: In the 'Lineage' tab of a custom_data_transformation_process entity, the side panel titled 'Columns' displays "No mapped columns found," and there is no button or option to switch to column-level lineage view. This happens even when the columnMapping attribute is correctly populated in the process entity. 3. Error message when trying to edit column mappings in the process lineage panel: If I attempt to edit the column mapping in the lineage side panel of the process, I receive the error: "Unable to map columns for this asset. It's a process type asset that doesn't have a schema." (This is expected, as processes don’t have schemas, but it confirms that the UI is not interpreting the columnMapping for visualization purposes.) Context and Steps Taken: I have followed the Apache Atlas documentation and modeling patterns for lineage: Typedef column: Defined with superTypes: ["Referenceable"]. RelationshipDef table_columns: Defined as a COMPOSITION between DataSet (extended by yellowbrick_table) and column, with cardinality: SET on the column side. Typedef yellowbrick_table: Contains an attribute columns with typeName: "array<column>" and relationshipTypeName: "table_columns". Typedef custom_data_transformation_process: Extends from Process and includes a columnMapping attribute of typeName: "array<string>". Entities Uploaded (JSON): * Table entities include complete definitions of their nested columns in the columns attribute. * Process entities include the columnMapping attribute as a list of JSON strings, where each string represents a DatasetMapping with a nested ColumnMapping that uses only the column names (e.g., "Source": "COLUMN_NAME"). * I’ve tested with different browsers Despite these efforts, the issue persists. I would like to know if there are any additional requirements or known behaviors in the Purview UI regarding column lineage visualization for custom types. Specific Questions: 1. Is there any additional attribute or configuration required in the typedefs or entities to make the 'Schema' tab appear in my custom table entities? 2. Are there any specific requirements for the qualifiedName of tables or columns that could be preventing the column-level lineage from being visualized? 3. Could there be a known issue or limitation in the Purview UI regarding column-level lineage rendering for user-defined asset types? 4. Is there any way to verify on the Purview backend that the column composition relationships and the columnMapping of processes have been correctly indexed?" Annexes: # 1. Define the 'column' type # Ensure the superType for 'column' is 'Referenceable' typedef_payload_column = { "entityDefs": [{ "category": "ENTITY", "name": "column", "description": "Columna lógica para linaje columna a columna", "typeVersion": "1.0", "superTypes": ["Referenceable"], "attributeDefs": [] }] } # response = requests.post(typedef_url, headers=headers, json=typedef_payload_column) # print(f"Estado typedef column: {response.status_code} ({response.text.strip()[:100]}...)") # 2. Define the explicit relationship 'table_columns' # Ensure that the cardinality of 'endDef2' (columns) is 'SET' typedef_payload_table_columns_relationship = { "relationshipDefs": [ { "category": "RELATIONSHIP", "name": "table_columns", "description": "Relación entre una tabla y sus columnas", "typeVersion": "1.0", "superTypes": ["AtlasRelationship"], "endDef1": { "type": "DataSet", "name": "parentTable", "isContainer": True, "cardinality": "SINGLE", "isLegacyAttribute": False }, "endDef2": { "type": "column", "name": "columns", "isContainer": False, "cardinality": "SET", "isLegacyAttribute": False }, "relationshipCategory": "COMPOSITION", "attributeDefs": [] } ] } # response = requests.post(typedef_url, headers=headers, json=typedef_payload_table_columns_relationship) # print(f"Estado relationshipDef table_columns: {response.status_code} ({response.text.strip()[:100]}...)") # 3. Modify the table's typedef to use this relationship # Ensure the 'columns' attribute on the table points to 'table_columns' typedef_payload_yellowbrick_table = { "entityDefs": [{ "category": "ENTITY", "name": "yellowbrick_table", "description": "Tabla en Yellowbrick", "typeVersion": "1.0", "superTypes": ["DataSet"], "attributeDefs": [ { "name": "columns", "typeName": "array<column>", "isOptional": True, "cardinality": "LIST", "valuesMinCount": 0, "valuesMaxCount": -1, "isUnique": False, "isIndexable": False, "includeInNotification": True, "relationshipTypeName": "table_columns" } ] }] } # response = requests.post(typedef_url, headers=headers, json=typedef_payload_yellowbrick_table) # print(f"Estado typedef yellowbrick_table: {response.status_code} ({response.text.strip()[:100]}...)") # 4. Define the custom process type # Ensure the 'columnMapping' attribute is a string array typedef_payload_process = { "entityDefs": [{ "category": "ENTITY", "name": "custom_data_transformation_process", "description": "Proceso de transformación de datos con linaje de columna (Custom)", "typeVersion": "1.0", "superTypes": ["Process"], "attributeDefs": [ { "name": "columnMapping", "typeName": "array<string>", "isOptional": True, "cardinality": "LIST", "valuesMinCount": 0, "valuesMaxCount": -1 } ] }] } # response = requests.post(typedef_url, headers=headers, json=typedef_payload_process) # print(f"Estado typedef custom_data_transformation_process: {response.status_code} ({response.text.strip()[:100]}...)") example of generated json: [ { "typeName": "yellowbrick_table", "guid": "-105", "attributes": { "qualifiedName": "DB_DWH_EXTRACCION.HOGARES.TBL8_CONOCIMIENTO_CLIENTE@yellowbrick_conn", "name": "TBL8_CONOCIMIENTO_CLIENTE", "description": "Tabla origen: TBL8_CONOCIMIENTO_CLIENTE", "columns": [ { "typeName": "column", "guid": "-336", "attributes": { "qualifiedName": "DB_DWH_EXTRACCION.HOGARES.TBL8_CONOCIMIENTO_CLIENTE@yellowbrick_conn#CUENTA", "name": "CUENTA", "description": "Columna CUENTA de tabla tbl8_conocimiento_cliente", "type": "string", "dataType": "string" } }, { "typeName": "column", "guid": "-338", "attributes": { "qualifiedName": "DB_DWH_EXTRACCION.HOGARES.TBL8_CONOCIMIENTO_CLIENTE@yellowbrick_conn#TIPO_DOCUMENTO", "name": "TIPO_DOCUMENTO", "description": "Columna TIPO_DOCUMENTO de tabla tbl8_conocimiento_cliente", "type": "string", "dataType": "string" } }, { "typeName": "column", "guid": "-340", "attributes": { "qualifiedName": "DB_DWH_EXTRACCION.HOGARES.TBL8_CONOCIMIENTO_CLIENTE@yellowbrick_conn#IDENTIFICACION", "name": "IDENTIFICACION", "description": "Columna IDENTIFICACION de tabla tbl8_conocimiento_cliente", "type": "string", "dataType": "string" } }, { "typeName": "column", "guid": "-342", "attributes": { "qualifiedName": "DB_DWH_EXTRACCION.HOGARES.TBL8_CONOCIMIENTO_CLIENTE@yellowbrick_conn#NOMBRE_1", "name": "NOMBRE_1", "description": "Columna NOMBRE_1 de tabla tbl8_conocimiento_cliente", "type": "string", "dataType": "string" } }, { "typeName": "column", "guid": "-344", "attributes": { "qualifiedName": "DB_DWH_EXTRACCION.HOGARES.TBL8_CONOCIMIENTO_CLIENTE@yellowbrick_conn#APELLIDO_1", "name": "APELLIDO_1", "description": "Columna APELLIDO_1 de tabla tbl8_conocimiento_cliente", "type": "string", "dataType": "string" } }, { "typeName": "column", "guid": "-346", "attributes": { "qualifiedName": "DB_DWH_EXTRACCION.HOGARES.TBL8_CONOCIMIENTO_CLIENTE@yellowbrick_conn#APELLIDO_2", "name": "APELLIDO_2", "description": "Columna APELLIDO_2 de tabla tbl8_conocimiento_cliente", "type": "string", "dataType": "string" } }, { "typeName": "column", "guid": "-348", "attributes": { "qualifiedName": "DB_DWH_EXTRACCION.HOGARES.TBL8_CONOCIMIENTO_CLIENTE@yellowbrick_conn#GENERO", "name": "GENERO", "description": "Columna GENERO de tabla tbl8_conocimiento_cliente", "type": "string", "dataType": "string" } }, { "typeName": "column", "guid": "-349", "attributes": { "qualifiedName": "DB_DWH_EXTRACCION.HOGARES.TBL8_CONOCIMIENTO_CLIENTE@yellowbrick_conn#EDAD", "name": "EDAD", "description": "Columna EDAD de tabla tbl8_conocimiento_cliente", "type": "string", "dataType": "string" } }, { "typeName": "column", "guid": "-350", "attributes": { "qualifiedName": "DB_DWH_EXTRACCION.HOGARES.TBL8_CONOCIMIENTO_CLIENTE@yellowbrick_conn#GENERACION", "name": "GENERACION", "description": "Columna GENERACION de tabla tbl8_conocimiento_cliente", "type": "string", "dataType": "string" } } ] } }, { "typeName": "custom_data_transformation_process", "guid": "-107", "attributes": { "qualifiedName": "linaje_process_from_tbl8_conocimiento_cliente_to_tbl_tmp_clien_conocimiento_cliente_c@yellowbrick_conn", "name": "linaje_tbl8_conocimiento_cliente_to_tbl_tmp_clien_conocimiento_cliente_c", "description": "Proceso que conecta TBL8_CONOCIMIENTO_CLIENTE a TBL_TMP_CLIEN_CONOCIMIENTO_CLIENTE_C", "inputs": [ { "guid": "-105" } ], "outputs": [ { "guid": "-106" } ], "columnMapping": [ "{\"DatasetMapping\": {\"Source\": \"DB_DWH_EXTRACCION.HOGARES.TBL8_CONOCIMIENTO_CLIENTE@yellowbrick_conn\", \"Sink\": \"DB_DWH_STAG.CLIENTES.TBL_TMP_CLIEN_CONOCIMIENTO_CLIENTE_C@yellowbrick_conn\"}, \"ColumnMapping\": [{\"Source\": \"CUENTA\", \"Sink\": \"CUENTA\"}, {\"Source\": \"TIPO_DOCUMENTO\", \"Sink\": \"TIPO_DOCUMENTO\"}, {\"Source\": \"IDENTIFICACION\", \"Sink\": \"IDENTIFICACION\"}, {\"Source\": \"NOMBRE_1\", \"Sink\": \"NOMBRE_1\"}, {\"Source\": \"APELLIDO_1\", \"Sink\": \"APELLIDO_1\"}, {\"Source\": \"APELLIDO_2\", \"Sink\": \"APELLIDO_2\"}, {\"Source\": \"GENERO\", \"Sink\": \"GENERO\"}, {\"Source\": \"EDAD\", \"Sink\": \"EDAD\"}, {\"Source\": \"GENERACION\", \"Sink\": \"GENERACION\"}]}" ] } } ]Purview Destruction of Records Not Working
Hi everyone, I work for a Microsoft Partner Organization, we are experiencing issues with our Purview Implementations as the records are not being destroyed from SharePoint as expected event though the audit log registered all successful approvals for destructions including details on the stages, the comments and the label. We have waited 15 days as per the Purview's documentation and the content is still in the source library. Is anyone else experiencing a similar issue? The instability of the platform is forcing us to stop offering Purview as a solution to our customers and we are leaning more toward developing our own records management solution. We have created a ticket, but Microsoft has not responded, there is no priority to resolve this issue and unfortunately our customers can't wait. If anyone has a solution, please share it with the community. If there are any Purview experts from Microsoft that can offer any ideas, I would appreciate it. Note: Auditing is on for the organization and the email enabled security group and the Disposition Reviewers are in the correct roles: Disposition Reviewer, Records Management, Compliance Admin, List View Explorer and Content Explorer. Also, my configuration is set correctly, I have proof as the audit log indicates content has been approved for destruction weeks ago. Thank you!
Insider Risk Management Alerts/Activities issue
Hello, we have a problem where Insider Risk Management is generating activity data/alerts based on false data (sort of). There is an activity called: EPOFILEARCHIVED or FileArchived that is done by the SenseCE.exe application. SenseCE is "Windows Defender Advanced Threat Protection Sense CE module" according to 3rd party source and "Data Loss Prevention Classification" according to another, I guess it is related as a service application for Endpoint DLP as well. Anyways, it is generating lots of false activity and there is not any actual way to exclude this activity (as an app or as an activity type) from Purview and it introduces false data into Insider Risk Management (which picks it up as an Archive activity). Anyone have similar issues or have another explanation why this activity is appearing? Perhaps there are ways to remedy this somehow? Example:eDiscovery keyword statistics.
Noticing with this roadmap item: https://admin.microsoft.com/AdminPortal/Home?#/MessageCenter/:/messages/MC1105008 specifically Expanded search condition builder with support for logical operators (AND, OR, NEAR) in the keywords field That when running a new search that the statistics generated for keywords claims that "Query does not contain keywords" and doesn't generate the Statistics reports for keywords anymore. Tried with keywords on multiple lines as well as same line but separated with OR statements. Is this known issue?
Auto-Apply Sensitivity Label
I am working on DLP and Information Protection feature of Purview to utilize the Sensitive Info Types. In DLP, I made a policy to send alerts when the SITs are found. In Information Protection, I have sensitivity label and configured to auto apply it once it detects the SIT. I use the same SIT on alert and in sensitivity label. Upon doing tests on several documents, I am receiving alerts via email because the SIT is found. However, the automatic application of SIT on the document does not persist, even it's been 2 weeks since I have created the document. Other information: *Upon publishing other labels, i did not configure any default labels on documents. *Tenant is US *Im working in the Philippines Here is the email i am receiving coming from the DLP policy. Here is my configuration of the sensitivity label.Purview sensitivity label modernization Label grouping Roadmap ID: 386900
Hello team, Purview will implement a new way to organize labels: https://admin.microsoft.com/Adminportal/Home?#/MessageCenter/:/messages/MC1111778 https://www.microsoft.com/en-ca/microsoft-365/roadmap?id=386900 Do you know where can I get information about how this upgrade will impact the current taxonomy for sensitivity labels, I am using label, label parent and sublabels. I reviewed in microsoft learn without any information about this feature.
