Providing a unified healthcare analytics solution for the era of AI
This blog is part of a series that explores the recent announcement of the public preview of healthcare data solutions in Microsoft Fabric. Healthcare data solutions in Microsoft Fabric is a comprehensive, end-to-end analytics SaaS platform that allows you to ingest, store, and analyze healthcare data from a variety of sources, including electronic health records and picture archiving and communication systems. With this platform, you can unlock new insights and drive value from your healthcare data.
Building Apps on Healthcare Data Solutions for Power Platform
Healthcare Data Solutions for Power Platform is a cloud solution helping customers and partners build healthcare applications using low code/no code tools and a FHIR-aligned data model. The solution also provides template applications, a Dataverse Healthcare API, Virtual Health Data Tables, and a FHIRlink Power Platform connector. The solution is free for Power Platform customers and can be customized and extended using Dataverse tools and custom code.Empowering multi-modal analytics with the medical imaging capability in Microsoft Fabric
This blog is part of a series that explores the recent announcement of the public preview of healthcare data solutions in Microsoft Fabric. The DICOM® (Digital Imaging and Communications in Medicine) data ingestion capability within the healthcare data solutions in Microsoft Fabric enables the storage, management, and analysis of imaging metadata from various modalities, including X-rays, CT scans, and MRIs, directly within Microsoft Fabric. It fosters collaboration, R&D and AI innovation for healthcare and life science use cases. Our customers and partners can now integrate DICOM® imaging datasets with clinical data stored in FHIR® (Fast Healthcare Interoperability Resources) format. By making imaging pixels and metadata accessible alongside clinical history and laboratory data, it enables clinicians and researchers to interpret imaging findings in the appropriate clinical context. This leads to enhanced diagnostic accuracy, informative clinical decision-making, and ultimately, improved patient outcomes.FHIRlink Connector Support for EPIC® on FHIR®
The Health and Life Sciences Data Platform team recently released an update to the FHIRlink connector introducing support for EPIC® on FHIR® connectivity. This is our initial release of connectivity for EPIC® on FHIR® application registrations configured with an application audience of Patient or Clinicians/Administrative Users.FHIRlink Power Platform connector Public Preview Release
Microsoft FHIRlink creates a direct connection between healthcare apps built on Microsoft Azure services and FHIR’s servers, bypassing the need to duplicate data from Microsoft Dataverse. FHIRlink reduces the complexity and cost of building low code/no code applications on Power Platform and Azure because developers can build their apps directly against the FHIR services rather than having to duplicate data between systems. Connect Power Automate Flows, Power Platform Canvas Apps, and Azure Logic Apps to various FHIR services and perform create, retrieve, update and delete operations directly on FHIR resources.Virtual Health Data Tables Create Update and Delete Support
Health organizations are considering low-code development to improve productivity, gain faster time-to-market, experiment more easily, and overall, be more agile when responding to market changes. A key blocker has been the inability to pull health data from multiple sources and manage it in a secure and compliant way. Microsoft Cloud for Healthcare includes configurable solutions to exchange data between Dataverse and external systems using the FHIR standard. Microsoft's Virtual Health Data Tables provides the ability to connect directly to Azure Health Data Services FHIR service from within Dataverse. As part of the latest release, Virtual Health Data Tables has been updated to include support for the create, update, and delete FHIR operations.
FHIRlink Power Platform connector General Availability
Microsoft FHIRlink creates a direct connection between healthcare apps built on Microsoft Azure services and FHIR’s servers, bypassing the need to duplicate data from Microsoft Dataverse. FHIRlink reduces the complexity and cost of building low code/no code applications on Power Platform and Azure because developers can build their apps directly against the FHIR services rather than having to duplicate data between systems. Connect Power Automate Flows, Power Platform Canvas Apps, and Azure Logic Apps to various FHIR services and perform create, retrieve, update and delete operations directly on FHIR resources.FHIRlink connector and Epic on FHIR for Power Platform development
The FHIRlink connector for Power Platform enables direct access to FHIR based endpoints. In this post, we look at a working sample application that connects Dataverse with Epic on EPIC® on FHIR® and Azure OpenAI to provide patient details and AI support for clinical users.
Integrating remote patient monitoring solutions with healthcare data solutions in Microsoft Fabric
Co-Authors: Kemal Kepenek, Mustafa Al-Durra PhD, Matt Dearing, Jason Foerch, Manoj Kumar Introduction Remote patient monitoring solutions rely on connected devices, wearable technology, and advanced software platforms to collect and transmit patient health data. They facilitate monitoring of vital signs, chronic conditions, and behavioral patterns. Healthcare data solutions in Microsoft Fabric offers a secure, scalable, and interoperable data platform as part of Microsoft for Healthcare. Such a unified data platform is crucial for integrating disparate data sources and generating actionable health insights. This article provides a reference architecture and the steps to integrate remote patient monitoring solutions with healthcare data solutions in Fabric. The integration is aimed at satisfying low data resolution use cases. With low data resolution, we address infrequent (hourly, daily, or less) transfer of aggregated or point-in-time-snapshot device data into healthcare data solutions in Fabric to be used in a batch fashion to generate analytical insights. Integration steps for high data resolution use cases, which necessitate high frequency transfer of highly granular medical device data (for example, data from EKGs or ECGs) to become input to either batch or (near) real-time analytics processing and consumption, is a candidate for a future article. There are several methods, solutions and partners available in the marketplace today that will allow you to integrate a remote patient monitoring solution with the healthcare data solutions in Fabric. In this article, we leveraged the solution from Life365 (a Microsoft partner). The integration approach discussed here is applicable to most remote patient monitoring solutions whose integration logic (code) can be run inside a platform that can programmatically access (for example, through REST API calls) Microsoft Fabric. In our approach, the integration platform chosen is the Function App service within Microsoft Azure. In the subsequent sections of this article, we cover the integration approach in two phases: Interoperability phase, which illustrates how the data from medical devices (used by the remote patient monitoring solution) can be converted into format suitable for transferring into healthcare data solutions in Fabric. Analytical processing and consumption phase, which provides the steps to turn the medical device data into insights that can be easily accessed through Fabric. Integration Approach Interoperability Phase Step 1 of this phase performs the transfer of proprietary device data. As part of this step, datasets are collected from medical devices and transferred (typically, in the form of files) to an integration platform or service. In our reference architecture, the datasets are trans ferred to the Function App (inside an Azure Resource Group) that is responsible for the integration function. It is important for these datasets to contain information about (at least) three concepts or entities: Medical device(s) from which the datasets are collected. Patient(s) to whom the datasets belong. Reading(s) obtained from the medical device(s) throughout the time that the patients utilize these devices. Medical device readings data may be point-in-time data capture, metrics, measures, calculations, collections, or similar data points. Information about the entities listed above will be used in the later step of interoperability phase (discussed below) when we will convert this information into resources to be transferred to the second phase that will perform analytical processing and consumption. In step 2, to maintain mapping between proprietary device data and FHIR® resources, you can use transformation templates, or follow a programmatic approach, to convert datasets received from medical devices into appropriate FHIR® resources. Using the entities mentioned in the previous step, the conversion takes place as follows: Medical device information is converted to Device resource in FHIR® * . Patient information is converted to Patient resource in FHIR®. Device reading information is converted to Observation resource in FHIR®. * Currently, healthcare data solutions in Fabric supports FHIR® Release 4 (R4) standard. Consequently, the FHIR® resources that are created as part of this step should follow the same standard. Transformation and mapping activities are under the purview of each specific remote patient monitoring integration solution and are not reviewed in detail in this article. As an example, we provided below the high-level steps that one of the Microsoft partners (Life365) followed to integrate their remote patient monitoring solution with healthcare data solutions in Fabric: Life365 team developed a cloud-based transformation service that translates internal device data into standardized FHIR® (Fast Healthcare Interoperability Resources) Observations to enable compatibility with healthcare data solutions in Microsoft Fabric and other health data ecosystems. This service is implemented in Microsoft Azure Cloud and designed to ingest structured payloads from Life365-connected medical devices —including blood pressure monitors, weight scales, and pulse oximeters— and convert them into FHIR®-compliant formats in real time. When a reading is received: The service identifies relevant clinical metrics (e.g., systolic/diastolic blood pressure, heart rate, weight, SpO₂). These metrics are mapped to FHIR® Observation resources using industry-standard LOINC codes and units. Each Observation is enriched with references to the associated patient and device, formatted in NDJSON to meet the ingestion requirements in healthcare data solutions in Fabric. The resulting FHIR®-compliant data is securely transmitted to the Fabric instance using token-based authentication. This implementation provides a consistent, standards-aligned pathway for Life365 device data to integrate with downstream FHIR®-based platforms while abstracting the proprietary structure of the original device payloads. For examples from the public domain, you can use the following open-source projects as references: https://github.com/microsoft/fit-on-FHIR® https://github.com/microsoft/healthkit-to-FHIR® https://github.com/microsoft/FitbitOnFHIR® https://github.com/microsoft/FHIR®-Converter Please note that above open-source repositories might not be up to date. While they may not provide a complete (end to end) solution to map medical device data to FHIR®, they may still be helpful as a starting point. If you decide to incorporate them into your remote patient monitoring integration solution, validate their functionality and make necessary changes to meet your solution’s requirements. For the resulting FHIR® resources to be successfully consumed by the analytics processing later (within healthcare data solutions in Fabric), they need to satisfy the requisites listed below. Each FHIR® resource, in its entirety, needs to be saved as a single row into an NDJSON-formatted file. We recommend creating one NDJSON file per FHIR® resource type. That means creating Device.ndjson, Patient.ndjson, and Observation.ndjson files for the three entities we reviewed above. Each FHIR® resource needs to have a meta segment populated with inclusion of lastUpdated value. As an example: "meta":{"lastUpdated":"2025-05-15T15:35:04.218Z", "profile":["http://hl7.org/FHIR®/us/core/StructureDefinition/us-core-documentreference"]} Cross references between Observation and Patient, as well as between Observation and Device FHIR® resources need to be represented correctly, either through formal FHIR® identifiers or logical identifiers. As an example, the subject and device attributes of Observation FHIR® resource need to refer to Patient and Device FHIR® resources, respectively, in this manner: "subject":{"reference":"Patient/d3281621-1584-4631-bc82-edcaf49fda96"} "device":{"reference":"Device/5a934020-c2c4-4e92-a0c5-2116e29e757d"} For Patient FHIR® resource, if MRN is used as the identifier, it is important to represent the MRN value according to the FHIR® standard. Patient identifier is a critical attribute that it is used to establish cross-FHIR®-resource relationships throughout the analytics processing and consumption phase. We will review that phase later in this article. At a minimum, a Patient identifier, which uses MRN coding as its identifier type, needs to have its value, system, type.coding.system, and type.coding.code (with value “MR”) attributes populated correctly. See an example below. You can also refer to a Patient FHIR® resource example from hl7.org. "reference": null, "type": "Patient", "identifier": { "extension": null, "use": null, "value": "4e7e5bf8-2823-8ec1-fe37-eba9c9d69463", "system": "urn:oid: 1.2.36.146.595.217.0.1", "type": { "extension": null, "id": null, "coding": [ { "extension": null, "id": null, "system": "http://terminology.h17.org/CodeSystem/v2-0203", "version": null, "code": "MR", "display": null, "userSelected": null } "text": null }, ... With Step 3, to perform the transfer of FHIR® resource NDJSON files to healthcare data solutions in Fabric: Ensure that the integration platform (Azure Function App, in our case) has permission to transfer (upload) files to the healthcare data solutions in Fabric: Find the managed identity or the service principal that the Azure Function App is running under: Navigate to the Azure portal and find your Function App within your resource group. In the Function App's navigation pane, under "Settings," select "Identity". Identify the Managed Identity (if enabled): If System-assigned managed identity is enabled, you'll see information about the system-assigned managed identity, including its object ID and principal ID. If User-assigned managed identity is linked, the details of that identity will be displayed. You can also add user-assigned identities here if needed. Service Principal (if applicable): If the Function App is configured to use a service principal, you'll need to look for the service principal within the Azure Active Directory (a.k.a. Microsoft Entra ID). You can find this by searching for "Enterprise Applications" within Azure Active Directory and looking for the application associated with the Function App. Grant Azure Function App’s identity access to upload files: Having been logged into Fabric with an administrator account, navigate to the Fabric workspace where your healthcare data solutions instance is deployed. Click on the “Manage Access” button on the top right. Click on “Add People or Groups” Add the managed identity or the service principal, which is associated with your Azure Function App, with Contributor access by selecting “Contributor” from the dropdown list. Using a coding environment, similar to the Python example provided below, you can manage the OneLake content programmatically. This includes the ability to transfer (upload) the NDJSON-formatted files, which have been created earlier, to the destination OneLake folder. from azure.identity import DefaultAzureCredential from azure.storage.filedatalake import DataLakeFileClient, DataLakeFileSystemClient # Replace with your OneLake URI onelake_uri = "https://your-account-name.dfs.core.windows.net" # Replace with the destination path to your file file_path = "/<full path to destination folder (see below)>/<entity name>.ndjson" # Get the credential credential = DefaultAzureCredential() # Create a DataLakeFileClient file_client = DataLakeFileClient( url=f"{onelake_uri}{file_path}", credential=credential ) # Upload the file with open("<entity name>.ndjson", "rb") as f: file_client.upload_data(f, overwrite=True) print(f"File uploaded successfully: {file_path}") The destination OneLake folder to use for the remote patient monitoring solution integration into healthcare data solutions in Fabric is determined as follows: Navigate to the bronze lakehouse created with the healthcare data solutions instance inside the Fabric workspace. The lakehouse is named as “healthcare1_msft_bronze”. “healthcare1” segment in the name of the lakehouse points to the name of the healthcare data solutions instance deployed in the workspace. You might see a different name in your Fabric workspace; however, the rest of the lakehouse name (“_msft_bronze”) remains unchanged. Unified folder structure of healthcare data solutions is located inside the bronze lakehouse. Within that folder structure, create a subfolder named with the name of the remote patient monitoring solution you are integration with. See the screenshot below. This subfolder is referred to as namespace in healthcare data solutions documentation, and is used to uniquely identify the source of incoming (to-be-uploaded) data. NDJSON files, which have been generated during the previous interoperability phase, will be transferred (uploaded) into that subfolder. The full path of the destination OneLake folder to use in your file transfer (upload) code is: healthcare1_msft_bronze.Lakehouse\Files\Ingest\Clinical\FHIR®-NDJSON\<Solution-Name-as-Namespace> Analytics Processing and Consumption Phase Step 1 of this phase connects the interoperability phase discussed earlier with the analytics processing and consumption phase. As part of this step, you can simply verify that the NDJSON files have been uploaded to the remote patient monitoring solution subfolder inside the unified folder structure in bronze lakehouse of healthcare data solutions in Fabric. The path to that subfolder is provided earlier in this article. After the upload of the files has been completed, you are ready to run the data pipeline that will perform data ingestion and transformation so that the device readings data may be used for analytics purposes. In the Fabric workspace, where healthcare data solutions instance is deployed, find and open the data pipeline named “healthcare1_msft_omop_analytics”. As is the case with the bronze lakehouse name, “healthcare1” segment in the name of the data pipeline points to the name of the healthcare data solutions instance deployed in the workspace. You might see a different name in your Fabric workspace depending on your own instance. This data pipeline will execute four activities, first of which will copy the transferred files into another subfolder within unified folder structure so that they can be input to the ingestion step next. Then, the subsequent pipeline activities perform steps 2 through 4 as illustrated in the analytics processing and consumption phase diagram further above. Step 2 ingests the content from the transferred (NDJSON) file(s) to the ClinicalFHIR delta table of the bronze lakehouse. Step 3 transforms the content from the ClinicalFHIR delta table of the bronze lakehouse into flattened FHIR® data model content inside silver lakehouse. Step 4 transforms the flattened FHIR® content of silver lakehouse into OMOP data model content inside gold lakehouse. As part of step 5, you can develop your own gold lakehouse(s) through transforming content from the silver lakehouse into data model(s) best suited for your custom analytics use cases. Device data, once transformed into a gold lakehouse, may be used for analytics or reporting through several ways some of which are discussed briefly below. In step 6, Power BI reports and dashboards can be built inside Fabric that offer a visual and interactive canvas to analyze the data in detail. (Overview of Power BI - Microsoft Fabric | Microsoft Learn) As part of step 7, Fabric data share feature can be used to grant teams within external organizations (that you collaborate with) access to the data (External data sharing in Microsoft Fabric - Microsoft Fabric | Microsoft Learn). Finally, step 8 enables you to utilize the discover and build cohorts capability of healthcare data solutions in Fabric. With this capability, you can submit natural language queries to explore the data and build patient cohorts that fit the criteria that your use cases are aiming for. (Build patient cohorts with generative AI in discover and build cohorts (preview) - Microsoft Cloud for Healthcare | Microsoft Learn) Conclusion When integrated with healthcare data solutions in Fabric, remote patient monitoring solutions can enable transformative potential in enhancing patient outcomes, optimizing care coordination, and streamlining healthcare system operations. If your organization would like to explore the next steps in such a journey, please contact your Microsoft account team.Building Healthcare Research Data Platform using Microsoft Fabric
Co-Authors: Manoj Kumar, Mustafa Al-Durra PhD, Kemal Kepenek, Matt Dearing, Praneeth Sanapathi, Naveen Valluri Overview Research data platforms in healthcare providers, academic medical centers (AMCs), and research institutes support research, clinical decision making, and innovation. They consolidate data from various sources, making it accessible for comprehensive analysis and fostering collaboration among research teams. These platforms automate data collection, processing, and delivery, reducing time and effort needed for data management. This allows researchers to focus on their core activities while ensuring data security and regulatory compliance. The ability to work with multimodal data encourages interdisciplinary and interorganizational collaboration, uniting experts to address complex healthcare challenges. Current challenges Researchers face many common challenges as they work with multimodal healthcare data: Data integration and curation: The process of integrating various data types, such as clinical notes, imaging data, genomic information, and sensor data, presents significant challenges due to differences in formats, standards, and sources. Each AMC employs unique methods for data curation, with some utilizing on-premises solutions and others adopting hybrid cloud systems. No standardized approach currently exists for data curation, necessitating considerable organizational efforts to ensure data consistency and quality. Furthermore, data deidentification is often required to safeguard patient privacy. Data discovery and building cohorts: The lack of a unified multimodal data platform leads to the segregation of data across different modalities. Cohort discovery for each modality is performed separately and often lacks a self-service option, necessitating additional human resources to assist researchers in the data discovery process. This issue is particularly significant because researchers who require Institutional Review Board (IRB) approval cannot access the data beforehand but still need an effective method to identify and explore cohorts. Data delivery: Sensitive patient data, after institutional review board approval, must comply with privacy regulations like the Health Insurance Portability and Accountability Act (HIPAA) and the General Data Protection Regulation (GDPR), requiring secure transfer to prevent breaches. The data, sourced from various systems, needs processing for research readiness. Delivering unified data from modalities like imaging, genomics, and health records is challenging. Typically, research IT teams curate cohort data and deliver it to an SQL database or a file share, accessed by researchers via secure virtual machines. This method often leads to data duplication, creating significant overhead due to numerous ongoing research projects. Cost management: Research projects are funded by government grants and private organizations. Managing these costs is challenging. Research IT departments often implement chargebacks for transparency and accountability in resource use. However, there is a disconnect between funding models and operations. Research teams favor capital expenditure (CapEx) with upfront funding for long-term resources, while cloud platforms operate on operational expenditure (OpEx), incurring ongoing costs based on usage. This shift can lead to concerns about unpredictable costs and budgeting difficulties. Bridging this gap requires careful planning, communication, and hybrid financial strategies to align research needs with cloud-based systems. Compliance with regulations: Healthcare research uses sensitive patient data, requiring strict adherence to HIPAA and GDPR. Transparency in data handling is essential but complex. Researchers must document disclosures thoroughly, detailing who accessed the data and for what purpose. However, tracking and auditing are often fragmented due to inconsistent systems. Variability in disclosure requirements from different agencies adds to compliance challenges. Balancing an auditable trail with privacy and manageable administrative tasks is crucial. Research data platform requirements Ability to curate multi modal data into the research data platform Ability for researchers to identify cohorts (without seeing data) to submit data requests to IRB Automated data delivery after IRB workflow approves the request to access relevant data Tools for researchers as part of the same platform Secure and regulatory-compliant environment for research. An approach to building a research data platform using Microsoft Fabric This article serves as a guide to healthcare organizations, offering a point of view and a prescriptive guidance on building a research data platform using Microsoft Fabric. The solution uses several features from healthcare data solutions in Microsoft Fabric, including its discover and build cohorts capability, and features from the Fabric platform. Microsoft Fabric: is a unified, AI-powered data platform designed to simplify data management and analytics. It integrates various tools and services to handle every stage of the data lifecycle, including ingestion, preparation, storage, analysis, and visualization. Fabric is built on a Software as a Service (SaaS) foundation, offering seamless experience for organizations to make data-driven decisions. For additional details, refer to the following link: What is Microsoft Fabric - Microsoft Fabric | Microsoft Learn Healthcare data solutions in Fabric: Healthcare data solutions in Fabric help you accelerate time to value by addressing the critical need to efficiently transform healthcare data into a suitable format for analysis. With these solutions, you can conduct exploratory analysis, run large-scale analytics, and power generative AI with your healthcare data. By using intuitive tools such as data pipelines and transformations, you can easily navigate and process complex datasets, overcoming the inherent challenges associated with unstructured data formats. For additional details, refer to the following links: Healthcare data solutions in Microsoft Fabric - Microsoft Cloud for Healthcare | Microsoft Learn Discover and build cohorts: Discover and build cohorts (preview) capability in healthcare data solutions enables healthcare organizations to efficiently analyze and query healthcare data from multiple sources and formats. It simplifies the preparation of data for health trend studies, clinical trials, quality assessments, historical research, and AI development. It supports natural language queries for multimodal data exploration and cohort building, making it ideal for research and AI-driven projects. For additional details, refer to the following link: Overview of discover and build cohorts (preview) - Microsoft Cloud for Healthcare | Microsoft Learn The proposal for research data platform architecture builds upon the following foundational premises: Recognition of Fabric as the all-in-one data storage, processing, management and analytics platform with enterprise-level features around security, availability and self-service. Adoption of Fabric Workspace(s) as the security boundary (a secure logical container) for maintaining data platform items (data storage and processing assets). Fabric workspaces may be provisioned for and used by different research data platform stakeholders (groups of users) with different requirements around use cases, data privacy, data sensitivity and access security. Use of healthcare data solutions in Fabric, as the core capability to maintain healthcare data assets in a standard (interoperable) manner. Healthcare data solutions enables the storage and processing of several healthcare data modalities and formats that follow industry standards (for example, clinical modality in FHIR® NDJSON format and Clinical-Imaging modality’s DICOM® format). Industry standards make it easier for research data platform stakeholders to share (exchange) data and insights within their own organization as well as (when needed) with other organizations that they collaborate with. Use of Fabric native capabilities to address requirements that may not (yet) have been implemented for healthcare specific needs. This provides the research data platform stakeholders with the flexibility to develop various data storage and processing workloads easily in a low (or no) code manner. Fig – Conceptual architecture of research data platform in Microsoft Fabric Note: This diagram is an architectural pattern and does not constitute one to one mapping of existing Microsoft products. Organizing source data in data workspace (One Data Hub in the above diagram) Organize your enterprise data into a data workspace that could be leveraged for research purposes. This acts as a ‘One Data Hub’ for the research data platform. Multiple Lakehouse can be present in this workspace. There should be at least one Lakehouse that organizes data using ‘unified folder structure’ best practice. Convert data from non-supported format to healthcare data solutions supported format to leverage out of the box transformation for multimodal data: For healthcare data solutions supported modalities: Implement custom transformations to convert data to supported modalities/format. For unsupported modalities: Implement extensions to bronze Lakehouse to accommodate additional data modalities. Epic data availability: Epic supports FHIR data export using Bulk FHIR APIs. If your dataset meets the use cases of Epic Bulk Data, you can store the resulting FHIR resources into One Data Hub for further transformation. Avoid data content duplication: Data duplication cannot be totally avoided. However, the same file and same content are never duplicated. There will be situations when data needs to be transformed to suit the needs of existing transformation pipelines for accelerating research data platform development. Additionally, OneLake in Fabric storage, where Lakehouse is maintained, uses file compression. Healthcare data solutions in Fabric has functionality to compress raw files to zip and always writes structured data to delta parquet which is a higher compressed format. More information can be found here - Data architecture and management in healthcare data solutions - Microsoft Cloud for Healthcare | Microsoft Learn Curating data for research (One Analytics workspace in the above diagram) Implement and extend Silver Lakehouse: A flattened FHIR® data model is provided by healthcare data solutions out of the box within the Silver Lakehouse. Extending the existing data model is possible through adding new columns to existing tables or through adding new tables in the Silver Lakehouse. If there is a need to introduce a different data model altogether, it is best to implement it using a different Lakehouse. Implement and extend Gold Lakehouse: Deploy and extend Observational Medical Outcomes Partnership Common Data Model (OMOP CDM): Deploy OMOP CDM 5.4 out of the box with healthcare data solutions deployment. Extend OMOP CDM to accommodate additional modalities. For example, implement Gene sequencing, Variant occurrence and Variant annotation tables to add genomics modality into OMOP CDM or implement medical imaging data on OMOP CDM as described here - Development of Medical Imaging Data Standardization for Imaging-Based Observational Research: OMOP Common Data Model Extension - PubMed Implement custom Gold Lakehouse(s): Implement other custom Gold Lakehouse using Fabric tools that run your transformation logic from Silver to Gold. These Lakehouse cannot be connected to discover and build cohorts capability within healthcare data solutions. Customers that need access to custom data can connect their custom cohort browsers to the SQL Analytics Endpoint(s) of their custom Gold Lakehouse(s). Enable data de-identification: Microsoft provides several solutions that can be used to implement a comprehensive de-identification solution that customers expect. Refer to the articles below for details. Dynamic data masking in Fabric Data Warehouse - Microsoft Fabric | Microsoft Learn Row-level security in Fabric data warehousing - Microsoft Fabric | Microsoft Learn Column-level security in Fabric data warehousing - Microsoft Fabric | Microsoft Learn Announcing a de-identification service for Health and Life Sciences | Microsoft Community Hub Cohort discovery using cohort builder tool Microsoft’s cohort browser: Today Discovery and Build Cohort supports eyes-on cohort discovery. This is an out of the box solution that is part of healthcare data solutions in Fabric. When eyes off discovery is supported, researchers as well as research IT can benefit from both eyes off and eyes on discovery and cohort building. 3rd-party cohort browser (e.g., OHDSI Atlas): Most 3rd party cohort browsers (E.g. OHDSI Atlas) and home-grown cohort browsers typically support connection to a SQL endpoint. Microsoft Fabric platform provides the capability of exposing SQL endpoint from a Lakehouse that can be connected to a 3rd party cohort browser to perform cohort discovery. Automated data delivery Creating research workspaces with cohort needed for research: Create separate workspaces for different research projects to keep Fabric items distinct and project specific using Fabric APIs. Assign workspaces to a Fabric capacity: Note: When needed, and if the organization has more than one Fabric capacity provisioned, workspace assignment can be spread across different capacities to help manage cost and performance. Next, set up a Lakehouse and provide access for team members (as per IRB approval list). This ensures both access and security at the workspace level. Export data to research workspace (format desired by researchers): Currently, DBC exports data as CSV/JSON files stored in a Lakehouse within the same workspace. Shortcut the destination Lakehouse into research workspace to keep the sanity of cohort data. Tools for researchers: Fabric provides several data engineering and data science tools out of the box that researchers can leverage to perform research. The following are some of the documents that customers can use to enable researchers with the tools of choice. Data science in Microsoft Fabric - Microsoft Fabric | Microsoft Learn Create, configure, and use an environment in Fabric - Microsoft Fabric | Microsoft Learn Migrate libraries and properties to a default environment - Microsoft Fabric | Microsoft Learn Charge back: Fabric compute pricing depends on the chosen Fabric capacity SKU. Assigning different Fabric capacities to different projects or groups within the same cost center can facilitate chargeback. See the step mentioned above on assigning a workspace to a Fabric capacity during workspace creation. Manage historic data migration to the research data platform on Fabric In most instances, customers already possess a research data platform. They seek to transition to this proposed solution without disrupting their current research data flow and obligations. Follow this approach to migrate or use data from the existing platform to the new one: Use your current research data platform as a Lakehouse or a Data Warehouse in Fabric (take advantage of Shortcut and Mirroring features available in Fabric). Fabric offers cross-database query, i.e. allowing to query and join multiple Lakehouse and data warehouses in a single query. Customers can choose how and where to implement such queries to augment the healthcare data solutions datasets with their existing datasets, all natively in Fabric. A bridge/mapping layer can be built to link the old and the new in a cross-relational way. Conceptually, such an approach has also ties to Bring Your Own Database (BYO-DB) requirement, which is the ability to bring custom defined format and still be able to easily convert to healthcare data solutions specific format. Other workflow integration Integrate research data platform with IRB workflow: IRB workflows are dependent on the tools utilized. For instance, eIRB solution from Huron. While there is currently no direct integration between IRB workflows and the research data platform on Fabric, it is possible to develop a connector using Power Platform integration with Fabric. Specific details are not available at this time as this remains an exploratory initiative. Another approach will be to use Fabric REST APIs (as a pro-code method) that can enable richer integration between Fabric and the 3 rd -party system, and a better consuming user experience at the end. Capture logs necessary for “accounting of disclosures”: Logs in Fabric can be captured at event level. It’s up to the customer to decide the level and type of logs that need to be captured for accounting of disclosure. This will need some custom implementation. One such capability of Fabric that can be used is: Track user activities in Microsoft Fabric - Microsoft Fabric | Microsoft Learn FHIR® is a registered trademark of Health Level Seven International, registered in the U.S. Trademark Office and is used with their permission. DICOM® is the registered trademark of the National Electrical Manufacturers Association (NEMA) for its Standards publications relating to digital communications of medical information. If you are a Microsoft customer needing further information, support, or guidance related to the content in this blog, we recommend you reach out to your Microsoft account team in order to set up a discussion with the authors.
