Modification of DICOM metadata in Microsoft Fabric: A collaboration with Fujifilm’s Synapse® VNA

Introduction

In the context of enterprise healthcare systems, imaging studies are often archived in a Vendor Neutral Archive system (VNA), which represents the data source of DICOM and non-DICOM files in the healthcare data solutions in Fabric. As the name suggests, the VNA receives imaging studies from different specialty departments, i.e. modalities, such as Radiology, Cardiology, Oncology…etc. within the healthcare organization, and stores those imaging studies in a standard DICOM (DCM) file format on an archival storage. Modern VNA systems offer cloud-based archival storage, such as Blob Storage in Microsoft Azure.
Another key feature of VNA systems is to allow and maintain administrative updates and corrections made to already archived imaging studies. This may include modifications to patient ID, patient demographics, study descriptions, study deletes, and study merges. VNA systems follow a common pattern when processing such modifications to archived imaging studies. VNA systems generally maintain those modifications in an internal object store (for example internal database) and may not apply those modifications to the DCM files in the archival storage for the sake of efficiency. Only when an imaging study is retrieved, or exported, from the VNA, will the VNA then update relevant metadata to ensure the exported imaging study has the latest and greatest information.

The Challenge

This VNA pattern introduces a new challenge to healthcare data solutions in Fabric, that is, the DCM files may “drift” indefinitely,  which they may no longer contain the latest and greatest modifications introduced by the VNA after ingestion in the healthcare data solutions in Fabric. To solve this challenge, healthcare data solutions need to (1) receive those modifications in a standard format from the VNA, and (2) update the DICOM data in the related tables across the medallion Lakehouse. In doing so, the data state in healthcare data solutions in Fabric will be representative of that in the VNA, the source system of imaging studies. Albeit, healthcare data solutions will not update the underlying DCM files, as those should only be updated by the authoritative source system which is the VNA.

The Solution

Healthcare data solutions in Fabric now provide a new functionality to update the DICOM data in the Silver Lakehouse delta tables, namely, [ImagingMestastore] and [ImagingStudy]. This is achieved through patch and delete operations that are defined in a FHIR-compliant schema in an NDJSON file format, and can be applied at the Study, Series, and Instance Level of the modified DICOM study hierarchy. VNA systems can generate these operation files to communicate a change, modification, or delete in DICOM studies to the healthcare data solutions in Fabric.

To support the Patch and Delete operations for imaging studies previously ingested through DCM files, we are introducing FHIR Extensions at the ImagingStudy FHIR resource level, utilizing the DICOM Web JSON model to specify tag value updates. The Study, Series, and Instance identifiers as well as specific extension tags will be used in those operations. No other ImagingStudy properties will be used in those operations as part of the ImagingStudy FHIR resource. The following sections will provide examples of all three operations and their FHIR Extensions.

For more details, please refer to our public documentation page: Process DICOM modifications

A collaboration with Fujifilm’s Synapse^® VNA

This healthcare data solutions capability was developed through technical collaboration with the team from Fujifilm’s Synapse VNA. Synapse VNA is an industry-leading imaging interoperability solution that connects DICOM and non-DICOM content across the care continuum, helping organizations gain a more complete view of patient health.

Building on the Native Cloud Tiering announced last year, the Synapse VNA version 7.6.3   can now not only archive the imaging files (i.e. DCM files) on Azure Blog storage, but more importantly, it can communicate any modifications of the DICOM metadata through the FHIR extensions pattern provided with this new capability to the medallion Lakehouse in healthcare data solutions in Fabric. At a high level, the Synapse VNA will have two different pre-configured Azure blob storage containers; a DICOM Files container and Patch and Delete Operations container. The former will store the binary DICOM imaging data (i.e. dcm files), whereas the latter will store the modification operations in FHIR NDJSON file format. Leveraging the OneLake Shortcuts, the healthcare data solutions will shortcut those two containers to the respective locations in the Unified Folder Structure in the Bronze Lakehouse, from which the DICOM transformation pipeline will run to ingest the data in the Silver and Gold Lakehouse.

This collaborative effort between Fujifilm and Microsoft continues to extend beyond development. Our teams are working together on a real-world implementation for an early adopter customer, enabling billions of DICOM imaging studies to be archived in Synapse VNA and ingested into healthcare data solutions in Fabric. We look forward to sharing more updates as this collaboration progresses and new milestones are reached.