Healthcare and Life Sciences Blog articles

What We Took Away from Build 2026 as Microsoft Field and Solution Engineers

michaelgoad — Wed, 10 Jun 2026 22:10:54 GMT

I pulled together four Microsoft Solution Engineers for a candid conversation about Build 2026, and I want to be upfront about what this is. No keynote recap, no announcement list. Four people who work with healthcare and enterprise customers every day, talking about what caught our attention and why.

I was joined by Arlie Hartman, a former CISO and now Principal SE with deep expertise in Purview and security; Thor Draper Jr., Senior Security SE covering security and identity; and Adam Hallbeck, a Principle Power Platform Copilot SE. Between the four of us we cover a pretty wide slice of the Microsoft field.

What we actually talked about

Microsoft Scout was one of the first things we got into. It's still frontier private preview, but I've been using it and it's already changing how I work. I had it reach out to Arlie on my behalf to schedule time, monitor his response, and book the meeting when he replied. It worked. That shift from "AI you type at" to "AI that does things while you're doing other things" is real, and Scout is one of the clearest examples of it right now.

Agent 365 got a lot of our attention too, especially the identity piece. Thor made a point that stuck with me: organizations are spinning up agents faster than they can track them. One customer had 1,400 agents and no clear picture of what any of them had access to. Agent 365 gives you sponsors, owners, and governance controls that map to how IT teams already manage people and service accounts. The mental model isn't new, the application to agents is.

Adam talked through Work IQ APIs and where that's heading, including how much easier it's getting to give agents access to organizational knowledge without having to wire up every tool individually. Arlie covered the Purview SDK changes in Foundry, which are a big deal for regulated industries because it moves the security configuration responsibility off developers and back to governance teams where it belongs. Thor walked through M-Dash and what a real DevSecOps process looks like when agents are doing the reconnaissance and triage work.

We also talked about Microsoft's in-house model lineup and what it means as models start to get treated more like infrastructure than differentiators. Copilot routing the right prompt to the right model quietly does a lot of work that most users never see.

The takeaway from our side of the field

Build 2026 felt different from where I sit. A year ago we were talking about what agents could do. Now we're talking about how to govern the ones already running. The product is catching up to the conversations we've been having with customers, and that matters a lot in healthcare where "we'll figure out governance later" isn't an option.

Watch the full conversation above. I'd love to hear what questions it surfaces for your organization.

A new chapter of efficient foundation models for medical imaging

ivantarapov — Thu, 11 Jun 2026 20:52:11 GMT

Authors: Ivan Tarapov, Naiteek Sangani, Mu Wei, Noel Codella, Mert Oez and Naveen Valluri, Microsoft Healthcare and Life Sciences

At HLTH 2024, we introduced three open-source healthcare AI foundation models on Microsoft Foundry: MedImageInsight (MI2), CxrReportGen (CXRRG), and MedImageParse (MIP). They quickly became the most popular healthcare industry models in the Foundry catalog, with consistent, growing usage by researchers, ISVs, and clinical teams over more than 15 months. Customer momentum has been real:

SECTRA has explored integrating MedImageInsight for real-time exam parameter determination.
University of Wisconsin is exploring the use of CxrReportGen to automate normal-case triage, focusing radiologists on complex work.
Milvue is fine-tuning CxrReportGen to extend its capabilities into musculoskeletal pathologies and image-based reporting.

Early experimentation with these models across different scenarios provided clear signals on where customers needed more support. Across these efforts, four themes consistently emerged:

“We don’t want to manage our own VMs.” Radiology IT or ML platform teams don’t want to manage GPU infrastructure for provisioning and scaling.
“We want to pay for what we use.” Idle GPUs are a tax. Customers want elastic endpoints that scale with workload.
“We need enterprise-ready endpoints.” Production deployments require HIPAA coverage, BAAs, SLAs, managed security, and a pathway to integration into the surfaces clinicians already use — PowerScribe, PowerShare Image Sharing, and PACS.
“We want a simplified fine-tuning process.” Foundation models require fine-tuning for integration into clinical workflows. Fine-tuning needs to be fast, efficient, and easy to perform.

We set out to address these priorities to make it easier for customers to experiment, evaluate, and build on these models.

Introducing healthcare AI premium models

Microsoft now offers premium versions of its medical imaging foundation models on Foundry, available today as a private preview to approved customers. The trusted core architecture has been further optimized and enhanced, with frequent updates using larger, curated datasets and the models are delivered as fully managed endpoints. With the introduction of these premium models, we are committing to maintaining state-of-the-art performance across a broad range of medical image processing tasks.

Premium models on Foundry are designed for teams aiming to build medical imaging AI - including developers, ISVs, and enterprise health systems standardizing on managed AI endpoints within their Azure environments. They provide a strong starting point for researchers and advanced practitioners who want to experiment and extend capabilities without requiring access to large, specialized datasets.

These models are not medical devices and are not intended for out-of-the-box clinical use or autonomous clinical decision-making. Instead, they are designed to be fine-tuned, validated, and deployed within customer-controlled workflows, with appropriate human oversight and regulatory processes applied by the implementing organization.

Unlike the open-weight versions of the models which run on a dedicated virtual machine hosted in customer subscription, the premium models are available as “managed endpoints” (also known as “serverless models“) where the models are hosted on Microsoft-owned infrastructure. Customers benefit from a more flexible, “pay-as-you-go” billing model with charges based on images processed rather than per hour of VM uptime.

These first premium models - MedImageInsight Premium and CxrReportGen Premium bring managed, commercial-ready delivery to two high-value imaging scenarios: multimodal image encoding and chest X-ray findings generation.

MedImageInsight Premium: a multimodal embedding model for medical imaging

MedImageInsight Premium builds on the open-source version with up to 16% performance gains on imaging benchmarks. It’s a single general-purpose embedding backbone across nine modalities including X-ray, CT, MRI, ultrasound, pathology, dermoscopy, OCT, fundus photography, and mammography, that powers:

Image quality analysis and study routing
Image-to-image similarity search
Zero-shot and fine-tuned classification
Metadata extraction and outlier detection
Downstream report generation (as an encoder)

How does it compare to other similar models?

The table below summarizes results from a published comparison of radiographic classification performance across several foundation-model backbones. In the benchmarks shown, MedImageInsight Premium achieves the highest reported mAUC among the models listed, supporting its use as a general-purpose imaging representation for downstream classification and fine-tuning workflows.

	MedImageInsight Premium	MedImageInsight OSS	MedSigLip
LTCXR (Chest Xray mAUC)	0.83	0.79	0.74
IRMA2009 (X-ray Exam Params)	0.94	0.92	0.90
BUSI (Breast Cancer Ultrasound mAUC)	0.98	0.97	0.93
HMCQU (Echocardiography view mAUC)	0.97	0.97	0.86
PCAM (Histopathology mAUC)	0.95	0.94	0.93
RSNAMAMM (Mammography AUC)	0.86	0.84	0.77
Gastrovision (Endoscopy (GI) AUC)	0.92	0.89	0.89

Why it matters: a stronger embedding space can help reduce the amount of labeled examples needed for downstream tasks, lower fine-tuning cost, and support faster iteration during model evaluation and application development.

CxrReportGen Premium: chest X-ray findings generation

CxrReportGen Premium generates a structured list of findings for chest X-rays. It accepts current and prior studies plus clinical context such as indication, technique, and comparison and can run inference in well under a second. It offers best-in-class performance and a solid foundation for production workloads.

In Microsoft testing on a proprietary real-world dataset, CxrReportGen Premium showed an approximately 330% performance increase compared to the open-weight version. Results may vary by dataset, workflow, and evaluation approach, so teams should validate performance using their own data and clinical requirements.

Findings generation on MIMIC-CXR dataset:

	CXRReportGen Premium	CXRReportGen OSS
1/RadCliq-v1	1.26	1.34
BLEU-2	0.27	0.31
BERTScore	0.47	0.50
SembScore	0.50	0.50
RadGraph	0.31	0.31

Findings generation on proprietary real-world dataset

	CXRReportGen Premium	CXRReportGen OSS
1/RadCliq-v1	2.74	0.83
BLEU-2	0.38	0.16
BERTScore	0.57	0.37
SembScore	0.60	0.43
RadGraph	0.43	0.18

“Milvue is building a radiology-native VLM. By working with Microsoft and leveraging CXRReportGen, we could start from a strong foundation allowing our team to focus on what matters most: turning foundation-model capability into clinically validated, workflow-ready radiology solutions.”

— Alexandre Parpaleix, Co-Founder/CEO, Milvue (milvue.com)

How does it compare to other similar models?

The comparison below highlights how CxrReportGen Premium differs from open-source and publicly available chest X-ray reporting approaches across availability, output style, grounding, training data, and benchmark context.

Model	Availability	Output Style	Grounded Reporting	Key Training Data	Benchmark Performance
CXRReportGen Premium	Internal (closed)	Findings ± Impression	Yes	MIMIC-CXR + proprietary	Internal only: 1/RadCliQ↑ 1.88; RadGraph F1↑ 0.36
CXRReportGen (open)	Public (MIT)	Grounded findings	Yes	MIMIC-CXR + proprietary	MIMIC-CXR: RadGraph F1↑ 1.34; ROUGE-L↑ 39.1
UniRG-CXR	Public (research)	Findings ± Impression	No	ReXrank datasets	RexRank family: RadGraph F1↑ ~0.26–0.40
MAIRA-2	Public (research)	Findings ± Impression	Yes	CXR reporting data + prior/context inputs	ReXrank family: RadGraph F1↑ ~0.13–0.23
CXR-RePaiR	Public (MIT)	Free-text retrieval	No	MIMIC-CXR corpus; CheXpert eval	External CheXpert: F1↑ 0.352
R2Gen	Public	Free-text reports	No	IU X-Ray + MIMIC-CXR	External CheXpert: F1↑ 0.191
CvT2DistilGPT2	Public (GPL-3.0)	Free-text reports	No	MIMIC-CXR + IU X-Ray	ReXrank family: RadGraph F1↑ ~0.10–0.27
MedGemma	Public	Findings / report-like	No / mixed	Broad medical multimodal data	ReXrank family: RadGraph F1↑ ~0.14–0.27

Benchmarks cited here are not apples-to-apples comparisons. “Internal / Proprietary Performance” reflects Microsoft internal or proprietary evaluations for CXRReportGen Premium/open CXRReportGen. “Public Benchmark Performance” reflects the best headline metric explicitly reported in the public benchmark family cited for each model (for example, MIMIC-CXR card metrics, CheXpert clinical-efficacy metrics, or ReXrank-style RadGraph results). Metric families represented here include composite quality metrics such as 1/RadCliQ, clinical structure metrics such as RadGraph F1, and lexical metrics such as ROUGE-L

Building our premium healthcare AI models

The premium models achieve a different performance checkpoint trained on a materially larger, curated, clinically-vetted data mix and are designed for an ongoing training cadence. Key details developers typically ask about include:

Curated, licensed data mix. Expanded coverage across modalities and underrepresented pathologies, with particular attention to dataset provenance, patient de-identification, and license compatibility with commercial deployment.
Architecture-compatible. Same embedding dimensionality for MedImageInsight and same input/output contract for CxrReportGen, so code written against the OSS endpoints can move over with minimal changes.
Fine-tuning friendly. Stronger base representations can help reduce the amount of data needed and thus support more efficient fine-tuning and adapter-based specialization.
Managed inference. Packaged and deployed on Azure ML managed endpoints with elastic autoscaling, using A100-class GPUs.

Responsible AI

Every premium model has gone through Microsoft’s Responsible AI review, and Data Science Board review — the same internal gates that govern our 1P healthcare AI surfaces. Each model ships with a detailed model card covering intended use, out-of-scope use, dataset provenance, known limitations, subgroup performance, and evaluation methodology. These models are not medical devices and are not cleared for autonomous clinical decision-making. They are designed to be fine-tuned, validated, and deployed by customers into their own clinical workflows, with Microsoft supporting partners through SaMD submission paths where applicable.

Managed Security

Managed endpoints in Foundry act as a secure, policy-enforced gateway to model execution rather than direct access to underlying models. Every request to an endpoint is authenticated and authorized through Microsoft Entra ID or scoped API credentials so that only approved users and services can invoke specific deployments. Endpoints can be isolated within private networks and exposed through controlled access paths to control unintended public exposure. All data is encrypted in transit using TLS and protected at rest within Azure, while built-in controls such as content filtering, rate limiting, and activity logging provide continuous governance and auditability. As Azure resources, these endpoints inherit enterprise-grade security, compliance, and monitoring capabilities - helping ensure that model access remains tightly controlled, observable, and aligned with organizational policies.

Open source vs premium at a glance

The driving force behind premium models is enabling teams to deploy solutions faster, with less overhead. For many teams, the biggest gain isn’t just in model quality - it’s in how these capabilities are delivered and supported in production. Premium models are designed to remove the operational burden of managing infrastructure while giving organizations the flexibility, commercial terms, and integrations they need to move from experimentation to deployment.

Fully managed endpoints. No GPU VMs to provision, patch, or scale. Elastic by default.
Pay for what you use. Usage-based inference pricing helps reduce idle GPU costs by charging for model usage rather than continuously running self-managed infrastructure
Integrated where you work. Designed to plug into PowerScribe via Dragon Copilot for radiology
Improved performance. Unlike the open-source models, the premium models achieve a different performance checkpoint trained on a materially larger, curated, clinically-vetted data mix and are designed for an ongoing training cadence.

	Open source (2024)	Premium (2026)
Benchmark performance	SOTA baseline at release	+7–15% on imaging benchmarks
License	Research / academic only	Commercial use
Deployment	Self-managed GPU VMs	Fully managed Microsoft Foundry endpoints
Pricing	Compute you provision	Per-hour inferencing — pay for what you use
Scaling	Customer-managed	Autoscale, elastic from zero
Fine-tuning	DIY via Azure Machine Learning	Native Foundry UX

Comparing costs

From a cost perspective, the tradeoff between managed endpoints and self-managed infrastructure depends on the workload.

Customers incur a fixed cost for a self-managed endpoint regardless of how many images processed. With serverless models the costs are incurred only when images are processed by the models.

As a general guideline, self-managed endpoints may become more cost-effective at higher sustained throughput, while managed endpoints can be more cost-efficient for lower or variable workloads, especially when operational overhead for GPU infrastructure is included. The exact break-even point depends on model choice, endpoint configuration, utilization pattern, and current pricing. With launch private preview pricing self-managed endpoints break even with managed endpoints at approximately 6,000 images/hour for MedImageInsight and 1,500 images/hour for CxrReportGen.

Getting started

MedImageInsight Premium and CXRReportGen Premium are available in a limited preview. To request access, use the links below:

MedImageInsight Premium: https://aka.ms/hls/mi2premium
CxrReportGen Premium: https://aka.ms/hls/cxrrgpremium

To learn more about premium models as well as other models for healthcare and life sciences in the Foundry catalog, as well as review additional resources, take a look at our project page.

Premium models are one part of a broader shift in how healthcare organizations adopt and operationalize AI. For a wider industry perspective on how these capabilities are shaping healthcare delivery, read more in our latest industry blog.

Modernizing radiology reporting—without disrupting care: A practical path to PowerScribe One

Jeanne_Nauman — Tue, 09 Jun 2026 16:00:00 GMT

With growing imaging volumes, increasing complexity, and the rapid emergence of AI, healthcare organizations are reevaluating how their reporting environments support clinicians and strengthen operational performance. They are faced with how to modernize without interrupting the work that matters most. We developed our PowerScribe One solution and implementation approach with that reality in mind.

In active production across a wide range of healthcare environments (including large integrated delivery networks, academic medical centers, independent radiology practices, and community hospitals), PowerScribe One reflects a solution that is both proven in practice and designed for what comes next. Over 250 organizations and 10,000+ radiologists use PowerScribe One to generate millions of reports each month.

This scale is significant; it’s validation of what we bring through our solution and support. It reflects a system and team tested across diverse environments, integration landscapes, and operational models, performing reliably in real-world conditions. Combining the strength of our solutions with an experienced Microsoft team, we deliver a seamless implementation that minimizes disruption.

Redefining the migration experience

As I’ve worked with customers modernizing their reporting environments, I’ve noticed a consistent pattern of concern: how to modernize without disrupting the workflows teams rely on or the care they deliver.

In my experience, even when a solution offers meaningful capabilities, customers still worry about the potential downsides of a prolonged migration. I understand that perspective. Many have worked with vendors who promise a “lift-and-shift” implementation but fall short of that expectation.

Migrations can introduce real challenges, including downtime, retraining, and workflow disruption. Over time, we’ve seen that successful transformation is driven not only by the strength of the technology, but also by how effectively the transition is managed.

With years of experience supporting PowerScribe environments, we’ve taken those insights and applied them to our approach. We defined what a successful PowerScribe One implementation looks like and developed a migration model designed to reduce risk while supporting adoption.

Rather than viewing migration as a single milestone, PowerScribe One transitions are designed as a structured journey with clearly defined phases and timing:

Discovery: Align on goals, workflows, and integration requirements
Build: Preparing the technical and operational foundation
Testing: Validating workflows end to end and addressing issues proactively
Production: Supporting go-live with a focus on stability and adoption

Each phase includes checkpoints and shared accountability to increase transparency and reduce uncertainty. Our Microsoft team works closely with our customers to build a project timeline that fits their needs while existing PowerScribe 360 workflows and content are leveraged, eliminating the need to rebuild from scratch.

I’d also like to highlight at the center of this migration model is a parallel transition strategy. We enable PowerScribe 360 and PowerScribe One to operate side-by-side during our customer’s migration period.

This approach provides organizations with the flexibility to:

Introduce PowerScribe One to early adopters
Validate workflows and integrations in a live environment
Phase adoption across teams
Maintain continuity throughout the transition

In this measured approach, our customers can move forward with confidence, ensuring that systems, workflows, and teams are ready for successful adoption. The result is a model that is both repeatable and adaptable, capable of supporting organizations with varying levels of complexity. Our efforts to center our implementation process on the customer experience shows how migration work is not simply a technical capability proposition. This focus reflects our broader philosophy: transformation should be deliberate, not disruptive.

From implementation to enablement

Minimizing disruption doesn’t end with implementation. It extends into how teams are supported, trained, and enabled in their day-to-day workflows. Our focus on enablement is especially important in radiology, where even small workflow disruptions can have outsized impacts on productivity and the radiologist experience.

With PowerScribe One, organizations not only gain access to a modern reporting solution but also support from teams with deep experience in radiology workflows, integrations, and large-scale deployments.

With that in mind, I’ve seen firsthand how the healthcare landscape has radically changed over the last six years. Our customers tell us how workforce shifts in radiology means they are adapting their staffing models and workflows to include telework. With these changes in the workforce, organizations benefit from training and support models that are flexible, digital, and accessible remotely.

We made live expert access (known to our customers as “drop-in help”) easily accessible through a simple QR code. It can be an ad hoc or scheduled engagement which ensures the offering aligns with a radiologist’s schedule. The feedback on this level of access we’ve received has been extremely positive and is resonating strongly with customers. I know that for any healthcare solution deployment to be successful, it requires a learning and support model that aligns with clinical schedules and operational realities.

Modernizing radiology reporting is both a technical and operational effort with its success depending on advancing capability without disrupting clinical continuity. The transition to PowerScribe One shows this balance is achievable through phased adoption, low-disruption deployment, and strong user readiness.

Organizations can modernize without affecting day-to-day care delivery with our structured approach, proven expertise and a focus on provider experience and patient outcomes.

If you want to learn more about our approach or PowerScribe One, I’ll be at SIIM26, June 10-12, please stop by the Microsoft booth at 630-632.

You can also discover how we partner with our customers why they decided to move to PowerScribe One by reading our Industry Blog.

Driving AI-Powered Healthcare: Advanced Analytics, AI, and Real-World Impact Workshop

CamilleWhicker — Thu, 04 Jun 2026 14:57:17 GMT

What We Covered

The evolving role of data in becoming a frontier AI organization
The modern data estate and how Microsoft Fabric unifies analytics
Architecture patterns for healthcare data platforms
Real-world healthcare and life sciences use cases driving impact
Building unified data foundations in Microsoft Fabric
Applying governance and security best practices
Activating data with AI and agent-based solutions

Key Takeaways

Unified data is foundational to scaling AI effectively
Microsoft Fabric simplifies the analytics stack and accelerates time to value
Governance and security must be built-in, not added later
AI-powered agents unlock new ways to operationalize data across clinical and business workflows
Hands-on experience is critical to moving from concept to deployment

Session Content and Resources

Workshop materials linked at the bottom of this post.

Becoming a Frontier Firm The State of Data & AI
The Modern Data Estate Inside Microsoft Fabric
Unified Data Foundation for Analytics Fabric as the Unifying Layer
Unlocking AI Securely Data Protection & Governance
Unified Data Foundation for AI Activating Data with Agents

What’s Next

If you’re looking to continue the momentum:

View our upcoming healthcare focused Data & AI workshops & webinars
Set up your free Microsoft Fabric trial:
Get started with SQL Fabric
Create a Data Agent
Discuss with your Microsoft partner
Join our upcoming virtual RTI Hands-on Lab June 11

Build Less. Deliver More. A lesson I learned building AI apps and workflows #Cowork

michaelgoad — Fri, 29 May 2026 21:03:42 GMT

I want to share something that happened recently, because I think it can be relatable.

A while back I built out a meta prompt for my weekly manager 1:1. If you've seen it, you know I put real thought into it. It scans my interactions across accounts, pulls out what matters, and turns it into a structured update I can walk into a meeting with. I was proud of that build, and honestly I still use a lot of it.

But then I decided to take it further.

I added a full briefing package to the workflow. An executive summary PowerPoint. An HTML web app covering four key areas. The whole thing automated and emailed out an hour before the meeting. I tested it, it ran clean, and I thought my manager was going to love it.

I asked him if he saw the email with the web app and he said he didn't see it yet, it got lost in his Outlook.

When he opened it and we went over it at the end of the meeting he said, "Hey, those four bullet points you sent me in Teams earlier? That's all I needed for this conversation."

That stung a little. But it was honestly some of the most useful feedback I've gotten.

The overbuild trap

Here's the thing about building with AI tools like Microsoft Copilot and Copilot Cowork. They make it fast and easy to create things we genuinely couldn't build on our own before. A PowerPoint in seconds. A web app from a prompt. An automated workflow that runs on a schedule. It's impressive, and it feels productive.

But fast and easy also means it's easy to build more than anyone actually needs.

I built for the output. I should have built for the person.

My manager didn't need a web app. He needed four clear points to drive a ten-minute conversation. The moment I understood that, the whole workflow got simpler, faster, and more useful.

Why this matters beyond just workflow design

As AI moves toward a consumption model, this lesson has a real cost attached to it. Every output your AI generates uses tokens. Every document it creates, every summary it writes, every email it sends — that's compute running in the background. If you're generating things nobody reads, you're spending budget on noise.

Knowing your audience isn't just good design practice. It's cost management.

Before you build, ask one question: what does this person actually need to do their job? Start there. Build that. If they need more, they'll tell you.

What I do now

My 1:1 workflow now does one thing. It looks through the past week, finds what matters across my accounts, and outputs four bullet points. That's what drives the conversation. No PowerPoint. No web app. Just the information my manager needs, in the format he'll actually use.

It took less time to build, costs fewer tokens to run, and works better than everything I built before it.

Build less. Deliver more.

Operationalizing AI powered medical imaging pipeline for cohort building

jaerwin — Thu, 28 May 2026 21:07:59 GMT

Authors:

Jared Erwin, Senior Software Engineer, HLS Nursing AI and Data Platform, Faculty UW School of Medicine
Manoj Kumar, Director, HLS - Data & AI HLS Frontiers AI
Alberto Santamaria-Pang, Principal Applied Data Scientist, HLS Frontiers AI and Adjunct Faculty, Johns Hopkins Medicine

Overview

In Part 1, of this series, we showed how natural language could be used to define medical imaging cohorts and retrieve relevant studies in seconds instead of months. That proof-of-concept demonstrated the value of the idea — but not how to make it repeatable, or production-ready.

This post focuses on how we turned that prototype into a production-oriented Azure Machine Learning pipeline — to scale execution and produce clear, versioned artifacts that could drive an interactive cohort exploration UI.

If you're building ML pipelines for medical imaging, or any domain where data is large, messy, and locked behind access controls, we hope our experience saves you time.

From scripts to a pipeline: Why Azure ML components?

The original hackathon implementation consisted of notebooks and scripts that required careful manual execution. To make the system repeatable and auditable, we standardized it using Azure ML pipelines.

Azure ML pipelines gave us:

Componentized execution — each processing step is a self-contained unit with defined inputs, outputs, and dependencies
Parallel branches — steps that don't depend on each other run concurrently
Reproducibility — every run is versioned and logged with full lineage
Compute flexibility — run on CPU for metadata extraction, GPU for model inference, without manual orchestration

The pipeline architecture

The pipeline consists of 5 python components arranged in a DAG with two parallel branches:

[0]scans a DICOM directory and extracts metadata from headers — study/series UIDs, modality, body part, slice counts.
[1]classifies each series by anatomy and orientation using a multi-tier strategy (more on this below).
[2] and [3] form the search pipeline: anatomy labels are converted to natural language text templates, then encoded with BiomedCLIP into a FAISS vector index.
[4]generates 2D UMAP coordinates from the embeddings for the interactive scatter plot visualization in the UI.

The image depicts a flowchart detailing the process of DICOM metadata extraction, anatomy classification, visualization enrichment, and text template generation, followed by the creation of a FAISS vector index.

Components 2 and 4 run in parallel after component 1 completes, saving roughly 10-15% of total execution time. It's a modest gain for a single run, but it adds up when iterating on pipeline parameters.

[1] Anatomy classification, integrating MedImageInsight

The Anatomy classification component in the pipeline relies on MedImageInsight (MI2). MedImageInsight is Microsoft's foundation model for medical image understanding, available through the Azure AI Foundry model catalog. Unlike generative models, MedImageInsight is an embedding model — it maps medical images and text into a shared 1024-dimensional vector space, enabling tasks like classification and similarity search by comparing image embeddings against text label embeddings.

Given a DICOM image, we compare its embedding against candidate labels (e.g., "Brain", "Chest", "Abdomen") to determine the body part, scan orientation, and other imaging characteristics through zero-shot classification.

We also may get directly annotated anatomy from component 0, the DICOM metadata extractor component. We can combine both data points to build our final search index.

[2] [3] FAISS index construction

As an input to the FAISS index, we first run component 2, the text template generator. This component takes the metadata and anatomy information from components 0 and 1 and feeds them into 5 different agents with different instructions on how to describe the DICOM study. This results in textual descriptions which some variation, referred to as text templates, which can be indexed in the next component

The FAISS index builder (component 3) uses BiomedCLIP to encode all text templates into 512-dimensional vectors:

MODEL_NAME = "hf-hub:microsoft/BiomedCLIP-PubMedBERT_256-vit_base_patch16_224" @torch.no_grad() def encode(self, texts: List[str], batch_size: int = 256) -> np.ndarray: embeddings = [] for i in range(0, len(texts), batch_size): batch = texts[i:i+batch_size] tokens = self.tokenizer(batch).to(self.device) batch_embeddings = self.model.encode_text(tokens) batch_embeddings = F.normalize(batch_embeddings, dim=-1) # L2 normalize embeddings.append(batch_embeddings.cpu().numpy()) return np.vstack(embeddings)

We L2-normalize all vectors and use faiss.IndexFlatIP (inner product), which is equivalent to cosine similarity on normalized vectors. For our current dataset sizes (thousands of series), flat indexing is fast enough. For hospital-scale datasets with millions of images, we might switch to IndexIVFFlat or IndexHNSW for approximate nearest neighbor search.

In the cohort explorer app, a user will enter a natural language query, which is then converted to embeddings using the same BiomedCLIP model. This allows a search using the FAISS index to find relevant DICOM studies.

[4] Visualization: making embeddings explorable

The scatter plot in the UI is often the first thing users interact with. It needs to show meaningful clusters without requiring users to understand dimensionality reduction.

Component 4 takes the embeddings from component 1 and projects them to 2D with UMAP:

umap = UMAP( n_components=2, n_neighbors=10, # Balances local vs. global structure min_dist=0.5, # Prevents over-clustering metric='cosine', # Matches our embedding similarity metric random_state=42 # Reproducible layouts ) coordinates_2d = umap.fit_transform(features)

Each point in the scatter plot corresponds to a single DICOM series produced by the pipeline, with color, grouping, and hover metadata derived directly from the JSON artifacts emitted by components 1 and 4.

Each pipeline run produces a small set of well-defined artifacts — metadata tables, embedding vectors, UMAP coordinates, and the FAISS index — which are consumed directly by the cohort exploration UI. The cohort explorer application can reload or switch between datasets.

The diagram is a screen capture of an Azure ML pipeline. It includes 5 pipeline components along with connecting arrows showing incoming and outgoing data, including the final outputs of the pipeline.

Pipeline execution: time, cost, and what we learned

Here's what a typical pipeline run looks like for a dataset of ~4,500 DICOM series:

Component	Task	Approximate Time (CPU)	Approximate Time (GPU)
0 - DICOM Metadata Extractor	Scan files, extract headers	5-10 min	5-10 min
1 - Anatomy Classification	Classify anatomy/orientation	90-120 min	5-10 min
2 - Text Template Generator	Generate 5 templates per series	5-10 min	5-10 min
3 - FAISS Index Builder	BiomedCLIP encoding + FAISS build	60-90 min	10-15 min
4 - Visualization Enrichment	UMAP + color assignment	20-40 min	5-10 min
Azure ML overhead	Compute provisioning, env setup	5-10 min	5-10 min
Total		~200-300 min	~30-50 min

Key observations:

Azure ML overhead is significant when doing quick iteration and testing. Compute provisioning, conda environment builds, and data mounting add several minutes before any component code runs. We first built each component as python code to run locally and debug before our first Azure ML run. This way we quickly iterated and avoided cost until we were ready.
BiomedCLIP encoding dominates on CPU. Component 3 is the bottleneck. Moving to GPU compute for this component cuts encoding time roughly in half, but GPU clusters cost more. For a pipeline you run occasionally, CPU is fine. For frequent re-indexing, GPU pays for itself.
Batch size tuning matters. The default BiomedCLIP batch size of 256 balances memory and throughput. On GPU, you can push to 512. On CPU with limited RAM, drop to 128.

At Scale: 120,000 Images, CPU vs. GPU

We ran the full pipeline against a larger dataset of ~120,000 images to understand how compute choice affects end-to-end time and cost:

	CPU Pipeline	GPU Pipeline
Pipeline compute time	4 days, 12 hours (108 hrs)	15 hours
Pipeline compute cost	~$0.25/hr × 108 hrs = ~$27	~$3.00/hr × 15 hrs = ~$45
MedImageInsight endpoint (MaaP on Standard_NC4as_T4_v3)	~$151	~$21
Total estimated cost	~$178	~$66

Both pipeline runs make the same ~120,000 classification calls to the MedImageInsight endpoint, but those calls are spread out over different time periods depending on how quickly and efficiently the pipeline can make the calls to MedImageInsight. The hourly cost for MedImageInsight on a Standard_NC4as_T4_v3 VM is ~$1.40/hr. Resulting in the estimated costs for MedImageInsight in the table above.

GPU compute was roughly 7× faster at about 0.37× the total cost when endpoint costs are included. This was a key learning and clearly indicates the benefits of the more powerful compute resources.

MedImageInsight can be deployed in two ways, depending on dataset size and operational needs.
For smaller or infrequently processed datasets, we deploy MedImageInsight as a managed Azure ML online endpoint and invoke it from the pipeline. This keeps the pipeline simpler and avoids managing the MedImageInsight compute directly, while offering comparable performance at modest scale.

For larger batch workloads, an alternative approach is to load MedImageInsight directly on the Azure ML pipeline’s GPU-backed compute. In this model, the pipeline handles both model loading and classification, eliminating per-request network round trips and the fixed cost of hosting a persistent endpoint.

While this approach requires slightly longer pipeline run time, it becomes more cost‑effective at scale by avoiding endpoint overhead and improving throughput during bulk processing.

Possible future enhancements

Additional modalities: Extending the pipeline and classification to CT, X-ray, and ultrasound imaging, and build on the pattern for pathology images
Image embeddings fusion: Combining MedImageInsight image embeddings with text embeddings for hybrid search
Condition-aware search: Enabling queries about findings and conditions, not just imaging parameters

The gap between a hackathon demo and a production system is where the real engineering happens. We hope sharing our journey helps others building similar systems.

If you’re interested in partnering with us to work toward this goal or need access to the GitHub repo with the pipeline and UI code, contact authors through your Microsoft account team or reach out to Microsoft HLS AI frontier team

The healthcare AI models in Microsoft Foundry are intended for research and model development exploration. The models are not designed or intended to be deployed in clinical settings as-is nor for use in the diagnosis or treatment of any health or medical condition, and the individual models' performances for such purposes have not been established. You bear sole responsibility and liability for any use of the healthcare AI models, including verification of outputs and incorporation into any product or service intended for a medical purpose or to inform clinical decision-making, compliance with applicable healthcare laws and regulations, and obtaining any necessary clearances or approvals.

Capturing clinical conversations across dozens of languages with Dragon Copilot

Karen_Couchon — Tue, 26 May 2026 12:58:47 GMT

Language can shape trust, understanding, and the patient experience

Patient care doesn’t happen in just one language. To better reflect the diversity of patients and communities physicians serve, Dragon Copilot now supports dozens of languages, enabling clinicians to capture clinical conversations naturally without changing how they document care. This expanded capability helps physicians engage more confidently with diverse patient populations, transition smoothly from one appointment to the next, and spend more time focused on care rather than documentation.

Care that reflects the languages patients speak

In real world care settings, patient visits may occur across multiple languages throughout the day. Physicians can welcome each patient and carry on a natural, back-and-forth conversation in the language patients are comfortable speaking, without interrupting the flow of the appointment. When the visit concludes, Dragon Copilot seamlessly generates the clinical note in English for clinician review and signoff.

Recording languages available in Dragon Copilot

With a broad range of recording languages available, Dragon Copilot reflects the diversity of regions, communities, and patient populations clinicians serve every day. This language coverage helps clinicians engage patients more naturally and inclusively, reducing communication barriers that can affect understanding and trust. By conversing with patients in the language they speak, clinicians can deliver more effective care and reach more patients.

Physicians can record conversations in Dragon Copilot in the following languages: Afrikaans, Arabic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian, Catalan, Chinese (Mandarin), Chinese (Cantonese), Croatian, Czech, Danish, Dutch, English, Estonian, Filipino (Tagalog), Finnish, French, Galician, German, Greek, Hebrew, Hindi, Hungarian, Icelandic, Indonesian, Italian, Japanese, Kannada, Kazakh, Korean, Latvian, Lithuanian, Macedonian, Malay, Marathi, Nepali, Norwegian, Persian, Polish, Portuguese, Punjabi, Romanian, Russian, Serbian, Slovak, Slovenian, Spanish, Swahili, Swedish, Tamil, Thai, Turkish, Ukrainian, Urdu, Vietnamese, and Welsh.

Building multilingual conversations into Dragon Copilot

Dragon Copilot’s multilingual ambient recording capability supports natural, real-world clinical conversations in multiple languages while delivering structured clinical documentation in English. It captures ambient speech during patient encounters, applies language identification, speech recognition, and clinical understanding, and transforms free-flowing dialogue into structured clinical notes. This allows clinicians to communicate in a patient’s preferred language while receiving an English summary aligned to documentation standards, helping reduce administrative burden and support more inclusive interactions.

Testing multilingual conversations with safety in mind

To evaluate multilingual ambient documentation, we started with clinical encounters in English. These served as our baseline so we could understand how performance changes when the same conversation is processed in different languages.

We transcribed each encounter into a target language, generated spoken audio, and ran it through Dragon Copilot’s multilingual workflow. The system then produced an English clinical note, which we compared to the original English reference. This allowed us to measure how well clinical meaning is preserved across languages, not just transcription accuracy, but the quality of the final clinical summary.

Conclusion: Designed for real world care

Multilingual recording support helps clinicians document conversations more naturally while maintaining consistent workflows and documentation standards. As Dragon Copilot continues to evolve, multilingual support remains focused on clarity, safety, and usability, so clinicians can spend less time documenting and more time with patients.

While Dragon Copilot captures conversations in many languages, it does not provide translation or interpretation. Clinicians should use the product only in languages they are comfortable practicing in and remain responsible for reviewing and validating all clinical content.

The Agent Era Has Already Arrived in Healthcare. Are You Ready to Govern It?

DolicaGopisetty — Tue, 05 May 2026 01:28:17 GMT

Start here. Answer honestly.

Right now, how many AI agents are running inside your organization? Who built them? Which patient data, claims information, or proprietary research are they configured to access?

If your CISO walked into your office tomorrow and asked for a complete inventory of every agent in your enterprise, including each one's owner, the systems it is permitted to access, and the policies that govern how it operates, could you produce that inventory before lunch?

When the analyst who built that clinical summarization agent moves to a new role next quarter, what happens to the agent? Does its access continue? Does anyone notice?

If a regulator opened an audit tomorrow, could you prove that every AI agent operating in your environment is subject to the same lifecycle controls, identity standards, and data protection policies you apply to your human workforce?

Could you disable a compromised agent enterprise-wide with a single click, the same way you would revoke a lost access credential?

If those questions made you hesitate, you are not alone. Almost no healthcare or life sciences organization can answer them confidently today. And that gap is exactly where the next decade of risk, and the next decade of competitive advantage, will be decided.

The quiet crisis nobody talks about yet

Healthcare and life sciences leaders are caught in a paradox.

You need AI to survive the operational pressures squeezing your organization from every direction. Physician burnout is at crisis levels, with 45.2% of US physicians reporting symptoms in recent Mayo Clinic research. Revenue cycle complexity continues to climb, and McKinsey now estimates that the cost to collect consumes 30 to 60 percent of net patient revenue at many provider organizations. Prior authorization backlogs delay care. Clinical trial timelines stretch into years. Documentation burden eats hours that belong to patients.

So you started piloting Microsoft 365 Copilot. You experimented with agents in Copilot Studio. Maybe a clinical team built an agent to draft discharge summaries. A revenue cycle group spun up an agent to triage denials. A medical affairs team built one to comb through literature. Each one delivered value. Each one was approved on its own merits. And then a quiet thing happened.

You lost track of how many agents you have.

According to KPMG's AI Quarterly Pulse Survey, 88 percent of organizations are now exploring or piloting AI agents. IDC projects that 1.3 billion agents will be in operation by 2028. Inside your own walls, the number is climbing fast. Each new agent is a digital identity that authenticates into your environment, accesses your data, and executes work on behalf of your business. Most have no formal owner. Most have no documented access scope. Most have no decommissioning plan. Most have never been reviewed by Compliance.

Microsoft's 2024 Data Security Index found that 84 percent of organizations lack confidence in their AI data security posture, and 40 percent have already experienced an AI related data security incident. That is not a future problem. That is a now problem.

If shadow IT was the defining governance challenge of the last decade, agent sprawl is the defining challenge of this one. And in healthcare and life sciences, where ePHI, member PII, and proprietary clinical trial data are at stake, the consequences are not theoretical. They are existential.

The reframe that changes everything

Here is the counterintuitive truth that separates HLS organizations that scale AI from those stuck in pilot purgatory.

Governance is not the brake on AI adoption. Governance is the accelerator.

When security, identity, and agent oversight are engineered in from day one, your teams stop tiptoeing. They build with confidence because the guardrails are real. They expand into clinical use cases because Compliance trusts the foundation. They scale wall-to-wall because IT can prove every agent is accounted for. The organizations that lead with trust end up moving faster in the long run, not slower.

This is the bet behind Microsoft Agent 365 and Microsoft 365 E7.

What Agent 365 and Microsoft 365 E7 actually are

Microsoft 365 E7, announced March 6, 2026 and now generally available, is the Frontier Suite. It is Microsoft's answer to a single question that every healthcare CIO, CISO, and COO is wrestling with: how do you run AI safely, at scale, across an entire organization?

E7 is not another SKU on top of your existing stack. It is one cohesive platform that brings together four essential capabilities:

Microsoft 365 E5 for your enterprise productivity, collaboration, and security foundation, including Microsoft Defender, Microsoft Purview, and Microsoft Intune.
Microsoft 365 Copilot for AI grounded in your organizational data through Work IQ, embedded in the flow of work for clinicians, researchers, operations teams, and administrators.
Microsoft Entra Suite for identity governance, Conditional Access, and Zero Trust network access, extended consistently across users, applications, and AI agents.
Microsoft Agent 365 as the centralized control plane to observe, govern, and secure every AI agent, whether built by Microsoft, your internal teams, or external partners.

Agent 365 is also available as a standalone capability. But the magic happens when it works alongside the rest of E7, because that is where AI, identity, security, and governance stop being separate disciplines and become one operating system for the agentic era.

The mental model that unlocks everything: agents are first-class digital identities

Here is the simplest way to understand what Agent 365 does.

Microsoft 365 governs your enterprise identities. Agent 365 governs your agent identities. The same control plane disciplines apply to both.

Think about the rigor you apply to any privileged identity in your environment, whether a service account, an API integration, or a third-party application connector. You issue it a unique identity in Microsoft Entra. You assign a human owner who is accountable. You scope its access to least privilege. You apply DLP, sensitivity labels, and Conditional Access. You monitor for anomalous behavior. You have a documented decommissioning path. Identities that no one watches over become identities that get exploited.

Now ask yourself how the last AI agent in your environment was created.

The honest answer at most organizations: someone opened Copilot Studio, pointed it at a SharePoint library of clinical protocols, gave it a name, and moved on. No documented owner. No access review. No retirement plan. Compliance was never consulted.

You would never stand up a privileged service account that way. Yet that is exactly how most organizations are standing up the fastest-growing class of digital identities in their environment.

Agent 365 closes that gap by extending the identity, security, and lifecycle controls you already trust for users and applications so they apply with the same rigor to AI agents.

Every agent receives a unique Entra Agent ID, a first-class identity in Azure AD with the same governance primitives as any other privileged identity.
Every agent has a designated human owner who is accountable for its scope and behavior.
Access is granted explicitly through Conditional Access and policy templates, so each agent operates only against the resources its purpose requires.
Microsoft Purview DLP and sensitivity labels govern which data the agent is permitted to read, generate, or share.
Microsoft Defender monitors agent activity for anomalies and surfaces alerts the same way it does for any other identity-driven risk.
Lifecycle rules flag or auto-retire agents that are dormant, orphaned, or risky, eliminating the unowned automations that quietly accumulate in every enterprise.

This is not metaphor. It is the actual architecture. The fastest path to governing agents is to extend the identity infrastructure you already trust.

The three pillars of Agent 365: Observe, Govern, Secure

Pillar 1: Observe. Know what is actually happening.

You cannot govern what you cannot see. The first job of Agent 365 is to give you complete, continuous visibility into every AI agent operating in your environment.

The Agent Registry is the single authoritative inventory of every agent, whether built by Microsoft, custom developed by your team, deployed by a partner, or discovered as a shadow agent operating without oversight. Each entry shows the owner, purpose, capabilities, lifecycle status, and business context. Agent Analytics tracks adoption, quality, performance, and business impact. Agent Map visualizes how agents connect with other agents, people, tools, and data sources, surfacing dependencies and risk concentrations you would never spot in a spreadsheet. Real time monitoring flows directly into Microsoft Defender, so unusual agent behavior generates alerts the same way unusual user behavior does today.

For a health system CISO, that means finally being able to answer the question: which agents are touching ePHI, and is every one of them authorized? For a life sciences compliance officer, it means audit ready visibility into every AI system operating across R&D, regulatory affairs, and commercial. For a payer operations leader, it means knowing which claims processing agents are actually delivering accuracy and throughput, and which are quietly underperforming.

Pillar 2: Govern. Set the rules. Control the lifecycle.

Visibility is the start. Control is what turns visibility into outcomes.

Agent 365 ensures that every agent is approved, compliant, and accountable from creation through retirement. IT led onboarding workflows make sure each agent launches with the right identity, access, and ownership before it ever touches data. Policy templates enforce data handling, permission, and usage rules consistently from day one through Defender, Entra, and Purview. Rules based agent management gives admins an automated If This Then That interface. If an agent is unused for 90 days, auto retire it. If an agent is flagged as risky, block it and alert the security operations team. No human in the loop required for the routine cases, full alerting and override for the exceptions.

Ownership enforcement requires every agent to have a designated human owner. When that owner leaves the organization, the platform flags the orphaned agent for bulk reassignment, so nothing operates without clear accountability. The Tools Gateway brokers and audits tool access for agents, enabling least privilege at the action level, not just the identity level.

For HLS specifically, that translates to outcomes you can take to your board. A hospital CIO can ensure any agent touching Epic or Cerner goes through standardized approval. A pharma IT director can enforce that clinical trial matching agents only touch de identified data unless elevated permissions are explicitly granted and documented. A payer compliance team can automatically retire agents tied to a completed open enrollment campaign instead of letting them silently expand the attack surface.

Pillar 3: Secure. Protect agents and data with the stack you already trust.

The final pillar is what makes Agent 365 production grade for healthcare and life sciences. Security and compliance are not bolted on. They are the same proven Microsoft security stack you already run for your users, extended natively to agents.

Microsoft Purview, your data security and compliance backbone:

Data Security Posture Management for AI gives visibility into how agents interact with sensitive data and detects risky usage patterns.
Data Loss Prevention stops agents from accessing or processing files labeled Highly Confidential, even when a user prompts them to.
Sensitivity labels are inherited automatically by agent outputs, governing how data is viewed, extracted, or shared downstream.
Insider Risk Management detects risky behavior by users interacting with agents, such as unusual prompt patterns or excessive access to sensitive data.
Communication Compliance monitors AI driven interactions for regulatory or ethical violations and unauthorized disclosures.
eDiscovery and Audit logs every agent interaction, giving legal, compliance, and IT teams the transparency required for HIPAA, GDPR, and FDA 21 CFR Part 11.
Oversharing Assessments run weekly checks for sensitive data exposure across SharePoint sites and agent access patterns.

Microsoft Entra, your identity control plane:

Entra Agent ID gives every agent a unique identity in Azure AD, so Conditional Access, role based access, and risk based policies apply individually.
Conditional Access for agents enforces policies like only allow this prior authorization agent to access claims data from approved devices and locations during business hours.
Identity Governance provides access packages for agents with reduced scope permissions and least privilege defaults.
Block at Scale lets you instantly disable all high-risk agents from Entra in a single action.

Microsoft Defender, your threat protection layer:

Security Posture Management identifies and remediates agent misconfigurations, such as agents running with no authentication.
Threat Detection and Blocking monitors suspicious agent activity, generates alerts, and blocks unauthorized tool invocations.
Threat Investigation and Hunting collects unified agent observability logs so SOC teams can forensically trace every action an agent took.
One Click Kill Switch instantly disables any agent and surfaces the complete audit trail of every action it took before being stopped.

For a hospital security operations team, that means the same DLP policies protecting patient records in email and Teams now protect agents that summarize clinical notes. For a life sciences data protection officer, it means agents accessing proprietary compound data respect the same sensitivity labels as human researchers. For a payer CISO, it means an anomalous claims agent can be killed in seconds, with a complete forensic record of every member record it touched.

Why this only works as an integrated platform

Individual capabilities are useful. Integration is what makes them transformative. Here is the contrast HLS leaders feel today versus what changes the moment E7 lights up.

Without an integrated platform, you operate with:

Fragmented tools for identity, security, compliance, and AI, each with its own console and its own gaps.
No centralized agent inventory, forcing your IT and security teams to track bots and automations in spreadsheets.
Inconsistent policy enforcement across agents, creating compliance gaps every audit team will eventually find.
Blind spots where agents access data, invoke tools, or interact with other agents without any oversight.
Manual triage when an incident hits, because nothing connects user identity, agent identity, and data classification in one view.

With Microsoft 365 E7, you gain:

A Unified Agent Registry providing a single source of truth for every agent, whether Microsoft built, custom developed, partner deployed, or shadow discovered.
Entra Agent ID giving each agent a unique identity, so Conditional Access, role based access, and risk based policies apply at the individual agent level.
Full lifecycle governance with standardized onboarding, periodic review, ownership transfers, auto retirement of dormant agents, and structured offboarding.
Policy by design, where Purview DLP, sensitivity labels, and compliance rules extend to all agent interactions through pre built templates applied consistently from day one.
One click disable to instantly freeze any agent, with Defender threat detection extended to agents and full audit trails for forensic investigation.
Expanded threat coverage that addresses agent sprawl, overprivileged access, tool misuse, misconfiguration, and inter agent risk patterns no legacy tool was designed to see.
Shared registry and controls that let IT, Security, and Compliance reference the same authoritative inventory across Defender, Entra, and Purview, eliminating the silos that slow incident response.

This is the reason E7 exists as a platform, not a bundle. AI, identity, security, and governance stop being separate disciplines and start operating as one system.

What this is actually worth: the Forrester numbers

Microsoft commissioned Forrester to conduct a Total Economic Impact study of Microsoft 365 Copilot, published in March 2025. The composite organization in that study, modeled on real customer interviews, achieved:

132 percent three-year ROI with payback in under one year.
9 hours saved per Copilot user per month through automation of routine work like drafting, summarizing, and analysis.
Up to 2.6 percent top line revenue lift through better qualified opportunities, improved win rates, and stronger retention in customer facing teams.
25 percent acceleration in new employee onboarding as new hires ramp faster on summarized institutional knowledge.

Those are the verified numbers. The bigger story for HLS is what they look like when applied to clinical, claims, and research workflows where every reclaimed hour is an hour that goes back to patients, members, or science.

AI is already defending AI

The same agentic capabilities transforming clinical and operational workflows are now embedded in your security stack. Microsoft Security Copilot agents work alongside human analysts inside Defender, Entra, Purview, and Intune, accelerating threat response and absorbing the manual load that today drowns most security operations teams.

Independent benchmarks back the impact. In a 162 admin randomized study published in 2025, the Conditional Access Optimization Agent in Microsoft Entra completed configuration tasks 43 percent faster and produced 48 percent more accurate Conditional Access policies than admins working without it. Security triage, alert investigation, and identity hygiene are following the same trajectory.

For HLS security teams already stretched thin, that is hours reclaimed every week to focus on the threats that actually matter, with the same Agent 365 governance applying to the security agents themselves. The defenders are governed by the same rules as the workforce they defend.

How HLS organizations are putting Agent 365 to work

Here is how the value shows up across the three biggest HLS segments.

For providers: reclaiming time for care

The challenge: clinicians spend more time on documentation than on patients. Care coordination is fragmented. Burnout is gutting retention.

The strategy: deploy agents that absorb administrative load while Agent 365 ensures every one of them respects ePHI boundaries.

Clinical documentation agents integrated with Microsoft Dragon Copilot structure dictation against EHR requirements, apply billing codes, and flag missing elements before submission.
Care coordination agents generate care plans, allocate tasks, and surface relevant patient context during multidisciplinary rounds, optimized for HL7 FHIR interoperability.
Patient intake and scheduling agents built in Copilot Studio handle appointment booking, reminders, eligibility verification, and referral management.
Handoff and shift summary agents pull from multiple systems to generate complete handoff summaries for nurses and physicians transitioning between shifts, reducing communication gaps that drive adverse events.

The aha moment: applied across a 10,000 employee health system, nine hours per user per month is more than one million reclaimed hours a year. That is the equivalent of hundreds of full time clinicians, returned to direct patient care, with every agent governed under the same Conditional Access and DLP policies your IT team already manages today.

For payers: transforming revenue cycle and member experience

The challenge: prior auth backlogs delay care. Denial rates climb. Member services teams drown in volume.

The strategy: agentic AI rewires the most expensive, most manual workflows in your operation while Agent 365 keeps every agent inside the lines on member PII.

Prior authorization agents autonomously gather clinical documentation, cross reference medical policy, determine approval criteria, and route decisions, accelerating turnaround from days to hours.
Claims processing agents automate billing and denial management. With cost to collect running 30 to 60 percent of net patient revenue at many organizations, even modest automation produces material margin recovery.
Denial resolution and appeals agents analyze denial patterns, surface root causes, generate appeal documentation, and track success rates over time, turning a cost center into a continuous improvement engine.
Member services agents integrated with Microsoft 365 Copilot Chat handle benefits inquiries, claims status, and self service triage, deflecting call volume and improving first contact resolution.
Fraud detection and risk adjustment agents scan claims data for anomalies and optimize coding accuracy for Medicare Advantage and ACA populations.

The aha moment: a payer CISO can disable an anomalous prior auth agent in one click and produce a complete forensic record of every member record it accessed, while Compliance simultaneously confirms the agent never violated DLP. That is regulatory readiness that legacy automation cannot deliver.

For life sciences and pharma: accelerating discovery and commercialization

The challenge: clinical trials take years. Regulatory submissions consume teams. Medical affairs cannot keep up with literature volume.

The strategy: orchestrate agents across R&D, regulatory, medical, and commercial, with Agent 365 enforcing the data classification rules that proprietary IP and clinical data demand.

Clinical trial matching agents scan patient profiles and eligibility criteria to surface trial opportunities, accelerating recruitment.
Regulatory document preparation agents assemble submissions, cross reference data across modules, and ensure consistency in FDA, EMA, and global filings.
Medical research and literature review agents powered by Microsoft GraphRAG retrieve research backed insights with verified source references, giving medical science liaisons trustworthy synthesis on demand.
Pharmacovigilance agents monitor safety databases, flag potential adverse events, and generate timely case reports.
Commercial insights and launch planning agents synthesize market data, payer policy, and HCP sentiment for sharper launch and field strategy.

The aha moment: cutting even three months off a regulatory cycle on a single high revenue product can mean tens of millions in additional sales, while Purview sensitivity labels guarantee every agent accessing proprietary compound data respects the same data classification as your senior researchers.

A phased path that actually works in regulated industries

In regulated industries, a big bang AI rollout is a recipe for incidents. The HLS organizations getting this right are following a five-phase pattern that builds expertise and validates governance before scale.

Establish. Form a cross-functional champion team across IT, Compliance, Clinical Operations, and Research. Define what risks you are mitigating and what outcomes you are unlocking. Inventory the agents already in flight.
Configure. Stand up identity, DLP, and policy templates in Microsoft 365 Admin Center, Power Platform Admin Center, and Microsoft Purview. Enforce that any agent handling PHI runs in a secure environment with audit logging on by default.
Pilot. Choose a small group of makers in a controlled environment. Start with non-critical workflows like internal reporting or scheduling before moving to clinical or member facing use cases. Run weekly reviews with Compliance and Security.
Empower. Launch role specific training for clinicians, researchers, makers, and IT. Stand up a Center of Excellence to provide templates, best practices, and reusable patterns. Promote success stories internally to build momentum.
Scale. Expand agent development across departments with governance as a guardrail, not a gate. Use pay as you go metering to track usage and optimize licensing. Refine policies continuously based on Purview signals and audit results.

The strategic insight: organizations that lead with governance reach scale faster than those that lead with experimentation. Trust is the unlock, not the obstacle.

Governance is a team sport

Here is the pattern we see again and again. The HLS organizations that succeed with AI at scale are not the ones with the smartest IT shop or the boldest Compliance officer. They are the ones whose IT, Security, Compliance, Clinical, Research, and Operations leaders sit at the same table on agent strategy from week one.

Agent 365 was designed for that table. The Agent Registry is the shared truth. Purview policies satisfy your Compliance officer. Entra controls reassure your CISO. The lifecycle workflows give your CIO confidence. The clinical and research outcomes give your COO and Chief Medical Officer the business case. Everyone gets the view they need from the same single source.

Stand up an agent governance council. Meet every two weeks. Use the Agent Registry as your standing agenda. Make decisions in plain sight. The organizations that do this consistently outperform on both speed and safety. The ones that try to keep AI inside a single function fall behind on both.

Who contributes what

Think back to the mental model. You would never let a single function authorize, configure, and oversee a new privileged system on its own, not when it touches ePHI, claims, or proprietary research. Security, IT, Compliance, Clinical, and the relevant business owner all weigh in because the stakes are too high for any one seat to carry alone. Agent governance demands the same multidisciplinary scrutiny, and the council is where that happens.

Each seat brings something the others cannot.

CIO. Owns the agent strategy and the platform investment. Translates board-level AI ambition into an operating model the rest of the organization can execute against.
CISO and Security Operations. Define agent identity standards, Conditional Access policies, and incident response playbooks. Without this seat, an anomalous agent touching ePHI becomes a breach instead of a contained event.
Chief Compliance Officer and Privacy. Translate HIPAA, GDPR, FDA 21 CFR Part 11, and state regulations into Purview policies and audit requirements. This is the seat that keeps you out of an OCR investigation or a 483 letter.
Chief Medical Officer and Clinical Operations. Validate that clinical agents are safe, accurate, and aligned with care standards. Own the clinical risk review for any agent that touches patient care, the same way you would for a new clinical protocol.
Chief Research Officer or Head of R&D. Govern how agents interact with proprietary trial data, compound libraries, and scientific IP. The seat that protects the next decade of pipeline value.
COO and Revenue Cycle Leadership. Prioritize the operational workflows where agents will move the needle on cost to collect, denial rates, and throughput, and own the business outcomes that justify the investment.
Center of Excellence Lead. Maintains templates, reusable patterns, and maker enablement. Turns every council decision into a guardrail builders can actually use the next morning.
Frontline champions. Clinicians, claims specialists, and researchers who pilot, give feedback, and carry credibility back to their peers. The seat that decides whether agents get adopted or quietly ignored.

When every one of these voices is in the room, your governance council operates like a tumor board for AI. Different lenses, one shared decision, full accountability. That is how regulated industries make complex calls safely, and it is exactly the muscle Agent 365 was built to support.

Seven questions to bring to your next leadership meeting

If you want to know whether your organization is ready, run through these together. The places you hesitate are exactly where Agent 365 and E7 deliver the most value.

Visibility. Do you know which AI agents, bots, and automations are running in your environment today, who built them, what they have access to, and whether they are still needed?
Control. If someone on your team builds a new AI agent tomorrow, what is the actual process to make sure it is approved and secured? Or could they deploy it with wide open access?
Security. What prevents an AI agent from reading or transmitting patient data it should not? Do you have a way to detect and stop a rogue or compromised agent?
Accountability. Who owns the outputs of an AI agent's actions? What is the offboarding process when the agent or its creator leaves?
Scale. Six months from now, you may have a hundred agents deployed across departments. Are your oversight and compliance structures ready for that volume?
Cross-functional alignment. How are your IT, Security, and Compliance teams partnering on AI today? Governance is a team sport.
Data readiness. How confident are you that your data estate is clean, labeled, and governed well enough for AI to surface accurate answers and not outdated or conflicting information?

If you hesitated on even one of those, you have just identified where Agent 365 and Microsoft 365 E7 will pay for themselves the fastest.

The path forward

Here is the honest truth. The healthcare and life sciences organizations that lead in the next decade will not be the ones that adopted AI first. They will be the ones that adopted AI safely, compliantly, and at scale, with intelligence and trust woven into every layer.

Microsoft Agent 365 and Microsoft 365 E7 give you the only integrated platform that brings AI, identity, security, and governance into one cohesive system, running in the flow of work you already use. This is not about adding another tool to your stack. It is about extending the investments you have already made in Microsoft 365, Entra, Defender, and Purview to cover the fastest-growing class of digital identities in your environment.

The agent era has already arrived. The question is whether you will govern it with confidence or chase it with anxiety.

We would love to help you lead.

Take the next step

Explore Microsoft Agent 365: The Control Plane for Agents
Microsoft Entra Agent ID: aka.ms/EntraAgentID
Learn more about Microsoft 365 E7, the Frontier Suite: Introducing Microsoft 365 E7
See Microsoft 365 Copilot in action: Microsoft 365 Copilot
Read the Forrester TEI study: The Total Economic Impact of Microsoft 365 Copilot

The Microsoft 365 Copilot Frontier Program: What Executives and IT Leaders Actually Need to Know

michaelgoad — Wed, 29 Apr 2026 20:36:25 GMT

If your business leaders are asking why they don't have the latest Copilot features they saw at Microsoft Ignite, someone has probably already said, "Have you looked at the Frontier program?"

That's where things get interesting.

Frontier is one of those programs that can be a real strategic advantage when you understand it well. But if you walk into it without the right governance in place, you're going to create headaches for your security and compliance teams fast.

Here's an honest, practical breakdown of what Frontier is, why organizations choose it, when you should wait, and what your governance teams need to know before you flip the switch.

What Is the Microsoft 365 Copilot Frontier Program?

Frontier is Microsoft's early access program that gives organizations access to the newest AI-powered Copilot capabilities before they reach general availability (GA).

Think of it as an on-ramp to the leading edge of Microsoft's AI roadmap.

When Microsoft's engineering teams build a new Copilot feature, it moves through a lifecycle:

Private Preview — Invitation only. A small group of design partners tests the feature under close partnership. Not self-service, not for everyone.
Frontier — Broader early access. Thousands of tenants can participate by opting in through the admin center. Still pre-GA, but real and working in your production environment.
General Availability (GA) — Full Microsoft SLA coverage, support agreements, and regulatory compliance standards your organization depends on.

Here's the part worth repeating: Features in Frontier are real working capabilities inside your production Microsoft 365 environment, but they are in active development. Microsoft is still refining them based on customer feedback.

A few things to understand right away:

Frontier is opt-in. An M365 administrator has to enable it. It doesn't happen automatically.
When you enable Frontier, you're not getting a separate test environment. These features run inside your existing tenant alongside your GA Copilot capabilities.
The feature set changes. Features get added as they mature and graduate out of Frontier into GA over time.

Why Organizations Choose to Enable Frontier

There are four strategic reasons I see most often.

Competitive velocity. In industries like financial services, healthcare, and professional services, staying ahead matters. Frontier lets your team start learning and building workflows around capabilities before your competition even knows they're coming. By the time a feature hits GA, your users are already fluent in it.
Direct influence on the product. This one is underappreciated. Microsoft actively collects feedback from Frontier participants. When your users encounter something that doesn't work the way your workflows require, that feedback goes directly to the engineering team. Your organization gets a seat at the table in shaping how these features evolve.
Organizational AI readiness. Participating in Frontier responsibly forces a healthy discipline. You need to mature your AI governance, adoption playbooks, and change management approach faster than you otherwise would. Many IT leaders I've talked to say that preparing for Frontier accelerated their overall Copilot adoption maturity because it forced them to get governance and IT strategy in place first.
Access to differentiated capabilities. Some features that debut in Frontier are genuinely transformational. Copilot Cowork. New reasoning models. Deeper cross-application intelligence. If those capabilities tie directly to your business outcomes, waiting for GA means leaving real value on the table.

One more thing worth saying directly: the choice to enable Frontier is a leadership decision, not an IT decision. It's about balancing how fast you want to move with how mature your governance actually is. This is a joint venture between IT and the business.

Frontier vs. Private Preview vs. GA: Feature Lifecycle Explained

Here's a quick reference so you and your leadership team are using the right language:

Stage	Access	SLA	How to Join
Private Preview	Invitation only	None	Microsoft selects you
Frontier	Opt-in via admin center	Preview feature expectations (not GA SLA)	Self-service
General Availability	All licensed users	Full Microsoft SLA	Automatic

The key callout: Frontier features do not carry the SLA commitments that apply to GA services. That matters a lot in regulated environments.

When Frontier Makes Sense for Your Organization

Frontier is a strong fit if:

Leadership actively values being first to adopt, with the discipline to do it responsibly
You already have a mature M365 Copilot deployment and power users who are hungry for more
You have clear IT governance and change management processes in place
Your compliance posture allows for preview feature participation

When You Should Wait

I want to be equally honest about when Frontier is not the right move yet.

Wait if you're in a heavily regulated environment and haven't completed a compliance assessment. Preview features may not have completed all compliance certifications. Talk to your Microsoft account team before you enable anything.

Wait if your M365 baseline deployment is still maturing. Get the foundations right first.

Wait if you don't have a clear feedback path from end users. Without a channel for users to report back to IT and business leaders, Frontier creates frustration instead of value.

Wait if your IT team is already stretched. Frontier requires active engagement with release notes, user communication, and feedback loops. If capacity is already thin, this will add to the bottleneck.

With the right framing, Frontier isn't a risk. It's a governance responsibility.

Five Governance Checkpoints Before You Enable Frontier

This section will save your compliance team the most headaches. Work through all five before you flip the switch.

Conduct a compliance assessment. Preview features may not have completed all compliance certifications. Work with your Microsoft account team to understand the compliance posture of specific Frontier features relevant to your industry.
Define your governance scope. Frontier doesn't have to be all-or-nothing. You can enable it for a defined set of users using Microsoft 365 security groups while keeping the broader organization on GA capabilities. More on that in the admin center walkthrough below.
Establish user communication protocols. Features can change quickly. Your users need to know what they're participating in, why their experience may differ from others, and how to submit feedback. ("Why does my UI look different than yours?" is a real conversation that happens all the time.)
Set up a feedback and monitoring cadence. Review Frontier release notes regularly. Track what's live in your tenant and synthesize user feedback back to Microsoft.
Plan for feature lifecycle transitions. Features can be updated, temporarily suspended, or graduated to GA. Your governance plan should address how you'll communicate changes and adjust workflows when that happens.

Think of governance here as a maturity accelerator, not a barrier.

How to Enable Frontier in the Microsoft 365 Admin Center

Here's exactly where to go and what to do.

Step 1: Navigate to Copilot Settings

Go to the Microsoft 365 admin center, navigate to the Copilot section, and select Settings. Click "View all" to see all settings on a single unified page. Use Ctrl+F to search for "Copilot Frontier."

Step 2: Scope Your Access

You'll have three options: enable for no one, for everyone, or for specific users. For most organizations, specific users is the right call. Set up a dedicated security group for your Frontier champions and assign access to that group only.

Step 3: Assign Frontier Agents

Enabling Frontier at the tenant level is just step one. You also need to assign specific Frontier agents to users. In the admin center, go to Agents > All Agents and search for "Frontier." From there, you can select individual agents (like Copilot Cowork) and assign them to your champion group or a subset of it.

This is the most common point of confusion: you can have Frontier enabled but still not have access to a specific agent like Cowork because you never assigned it. Both steps are required.

Step 4: Pull a Baseline Usage Report

Before your pilot starts, capture a baseline snapshot of your current Copilot usage. In the admin center, go to Reports > Usage > Microsoft 365 and look at the Copilot, Copilot Chat, and Agents tabs. Screenshot or export these. In four to eight weeks, you'll use this baseline to measure the impact of Frontier adoption across your pilot cohort and overall.

A Phased Rollout Model That Actually Works

Don't just turn Frontier on and hope for the best. Here's a five-phase model that turns it into a structured capability program.

Phase 1: Identify your Frontier champions. Target 50 to 200 users who are already Copilot power users, have a growth mindset toward AI, and can articulate business value from feature changes. These are your early adopters who will carry the signal back to the rest of the org.

Phase 2: Enable Frontier for your champion cohort. Follow the admin center steps above. Brief that group on what to expect, what's different, and how to submit feedback.

Phase 3: Evaluate and document. After four to eight weeks, pull up your Copilot usage dashboard and compare it to your baseline. Which features are driving measurable productivity gains? Document your findings.

Phase 4: Expand or adjust scope. Based on your champion cohort data, either expand to a broader user population or adjust scope if a specific feature is causing friction.

Phase 5: Establish steady-state governance. Formalize the feedback loop and user communication as a standard operating procedure within your Copilot governance framework. Start building documentation now so you're ready when features graduate to GA.

This approach turns Frontier from a feature toggle into a strategic capability program. That's where the real value shows up.

A Quick Decision Framework for Leadership

Before you bring this to your leadership team, run through these five questions:

Do we have a clear AI governance framework in place?
Are our Microsoft 365 GA deployments stable and delivering measurable value?
Have our compliance and legal teams assessed preview feature participation?
Do we have an identified Frontier champion cohort or IT bandwidth for a structured pilot?
Is there a specific business outcome we're trying to accelerate?

If you answered yes to four or five of those, you're in a strong position to move forward.

If you have two or more no's, invest in those foundations first. Getting governance, bandwidth, and a clear use case in place before enabling Frontier. It isn't about slowing down, it's setting yourself up to actually get value from it.

Bottom Line

The Microsoft 365 Copilot Frontier program is a strategic option for enterprise organizations that want to shape the future of AI productivity tools, not just consume them. But it's not for everyone, and it's not designed to be.

It's built for organizations that have the governance maturity, leadership alignment, and operational capacity to engage with early access AI responsibly.

When you do it right, Frontier can accelerate your AI program, sharpen your competitive edge, and give your organization a direct voice in how Microsoft AI evolves.

The tools are all right there in the admin center. It's just a matter of knowing where to look and using them intentionally.

Have questions about Frontier readiness or want to talk through your organization's Copilot governance strategy? Drop them in the comments or reach out directly.

Reimagining Cancer R&D with Agentic AI Using GigaTIME in Microsoft Discovery

Alberto_Santamaria — Tue, 28 Apr 2026 02:29:44 GMT

@Alberto Santamaria-Pang,
Principal AI Data Scientist, Industry Solutions Engineering Healthcare
Adjunct Faculty at Johns Hopkins School of Medicine
@Alexander Mehmet Ersoy,
Dir. Industry Advisory, Healthcare & Life Sciences

1. Introduction: From Images to insight in modern oncology

What if we could characterize every single cell in a tumor not just by how it looks under the microscope, but by the biological signals that shape how it behaves, how it evades the immune system, and how it responds to therapy? This question sits at the heart of modern oncology and precision medicine. Advances in artificial intelligence and spatial biology are rapidly lowering the barrier to understanding cancer at cellular and molecular resolution, supporting research into more precise, more personalized, and ultimately more effective treatments. Immuno-oncology already offers a glimpse of what becomes possible when therapy is guided by biology rather than averages. For example, the FDA approval of tisagenlecleucel for relapsed or refactory B-cell acute lymphoblastic leukemia was supported by an overall remission rate of 82.5%, underscoring how meaningful outcomes can be when treatment aligns with the right biological signals [1]. The challenge is scale: how do we make this type of biologically informed decision-making feasible across millions of patients, diverse tumor types, and real-world clinical settings?

Two recent Microsoft innovations help address that challenge, at different layers of the R&D stack: The GigaTIME AI Framework (a model and workflow for virtual mIF generation from routine pathology) and Microsoft Discovery platform (the agentic R&D platform that orchestrates data, tools, and AI Agents). In this time, we introduce GigaTIME in general (including a practical tutorial on how model can be used), and then show how GigaTIME could be used within, and in the context of, the Discovery platform as one tool that helps accelerate precision oncology discovery.

2. GigaTIME: Scaling tumor microenvironment insight from routine pathology

A routine hematoxylin and eosin (H&E) slide is a common cost-efficient diagnostic tool used to understand the specifics of patient’s oncological condition. It is like a high-resolution photograph of a complex cellular community. An H&E slide captures structure, morphology, and organization in remarkable detail, but it cannot fully reveal how cells are communicating or which molecular programs are active beneath the surface. This is why multiplex immunofluorescence (mIF) and related spatial proteomics assays have become so valuable in oncology research: they reveal protein patterns linked to immune identity, checkpoint signaling, proliferation, and tumor context. Their broad use, however, is limited by cost and throughput, which makes large-scale tumor immune microenvironment analysis difficult [2]. GigaTIME provides an important bridge. It translates routine H&E pathology slides into virtual mIF images across 21 protein channels, making it possible to infer spatially resolved, biologically meaningful virtual mIF patterns from a much more accessible input. In this blog, we focus on what that means at the tissue level: how to interpret selected virtual mIF signals, how to localize them in cellular context, and why that matters for understanding tumor–immune interactions in oncology [3].

Figure 1. GigaTIME Workflow schematic.

2.1 Reading virtual mIF signals in context

To make the virtual mIF panel easier to interpret, it helps to think of the tissue as two interacting compartments: the tumor compartment, where malignant growth and tumor-associated programs dominate, and the stroma or host compartment, where immune cells, vasculature, and connective tissue either resist, reshape, or sometimes enable tumor progression. The most important biology often happens at the boundary between these two worlds. Rather than reading the panel as a flat list of proteins, we can read it as a guide to tumor geography, immune access, checkpoint context, proliferation, and tissue infrastructure.

Table 1. Selected markers produced in GigaTIME.

Marker	What it represents	Why it matters biologically
CK	Tumor-rich epithelial regions	Defines where the tumor compartment is located.
DAPI	Cell nuclei	Anchors localization at the single-cell level.
CD8	Cytotoxic T cells	Helps assess whether immune cells are infiltrating tumor regions.
CD68	Macrophage-associated signal	Highlights myeloid context at tumor borders or within tumor-rich tissue.
PD-1 / PD-L1	Checkpoint-associated signaling	Provides context on whether immune activity may be locally restrained.
Ki67	Proliferation	Indicates whether tumor-rich regions are actively cycling.
CD34	Vasculature	Helps interpret access routes and stromal context around the tumor.

In this blog, we focus on a small set of markers that are especially useful for reading tumor geography, immune access, checkpoint biology, proliferation, and vascular organization. To make that concrete, we implemented a practical notebook that shows how the GigaTIME model can be deployed as an endpoint, used for inference on H&E patches, and combined with single-cell localization to support downstream phenotyping and interpretation. The main point is not any one marker in isolation, but how marker combinations organize in space and help us ask more meaningful questions about tumor–host interaction.

2.2 From H&E to virtual mIF: how GigaTIME works

The starting point is a sample-level H&E patch from the test dataset, paired with a compressed label file that contains binary marker masks and cell-segmentation scaffolds used downstream. The workflow is intentionally practical: load the H&E input, generate or reuse GigaTIME predictions, visualize selected virtual mIF channels, refine those predictions with single-cell localization, and summarize the results as virtual phenotypes and per-marker counts [4].

At the model-output stage, GigaTIME produces a multi-channel spatial prediction stack from the H&E patch. In the notebook, each channel can be visualized as a virtual mIF map indicating where the model predicts marker-associated signal in the tissue. However, these raw virtual mIF maps are not yet cell phenotypes. To make them biologically interpretable, the notebook converts dense predictions into cell-aware assignments. It uses labels_dapi for nuclear regions and labels_dapi_expanded for expanded cell regions, then computes the fraction of positive pixels within each segmented region. Marker positivity is assigned only when the overlap exceeds a threshold, with localization adjusted according to expected marker biology, such as nuclear versus non-nuclear signal [5].

This same localization scaffold also supports validation. Because the reference files provide binarized marker masks together with shared nuclei and expanded-cell labels, predicted signal and reference signal can be compared in the same segmented cellular space rather than only as unstructured image intensities. Once virtual mIF maps are tied to individual nuclei or cell regions, they become both quantitative and spatial, supporting measurements of infiltration, compartment-specific localization, and per-marker cell counts that can be aggregated across samples. You can access the tutorial here: https://aka.ms/gigatime-sample.

Figure 2. Example H&E patch and virtual mIF output.

2.3 Virtual Phenotyping

Once the virtual mIF maps have been localized to segmented cells, they can be interpreted as spatial phenotypes rather than diffuse prediction maps. In this tutorial, we use a limited sample dataset to demonstrate how these localized overlays can be reproduced and read biologically in practice. The goal is not to make broad claims from a small set of examples, but to show how virtual phenotyping connects marker prediction, cellular localization, and tumor microenvironment interpretation. In real applications, this type of workflow would typically require additional fine-tuning and validation to account for imaging conditions, tissue context, cohort composition, and study-specific marker panels.

At a high level, the figures in this section can be read through four themes: tumor–immune interaction, immune system structure, immune checkpoint biology, and stromal and vascular context. These themes translate localized virtual mIF signals into biologically meaningful spatial patterns. Rather than reading each marker in isolation, we can read how marker combinations organize near tumor-rich regions, immune niches, and tissue boundaries. These same concepts are already used in modern oncology, where immune infiltration, immune organization, checkpoint signaling, and vascular or stromal remodeling all shape how therapies are developed and interpreted [6–9].

Theme	Biological interpretation	Example marker trends	Therapy relevance
Tumor–immune interaction [6]	Tumor-rich compartment is being accessed by immune cells, shaped by myeloid cells, and actively proliferating.	• Higher CD8 near CK-rich regions suggests immune infiltration; • CD68 concentrated at the tumor border suggests a myeloid interface or barrier; • Higher Ki67 within CK-rich regions suggests active tumor proliferation.	Higher intratumoral CD8 is generally favorable for anti-tumor immunity; border-restricted CD68 may reflect a suppressive interface; high Ki67 in CK-rich regions is generally unfavorable because it suggests active tumor growth.
Immune system structure [7]	Immune compartment appears coordinated, sparse, balanced, or spatially segregated.	• CD3 and CD20 co-localized in organized clusters suggests structured lymphoid neighborhoods; • Balanced CD4 and CD8 distributions suggests a coordinated immune context; • Fragmented or separated patterns suggest a less organized response.	Organized lymphoid structure and balanced adaptive immune populations are generally favorable; fragmented or sparse immune organization may indicate weaker local immune coordination.
Immune checkpoint biology [8]	Immune cells are present but may be locally restrained by inhibitory signaling.	• CD8 overlapping with PD-L1 suggests immune presence in a potentially suppressive niche; • CD3 overlapping with PD-1 suggests T cells in a checkpoint-associated state consistent with local restraint.	Context-dependent: this may indicate a restrained immune response that could be relevant to checkpoint blockade, but not automatically a positive or negative finding in isolation.
Stromal and vascular context [9]	Tissue structure supports access, creates barriers, or concentrates inflammatory niches.	• CD34 aligned near CK-rich regions suggests vascular routes close to tumor compartments; • Tryptase and CD68 clustered in stromal or perivascular regions suggests innate inflammatory niches that may shape local signaling and access.	Context-dependent: vascular proximity can support access, while stromal or perivascular inflammatory niches may either facilitate response or reinforce barriers depending on the broader microenvironment.

Table 2. Quick guide to interpreting virtual phenotyping themes.

2.3.1 Tumor–immune interaction

Figure 3. Tumor–immune interaction.

We begin with a central question in the tumor microenvironment: can immune cells reach the tumor? In Figure 3, the CK-centered overlays provide a compact way to read this biology. CK + CD8 shows tumor-rich regions alongside cytotoxic T-cell signal, allowing us to ask whether immune cells are infiltrating tumor nests, remaining at the border, or being excluded from the tumor core. CK + CD68 adds macrophage context and helps highlight whether myeloid cells are embedded within tumor-rich regions or concentrated at the tumor–stroma interface. CK + Ki67 complements these immune overlays by showing whether the same tumor-rich regions also display strong proliferative activity.

Read together, these panels provide a concise illustrative summary of tumor geography, immune access, myeloid interface biology, and growth state. Are immune cells entering the malignant compartment, or is access limited? Are macrophages mixing with tumor cells or forming a border-associated niche? Are tumor-rich regions relatively quiescent, or are they actively cycling? Even in a tutorial setting, this combination of overlays shows how virtual markers can move beyond visualization and support structured interpretation of the tumor immune microenvironment.

2.3.2 Immune system structure

Figure 4. Immune system structure.

Virtual phenotyping is also useful for understanding how immune populations are organized beyond the tumor border itself. In Figure 4, overlays such as CD3 + CD20 and CD4 + CD8 provide a view into the composition and organization of the lymphoid compartment. Rather than asking only whether immune cells are present, these panels help us ask whether the immune landscape appears coordinated, sparse, balanced, or spatially segregated. This matters because immune presence alone does not fully capture immune effectiveness; spatial arrangement can suggest very different biological states.

2.3.3 Immune checkpoint biology

Figure 5. Immune checkpoint biology.

Checkpoint biology provides another layer of interpretation that is especially relevant in immuno-oncology. In Figure 5, overlays such as CD8 + PD-L1 and CD3 + PD-1 help connect immune presence with local regulatory signals. These panels are useful because they show that immune cells may be present in the tissue and still not be fully effective if their activity is being restrained by checkpoint-associated biology. Spatial overlap between T-cell markers and checkpoint-associated signal does not, by itself, prove immune exhaustion or therapeutic response, but it can provide context that is consistent with restrained or suppressed immune activity.

2.3.4 Stromal and vascular context

Figure 6. Stromal and vascular context.

The tumor microenvironment is also shaped by the surrounding tissue infrastructure. In Figure 6, overlays such as CD34 + CK and Tryptase + CD68 help reveal how vessels, stromal niches, and innate immune populations are positioned relative to tumor-rich regions. These patterns matter because immune access, tumor expansion, and local signaling are all influenced by the organization of the supporting tissue around the tumor. By including vascular and stromal context, the notebook helps show how virtual markers can support a more complete spatial interpretation of tumor–host interaction.

These examples show how virtual phenotyping transforms raw virtual mIF maps into interpretable spatial summaries of the tumor microenvironment. After localization, the outputs are no longer just probability maps; they become cell-aware patterns that can be read in terms of immune infiltration, tumor growth, checkpoint context, stromal organization, and compartment-specific localization.

The goal of examples is reproducibility and interpretation rather than broad biological generalization. The limited dataset is useful because it makes the workflow easy to follow and the figures easy to inspect, but real deployment would require additional tuning, validation, and adaptation for the target imaging workflow and marker set. Even with that caveat, this workflow illustrates the practical value of GigaTIME: virtual mIF predictions become most useful when they are localized, contextualized, and interpreted as part of a spatial system rather than as isolated channels.

3. Microsoft Discovery: Transform the end‑to‑end discovery process from hypothesis generation to simulation, evaluation, iteration, and design

Microsoft Discovery is designed as an enterprise agentic AI platform. It is built around a graph-based knowledge engine and teams of specialized AI agents that collaborate with scientists throughout the discovery cycle from literature reasoning and hypothesis formation to simulation and iterative learning. With Microsoft Discovery, teams can:

Accelerate end‑to‑end research with autonomous, multi‑agent systems that conduct literature analysis, scientific reasoning, simulation, and tool execution at scale
Unify institutional knowledge through GraphRAG‑powered Bookshelves that transform proprietary documents and scientific data into structured, queryable knowledge graphs
Scale advanced computation on Azure supercomputing infrastructure to support large‑scale simulation, modeling, and design‑space exploration
Collaborate with confidence in enterprise‑grade workspaces featuring built‑in RBAC, managed identities, and full data sovereignty

Figure 7. Microsoft Discovery.

Importantly, Discovery does not treat AI outputs as final answers. Instead, it embeds them into an explicit scientific reasoning loop, where:

Knowledge is represented as contextual, versioned graphs rather than static text
Conflicting evidence and assumptions are surfaced, not hidden
AI agents specialize, adapt, and learn across iterations
Researchers remain in control, with traceable sources and explainable steps

All outputs are intended to support, not replace, expert scientific and clinical judgment.

Figure 8. Microsoft Discovery Scientific Reasoning Loop.

Built on Microsoft Azure, Microsoft Discovery orchestrates teams of specialized AI agents using a graph-based knowledge engineering framework and able to leverage AI models available through Microsoft Foundry. The platform integrates advanced AI, high-performance computing (HPC) and quantum capabilities, and can connect insights back to the physical world to enable continuous experimentation and refinement. Meanwhile, Microsoft Discovery remains fully extensible to an organization’s own models, agents, tools, and datasets while meeting stringent enterprise requirements for trust, governance, security and compliance.

Figure 9. Enterprise Agentic R&D Platform Microsoft Discovery.

4. Using GigaTIME within Microsoft Discovery for precision oncology R&D

Microsoft Discovery is the overall agentic R&D platform. GigaTIME is one of the many AI tools that can be used on the Discovery platform to generate spatially resolved tumor microenvironment features from routine pathology, and then connect those features to downstream reasoning, validation, and iteration. GigaTIME provides population-scale, spatially resolved tumor microenvironment features derived from routine pathology.

When GigaTIME runs as a standalone notebook or point solution, the pipeline is often held together by ad hoc storage, cross-team handoffs, and manual input/output tracking (for example, whole slide images and patches in one location, predictions in another, single-cell localization outputs elsewhere, and downstream analyses in separate scripts).

In Microsoft Discovery, the pipeline is reshaped with governed ingestion, model execution, post-processing/feature extraction, and iterative reasoning. So that each stage produces typed, versioned inputs for the next instead of “files you have to hunt down”. Operationalizing GigaTIME in Discovery shifts the day-to-day experience from “run a model, then assemble context elsewhere” to “ask, explore, and iterate in one governed workspace”. In addition to that, Microsoft Discovery provides comprehensive suite of tools that transform data from sources like science catalog and AI models into actionable insights and validated findings. These tools include intelligent multi-agent orchestration, a cognitive discovery engine, a bookshelf, high-performance compute and validation of hypotheses, scientific reasoning, and an iteration framework.

Within a Discovery Platform, researcher can build customized analytics workflows for image ingestion, model inference, visualization, and these can become standardized building blocks rather than one-off analyses. Because the platform is extensible, teams can integrate additional models from Microsoft Foundry, third-party tools, or in-house pipelines alongside GigaTIME, creating a governed, end-to-end tumor immune phenotyping and discovery workflow.

Figure 10. Microsoft Discovery platform using GigaTIME as R&D tool (alongside other models, data sources, and R&D capabilities)

In the future, we expect Discovery to empower the research community to explore several other R&D applications by incorporating new models like GigaTIME alongside additional tools, datasets, experimental systems, and domain knowledge, including:

Exploring tumor responses to immunotherapy treatment by linking spatial immune context
Supporting drug-discovery research by connecting spatial phenotypes to molecular pathways and targets
Helping researchers generate hypotheses about candidate biomarkers and therapeutic targets by contextualizing population-scale signals against prior evidence in a knowledge graph.
Informing research on treatment stratification using cell-aware spatial signatures beyond bulk averages

GigaTIME and Microsoft Discovery are intended for research and development purposes. They are not medical devices and are not intended to diagnose, prevent, monitor, predict, prognose, treat, or alleviate any disease or condition. Any clinical application would require separate validation and applicable regulatory clearance.

5. From tutorial to platform scale impact

The virtual phenotyping the tumor immune microenvironment with GigaTIME shows that virtual mIF outputs are most useful value when they are localized, contextualized, and interpreted as part of a spatial system rather than isolated channels. When integrated into Microsoft Discovery, these outputs form the foundation for scalable, auditable, and collaborative oncology R&D.

With this integration, Microsoft Discovery reflects a broader shift in how AI is applied to science. The objective is no longer simply to run individual models or analyses faster, but to help evolve how R&D is conducted by embedding reasoning, learning, and orchestration directly into the scientific process. In this way, outputs from tools like GigaTIME can be translated into testable hypotheses and validated decisions.

Ultimately, this about providing tools that can help researchers examine complex systems, structure their reasoning, and iterate on their analyses.

Microsoft Discovery is now available in preview. Ready to take the next steps and try out platform with GigaTIME and any other Microsoft 1P or 3P Models available through Microsoft Foundry:

Microsoft Discovery expended preview announcement https://aka.ms/MicrosoftDiscoveryBlog

Learn and practice how Microsoft Discovery can help scientists and engineers transform research and development at https://aka.ms/microsoftdiscovery

Follow our tutorial notebook to understand how to deploy GigaTIME using Microsoft Foundry model catalog, reproduce the results described here, and understand how to use it for your own workloads: https://aka.ms/gigatime-sample

Access GigaTIME model card, learn model details and access deployment.

This post contains forward-looking statements regarding potential future capabilities, research directions, and applications of GigaTIME and Microsoft Discovery. These statements reflect current plans and expectations, are subject to change without notice, and do not constitute a commitment to deliver any functionality, feature, code, or service. Actual results may differ.

Special thanks to Microsoft cross functional team for their great support:

@Jeya Maria Jose Valanarasu, Sr. Scientist, Microsoft Research Health Futures
@Naoto Usuyama, Principal Researcher at Microsoft Research Health Futures
@Hao Qiu, Data Scientist, HLS Frontiers
@Ivan Tarapov, Senior Director, Multimodal Healthcare AI at Microsoft
@Saumil Shrivastava, Principal Product Manager, Microsoft Foundry
@Bella Chan, Principal Product Manager, Microsoft Discovery
@Ash Jogalekar, Senior Program Manager, Microsoft Discovery
@Nihit Pokhrel, Senior Product Manager, Microsoft Discovery
@Lily Kim, General Manager, Microsoft Discovery
@Samuel De Freitas Martins, Senior Director, Strategy and Partnerships
@Mu Wei, Principal Applied Science Manager, Health and Life Sciences
@Hoifung Poon, General Manager, Microsoft Research Health Futures

References

[1] U.S. Food and Drug Administration. FDA approves tisagenlecleucel for B-cell ALL and tocilizumab for cytokine release syndrome. 2017.

[2] Valanarasu JMJ, et al. Multimodal AI generates virtual population for tumor microenvironment modeling. Cell. 2026.

[3] Valanarasu JMJ, et al. Multimodal AI generates virtual population for tumor microenvironment modeling. Cell. 2026.

[4] Sood Anup et. al., Comparison of Multiplexed Immunofluorescence Imaging to Chromogenic Immunohistochemistry of Skin Biomarkers in Response to Monkeypox, Viruses 12 (8), 787

[5] Santamaria-Pang, A., et.al., Automated Phenotyping via Cell Auto Training (CAT) on the Cell DIVE Platform, 2019 IEEE BIBM,

[6] Brummel K, Eerkens AL, de Bruyn M, et al. Tumour-infiltrating lymphocytes: from prognosis to treatment selection. British Journal of Cancer. 2023.

[7] Zhao L, Jin S, Wu H. Tertiary lymphoid structures in diseases: immune mechanisms and therapeutic advances. Signal Transduction and Targeted Therapy. 2024.

[8] Sun Q, Hong Z, Zhang C, et al. Immune checkpoint therapy for solid tumours: clinical dilemmas and future trends. Signal Transduction and Targeted Therapy. 2023.

[9] Choi Y, Jung K. Normalization of the tumor microenvironment by harnessing vascular and immune modulation to achieve enhanced cancer therapy. Experimental & Molecular Medicine. 2023.

Modernizing Digital Health Record Governance with Microsoft Entra Identity Governance

Randall_Irwin — Mon, 20 Apr 2026 13:38:42 GMT

The digital transformation of healthcare continues to accelerate. Clinicians expect near-instant access to Electronic Health Records (EHRs), clinical workflows increasingly span cloud and on-premises systems, and regulatory pressures around identity, access, and auditability have never been higher.

For healthcare security and IT leaders, one challenge consistently rises to the top: ensuring the right clinicians have the right access to EHR systems—no more, no less—throughout their lifecycle.

Microsoft Entra Identity Governance was built to help address these challenges. By connecting authoritative workforce data to Microsoft Entra, automating joiner-mover-leaver processes, governing access through access packages, and recertifying access over time with access reviews, organizations can move from manual administration to policy-driven automation across the workforce lifecycle.

This represents an important evolution for healthcare organizations that have historically relied on on-premises identity tooling to synchronize data among HR systems, directories, and clinical applications. With Entra Identity Governance Microsoft provides cloud-driven identity lifecycle automation, application provisioning, entitlement management, and access reviews that can be applied to users, guests, agents, groups, and enterprise applications—including EHR systems.

EHR platforms such as Epic, Oracle Health (Cerner), and Meditech were designed to support complex clinical roles, dynamic care teams, and granular security models. Our goal with Entra Identity Governance is to simplify and automate the provisioning and lifecycle of these digital health records.

Provisioning

Provisioning starts with a source of authority. Microsoft Entra Identity Governance HR-driven provisioning creates digital identities based on human resources systems, and Microsoft’s API-driven inbound provisioning extends that model by supporting integration with virtually any system of record, including credential systems, payroll systems, spreadsheets, flat files, and SQL tables.

Once workforce data is in Microsoft Entra ID, IT administrators can standardize attribute mappings and establish the identity foundation for joiner, mover, and leaver processes. Entra Identity Governance Lifecycle Workflows can automate downstream tasks after the identity is established, helping organizations coordinate onboarding, internal moves, and offboarding with less manual effort.

From there, Microsoft Entra automatic app provisioning can create, maintain, and remove user identities and entitlements in connected applications. Provisioning is supported by using connectors, protocols, agents, and Azure function and logic apps for SCIM, LDAP, SQL, REST, SOAP, PowerShell, and even custom ECMA and API based scenarios. For healthcare organizations, that means Microsoft Entra can serve as the control plane for governed downstream access to the directories, groups, enterprise applications, and electronic health record (EHR) systems of their choice.

Entitlement Management

Provisioning establishes the identity, but Microsoft Entra Entitlement Management governs what that identity can request and maintain access to. Entitlement management is the identity governance capability that automates access request workflows and access assignments. The core construct is the Access Package, which bundles all resources a user needs together in one governed unit.

Access packages can include applications, entitlements, groups, Teams, and SharePoint Online sites. Policies control who can request access, whether approvals are required, whether business justification is collected, and how long the assignment should last. This helps organizations move away from one-off entitlement decisions and toward a repeatable, policy-driven model that is automated.

Electronic Health Records may have hundreds or several thousand granular entitlements within them. Using Microsoft Entitlement Management and Access Packages customers can model clinical roles and automatically assign entitlements to users throughout their lifecycle. This easily enables RBAC (role based access control) and ABAC (attribute based access control) scenarios. Instead of manually stitching together individual permissions, organizations can publish business-friendly access packages for healthcare roles that are approved, time-bound, and easier to audit.

Access Reviews

Assigning access is only part of the governance challenge; organizations also need a way to verify that access is still appropriate over time. Access reviews in Microsoft Entra Identity Governance help organizations manage group memberships, access to enterprise applications, and role assignments so that only the right people retain access at the right time.

Access Reviews can be scheduled or ad hoc, delegated to managers, resource owners, or users for self-attestation, and tracked for compliance or policy reasons. These reviews can be performed with business-critical application access, external users, and even scenarios where systems are disconnected from Entra ID.

When a review finishes, Microsoft Entra Identity Governance will apply the outcome and remove access from users who no longer need it. In a healthcare context, that gives security and compliance teams a structured way to recertify access to the groups, access packages, and applications tied to EHR workflows that clinicians need. Overall, this reduces access creep and maintains clearer audit evidence for ongoing governance and compliance.

Microsoft Entra Suite

You can experience the benefits described in this article by deploying Microsoft Entra Identity Governance, which is part of the Microsoft Entra Suite, the industry’s most comprehensive Zero Trust access solution for the workforce. The Microsoft Entra Suite provides everything needed to verify users, prevent overprivileged permissions, improve threat detections, and enforce granular access controls for all users and resources, including electronic health records.

Get started with the Microsoft Entra Suite with a free 90-day trial.

For additional details, please reach out to your Microsoft Representative or Microsoft Partner.

Read more on this topic

Learn more about Microsoft Entra

Prevent identity attacks, ensure least privilege access, unify access controls, and improve the experience for users with comprehensive identity and network access solutions across on-premises and clouds.

Driving AI‑Powered Healthcare: A Data & AI Webinar and Workshop Series

CamilleWhicker — Thu, 04 Jun 2026 13:15:22 GMT

Across these sessions, you’ll learn how healthcare organizations are using Microsoft Fabric, advanced analytics, and AI to unify fragmented data, modernize analytics, and enable intelligent, scalable solutions, from enterprise reporting to AI‑powered use cases.

Whether you’re just getting started or looking to accelerate adoption, these sessions offer practical guidance, real‑world examples, and hands‑on learning to help you build a strong data foundation for AI in healthcare.

Date	Topic	Details	Location	Registration Link
May 6	Webinar: Microsoft Fabric Foundations - A Simple Path to Modern Analytics and AI	Discover how Microsoft Fabric consolidates fragmented analytics into a single integrated data platform, making it easier to deliver trusted insights and adopt AI without added complexity.	Virtual	Register
May 13	Webinar: Reduce BI Sprawl, Cut Cost and Build an AI-Ready Analytics Foundation	Learn how Power BI enables enterprise BI consolidation, consistent metrics, and secure, scalable analytics that support both operational reporting and emerging AI use cases.	Virtual	Register
May 19-20	In Person Workshop: Driving AI‑Powered Healthcare: Advanced Analytics, AI, and Real‑World Impact	Attend this two‑day, in‑person event to learn how healthcare organizations use Microsoft Fabric to unify data, accelerate AI adoption, and deliver measurable clinical and operational value. Day 1 focuses on strategy, architecture, and real‑world healthcare use cases, while Day 2 offers hands‑on workshops to apply those concepts through guided labs and agent‑powered solutions.	Chicago	Register
May 27	Webinar: Unified Data Foundation for AI & Analytics - Leveraging OneLake and Microsoft Fabric	This session shows how organizations can simplify fragmented data architectures by using Microsoft Fabric and OneLake as a single, governed foundation for analytics and AI.	Virtual	Register
June 3-4	In Person Workshop: Driving AI‑Powered Healthcare: Advanced Analytics, AI, and Real‑World Impact	Attend this two‑day, in‑person event to learn how healthcare organizations use Microsoft Fabric to unify data, accelerate AI adoption, and deliver measurable clinical and operational value. Day 1 focuses on strategy, architecture, and real‑world healthcare use cases, while Day 2 offers hands‑on workshops to apply those concepts through guided labs and agent‑powered solutions.	New York	Register
June 10	Webinar: From Data to Decisions: How AI Data Agents in Microsoft Fabric Redefine Analytics	Join us to learn how Fabric Data Agents enable users to interact with enterprise data through AI‑powered, governed agents that understand both data and business context.	Virtual	Register

How to Compute GPU Capacity for GPT Models (GPT‑4o and Later)

Yan_Liang — Mon, 30 Mar 2026 15:07:49 GMT

When deploying large language models like GPT‑4o, capacity planning is no longer about picking a GPU SKU. Instead, Azure abstracts GPU compute behind Provisioned Throughput Units (PTUs)—a model‑centric way to reason about GPU usage, throughput, and latency.

This post explains how GPU capacity is computed for GPT‑4o‑class models, and how to translate your workload into the right number of PTUs.

From GPUs to Tokens: The Mental Shift

With GPT‑4o and newer models, Azure does not expose GPUs directly. Instead:

GPU compute is consumed as token throughput
Throughput is measured in tokens per minute (TPM)
Capacity is provisioned using PTUs, which represent a fixed slice of GPU processing capacity

A PTU is not “one GPU.” It is a guaranteed amount of model‑processing capacity, backed by GPUs under the hood and optimized by Azure for that specific model. [learn.microsoft.com], [learn.microsoft.com]

The Key Change with GPT‑4o

For GPT‑4o and later models, input and output tokens are metered separately.

That matters because:

Input tokens (prompt processing) stress the model differently than
Output tokens (generation), which are more GPU‑intensive

Azure therefore assigns separate TPM budgets per PTU for input and output tokens.

GPT‑4o Throughput per PTU

For gpt‑4o, the effective per‑PTU capacities are:

Metric	Value
Input TPM per PTU	~2,500
Output TPM per PTU	~625
Input : Output ratio	4 : 1

These ratios are baked into Azure’s PTU calculators and provisioning logic.

The Core Formula

To compute required GPU capacity (PTUs):

Then:

Round up
Apply minimum deployment constraints (e.g., 15 PTUs for Global / Data Zone)

Step‑by‑Step Example

Assume this workload:

800 input tokens
150 output tokens
30 requests per minute

Compute TPM

Input TPM

Output TPM

Convert to PTUs

Input side

Output side

Take the bottleneck

Apply Azure’s minimum deployment size → 15 PTUs required.

This is why tables often show PTUs higher than a simple TPM ÷ constant calculation.

Why Output Tokens Matter More

Output tokens:

Are generated sequentially
Consume GPU compute longer per token
Drive latency and tail performance

That’s why GPT‑4o uses a 4:1 input‑to‑output ratio, and why output TPM often becomes the bottleneck in chatty or agentic workloads. [modelavail...bility.com]

Practical Guidance

Short prompts, long answers → output‑bound → more PTUs
Large prompts, short answers → input‑bound → more PTUs
Stable traffic → PTUs give predictable latency
Spiky traffic → consider Standard + spillover

Azure recommends validating sizing with the PTU Calculator and real traffic benchmarks before committing long‑term reservations.

Final Takeaway

For GPT‑4o and newer models, GPU sizing is token‑driven, not hardware‑driven.
PTUs abstract GPUs, and the required capacity is simply the maximum of input‑bound and output‑bound throughput needs.

Once you understand that, GPT‑4o capacity planning becomes predictable, explainable, and much easier to operate at scale.

Configuring Noise Detection and Barge‑In with Azure Voice Live API

Yan_Liang — Mon, 30 Mar 2026 14:17:45 GMT

Natural voice conversations depend on two things: knowing when a user is speaking and letting them interrupt naturally. Without reliable noise detection and barge‑in, voice agents feel rigid and frustrating—especially in real‑world environments like call centers or mobile scenarios.

Azure Voice Live API addresses this by providing built‑in noise handling, server‑side Voice Activity Detection (VAD), and native barge‑in support—all configurable with a single session update.

How Voice Live Handles Noise and Interruption

Voice Live performs server‑side speech detection on the incoming audio stream. Instead of relying on simple volume thresholds, it can use Azure Semantic VAD, which is more resilient to background noise and conversational fillers.

When enabled:

Background noise is ignored

Speech start and stop are detected automatically

User speech can interrupt the assistant mid‑response (barge‑in)

All of this happens without stitching together separate STT, silence detection, or TTS cancellation logic.

The Key Configuration: session.update

Noise detection and barge‑in are configured using the session.update event, typically sent immediately after opening the Voice Live WebSocket session from client side such as ACA or Azure function.

Below is a recommended baseline configuration:

{

"type": "session.update",

"session": {

"modalities": ["text", "audio"],

"input_audio_format": "pcm16",

"output_audio_format": "pcm16",

"input_audio_sampling_rate": 24000,

"turn_detection": {

"type": "azure_semantic_vad",

"threshold": 0.5,

"prefix_padding_ms": 300,

"silence_duration_ms": 500,

"interrupt_response": true,

"auto_truncate": true

}

What this configuration enables:

Ø Azure Semantic VAD for robust speech detection

Uses Azure’s semantic model to detect actual speech, not just sound energy. This dramatically reduces false positives from background noise.

Noise‑tolerant turn detection

True barge‑in via interrupt_response: true

Immediate stop and truncation of AI audio when interrupted

Ø threshold

Controls how sensitive speech detection is.

Lower value → more sensitive
Higher value → less sensitive

Typical values:

0.3–0.4 for quiet environments
0.5–0.7 for noisy call centers

What Happens at Runtime

Once configured:

The assistant begins speaking
The user starts talking
Voice Live detects speech server‑side
AI audio stops immediately
Unplayed audio is discarded
The user’s speech becomes the active turn
No custom interruption logic is required—your application simply reacts to speech start events.

Common Mistakes to Avoid

Sending audio before session.update
Forgetting interrupt_response: true
Using overly aggressive thresholds
Ignoring speech start events on the client

Best Practices

Use Semantic VAD in noisy environments (call centers, mobile)

Tune the threshold (higher for noisy spaces, lower for quiet rooms)

Enable echo cancellation and noise suppression on the client microphone

Always enable auto_truncate when using barge‑in

Sample JavaScript code:

import WebSocket from "ws";

const VOICELIVE_URL =

"wss://<your-resource>.services.ai.azure.com/voice-live/realtime" +

"?api-version=2025-10-01&model=<model>";

// Use Entra ID token or api-key header

const ws = new WebSocket(VOICELIVE_URL, {

headers: {

// Recommended:

// Authorization: `Bearer ${process.env.AZURE_AI_TOKEN}`

// or api-key for non-browser clients

// "api-key": process.env.AZURE_VOICELIVE_API_KEY

}

});

ws.on("open", () => {

console.log("Connected to Voice Live");

ws.send(JSON.stringify({

type: "session.update",

session: {

modalities: ["text", "audio"],

input_audio_format: "pcm16",

output_audio_format: "pcm16",

input_audio_sampling_rate: 24000,

turn_detection: {

type: "azure_semantic_vad",

threshold: 0.5,

prefix_padding_ms: 300,

silence_duration_ms: 500,

interrupt_response: true,

auto_truncate: true

}

}));

console.log("session.update sent");

});

Image Search Series Part V: Building Histopathology Image Search with Prov-GigaPath

Alberto_Santamaria — Mon, 16 Mar 2026 22:07:53 GMT

@Alberto Santamaria-Pang, Principal AI Data Scientist, Healthcare ISE and Adjunct Faculty, Johns Hopkins School of Medicine

@Asma Ben Abacha, Senior Applied Scientist, HLS AI

@Manoj Kumar, Director HLS, Data & AI HLS Frontiers AI

@Jameson Merkow, Principal Applied Data Scientist

@Mu Wei,

@Ivan Tarapov, Senior Director, Multimodal Healthcare AI at Microsoft

1. Introduction

In earlier posts, we showed how to build a practical 2D medical image search system: take an image, turn it into an embedding with a foundation model, and use similarity search to find the closest prior cases [1]. We also demonstrated why radiology + pathology together matters for cancer workflows, where imaging findings and tissue evidence complement each other and can be combined in a single pipeline [2,3]. But in real clinical practice, prediction alone isn’t enough. Doctors routinely need to pull up similar prior cases across modalities, compare patterns, and check whether what appears on MRI lines up with what is confirmed under the microscope.

This post focuses on making that workflow practical for pathology. Using a pathology foundation model (Prov-GigaPath) as a retrieval backbone, we convert pathology images into searchable embeddings, build an index, and return the most similar slides in seconds—using the same retrieval pattern introduced in the image search series [1]. Because this approach fits naturally alongside radiology representations used in multimodal pipelines, it helps close the radiology–pathology gap and supports diagnostic concordance with evidence clinicians can directly review [2,3]. An overview of the end-to-end workflow is shown in Figure 1.

Figure 1. Histopathology image search workflow with linked radiology (MRI) context.

2. Histopathology Data

Even with strong foundation models, clinicians still face a practical problem: they need to compare current cases against prior cases across radiology and pathology, not just receive a prediction. In real workflows, diagnostic concordance often hinges on questions like: “Do these MRI findings match what we see in the tissue?” and “Have we seen a similar slide pattern before, and what did it correspond to on imaging?” Our tutorial 2d_pathology_image_search.ipynb addresses this gap by treating pathology as a search problem: extract embeddings from pathology images, index them, and retrieve the most similar prior cases so clinicians can review evidence rather than rely only on model outputs.

A second problem is interoperability. Clinical systems evolve quickly, and the retrieval layer must remain usable even as models change. The architecture in this workflow is intentionally simple and model-agnostic: any foundation model that produces embeddings can plug into the same pipeline (embed → index → retrieve). In this tutorial we use pathology embeddings from Prov-GigaPath and take advantage of an existing radiology–pathology mapping (MRI linked to pathology cases) to make retrieval more impactful: once a relevant pathology case is retrieved, the corresponding radiology context can also be surfaced to support concordance. In this notebook the mapping already exists, but in practice the same idea can be extended to multi-modal indexing, where both pathology and radiology embeddings are indexed (separately or in an aligned space) so that search can pull relevant information across modalities as part of a single workflow.

For this tutorial, we use pre-computed pathology embeddings derived from TCGA-GBMLGG, a curated cohort of 170 subjects with H&E-stained histopathology slides and tumor Grade labels (0/1/2). We split the cohort into ~80% training (to build the FAISS index) and ~20% test (to evaluate retrieval performance), with each image represented as a 1536-dimensional embedding generated by GigaPath (Table 1).

Table 1. TCGA-GBMLGG dataset summary and embedding configuration

Property	Value
Total subjects	170
Split	~80% train / ~20% test
Tumor grades	Grade 0, Grade 1, Grade 2
Image type	H&E-stained histopathology slides
Embedding dimension	1536 (GigaPath)

3. Building the Image Search Engine

To build the pathology search engine, we follow the same practical steps described in the 2D image search blog: (1) turn each image into an embedding using a foundation model, (2) build a vector index (we use FAISS) over those embeddings, and (3) retrieve the nearest neighbors for any new query image. Concretely, we take a pathology image (typically a tile/patch from a whole-slide image), run it through the pathology foundation model to produce a spatial feature map, and then apply adaptive pooling to convert that variable-sized feature map into a fixed-length embedding. Adaptive pooling matters because it guarantees a consistent embedding shape even when patch sizes or resolutions vary. Without that, indexing and distance comparisons become brittle and hard to scale.

Once we can reliably generate embeddings, the rest of the search engine is straightforward: we compute embeddings for the pathology corpus, build a FAISS index (e.g., flat L2 for a baseline), and then run query → embedding → nearest neighbors to retrieve similar pathology cases. Example retrieval results across tumor grades (0–2) are shown in Figure 2. To make “similarity” more clinically meaningful, we optionally apply a lightweight adapter implemented as a small MLP, on top of the foundation embeddings. In the notebook, the adapter takes 1536-D GigaPath embeddings as input (in_channels=1536) and produces a compact 254-D representation (adapter_emb_size=254), trained with a simple 3-class objective (num_class=3, Grades 0/1/2). This is intentionally lightweight compared with retraining the foundation model: we only learn a small mapping from embeddings to a better-aligned space, then rebuild the FAISS index using the adapted vectors (gigapath_adapter_features) to improve retrieval relevance. The effect of this optimization is visualized in Figure 3, which contrasts the baseline embedding space with the adapter-optimized space.

Figure 2. Nearest-neighbor retrieval examples for Grade 0, Grade 1, and Grade 2 queries.

Figure 3. Embedding space before and after lightweight adapter optimization.

4. Results

We evaluated pathology image retrieval using cancer Grade (0/1/2) as the clinical label. For each query pathology tile/patch in the test set, we searched a FAISS index built from the training set embeddings and computed Precision@K, defined as the fraction of the top-K retrieved items that share the same Grade as the query. In the notebook, we evaluate K = [1, 3, 5], comparing baseline embeddings.

Table 2. Overall retrieval precision before and after refinement

Overall Precision@K using (i) baseline GigaPath embeddings and (ii) refined adapter-informed embeddings.

Embedding space	Precision@1	Precision@3	Precision@5
Baseline (GigaPath)	0.5795	0.5593	0.5739
Refined (adapter-informed)	0.7727	0.7967	0.7689

Overall retrieval quality improves substantially after refinement (Table 2), with consistent gains across all K values, indicating that nearest neighbors become more aligned with Grade-consistent similarity.

Table 3. Precision by cancer Grade before and after refinement

Precision@K stratified by pathology cancer Grade (0/1/2) for baseline vs refined embeddings.

Grade	Baseline P@1	Baseline P@3	Baseline P@5	Refined P@1	Refined P@3	Refined P@5
0	0.5000	0.5000	0.5500	0.6250	0.6667	0.6250
1	0.3636	0.3030	0.3091	0.8182	0.8485	0.7818
2	0.8750	0.8750	0.8625	0.8750	0.8750	0.9000

Performance differs by Grade (Table 3). Baseline retrieval is strongest for Grade 2, moderate for Grade 0, and weakest for Grade 1, suggesting Grade 1 is the most challenging cohort under raw embeddings. After refinement, Grade 1 improves markedly across all K values, while Grade 2 remains high and improves slightly at deeper retrieval (P@5).

Table 4. Absolute improvement in precision after refinement

Absolute change (Δ = refined − baseline) in Precision@K, overall and by Grade.

Cohort	Δ Precision@1	Δ Precision@3	Δ Precision@5
Overall	+0.1932	+0.2374	+0.1951
Grade 0	+0.1250	+0.1667	+0.0750
Grade 1	+0.4545	+0.5455	+0.4727
Grade 2	+0.0000	+0.0000	+0.0375

As summarized in Table 4, the improvements are driven primarily by Grade 1 (ΔP@1 = +0.4545; ΔP@3 = +0.5455; ΔP@5 = +0.4727). Note on Grade 2: ΔP@1 and ΔP@3 are 0.0000 because baseline retrieval for Grade 2 is already high (P@1 = 0.8750, P@3 = 0.8750; Table 3), so the adapter does not change top-1/top-3 correctness for that cohort. The improvement appears at deeper retrieval (ΔP@5 = +0.0375), suggesting the adapter mainly refines ranking beyond the top few results rather than increasing an already strong “hit rate.”

Collectively, these results indicate that the refined embedding space makes similarity more grade-consistent, which is exactly what diagnostic concordance workflows need: when clinicians retrieve “similar” pathology cases, they want those neighbors to reflect clinically relevant groupings (here, tumor grade), and to remain interpretable when linked to the corresponding radiology context. The fact that Grade 1 benefits most is also plausible from a pathology standpoint: intermediate grades often show more overlap in morphology with both lower and higher grades (i.e., less separable visual patterns), while higher grades may exhibit more distinctive features that are easier to retrieve correctly even without refinement. In that sense, the lightweight adapter acts as a targeted calibration step: shaping the embedding space so that ambiguous, overlapping cases (like Grade 1) are pulled closer to the right neighbors.

Figure 4. Histopathology (H&E) retrieval with linked radiology (MRI) context.

5. Conclusion

We built a practical histopathology image search engine using a simple, reusable pattern: generate Prov-GigaPath embeddings from pathology tiles (with adaptive pooling to produce fixed-length vectors), index them with FAISS, and retrieve nearest neighbors for any query. This matters because retrieval makes foundation models actionable for clinicians, returning similar prior examples that can be reviewed and compared, rather than only producing a prediction score. The implementation is lightweight and interoperable: once embeddings are available, the remaining steps are standard vector indexing and search, and the same design naturally extends to multimodal workflows by linking retrieved H&E cases to their corresponding MRI context (or by indexing radiology and pathology embeddings side-by-side for cross-modal lookup).

Image Search Series: Blog Posts & Jupyter Notebooks

The Microsoft healthcare AI models, including MedImageInsight, are intended for research and model development exploration. The models are not designed or intended to be deployed in clinical settings as-is nor for use in the diagnosis or treatment of any health or medical condition, and the individual models’ performances for such purposes have not been established. You bear sole responsibility and liability for any use of the healthcare AI models, including verification of outputs and incorporation into any product or service intended for a medical purpose or to inform clinical decision-making, compliance with applicable healthcare laws and regulations, and obtaining any necessary clearances or approvals.

References

[1] Image Search Series Part 1: Chest X-ray lookup with MedImageInsight | Microsoft Community Hub
[2] Cancer Survival with Radiology-Pathology Analysis and Healthcare AI Models in Azure AI Foundry (Microsoft Healthcare & Life Sciences Blog) [3] Adapter training notebook (MedImageInsight). Microsoft healthcareai-examples GitHub repository (azureml/medimageinsight/adapter-training.ipynb)

Agents: Snack Pack

samhitaraman — Fri, 06 Mar 2026 01:07:32 GMT

It’s not just you.

Agents are everywhere, and everyone else seems to know exactly what they are.

If you’ve been nodding along while secretly thinking “I should probably look this up"... you've found the right place!

This Agents Snack Pack contains easy‑to‑digest content designed to help you understand agents, one piece at a time. Content is arranged from left to right in order of increasing complexity. If you're starting out, begin with Column 1 (Access) and progress as you build confidence!

#1. Discover

#2. Use

#3. Build

Drill in the fundamentals:

What are Microsoft 365 Copilot agents and how to use them | Microsoft

Ask big questions, let Researcher do the investigation.

Researcher: A reasoning agent in Microsoft 365 Copilot

Create your first agent:

Microsoft 365 Copilot Chat - Agent builder demo

Explore the Agent Store:

Agent Store in Microsoft 365 Copilot

When spreadsheets get messy, call in Analyst.

Analyst: A reasoning agent in Microsoft 365 Copilot - YouTube

*Create an Epic Tip Sheet Agent: https://youtu.be/ilL5j5qBdpE

*Agent requires premium Copilot license

Practical use cases:

Sales: Microsoft 365 Copilot Chat - Sales agent demo

Customer Service: Microsoft 365 Copilot Chat - Customer service agent demo

Demo: Leveraging Analyst to break down hospital occupancy data

https://youtu.be/0OG6yTpZoMU

*Build your first autonomous agent:

How to create an autonomous agent with Copilot Studio

*Agent requires premium Copilot license

Hungry for more? Join our live, instructor-led AgentRx sessions to get hands on guidance with Microsoft experts.

Bringing Organizational Knowledge into the Clinical Workflow

BertHoorne — Thu, 05 Mar 2026 17:40:47 GMT

This blog is co-authored by Hadas Bitran, Partner GM, Health AI, Microsoft Health & Life Sciences

Every day, clinicians spend valuable time looking for information that lives in different places. An email thread from a specialist colleague. A Microsoft Teams discussion about a complex case. Updated organizational processes buried in SharePoint or OneDrive. This information provides context that could be critical to their workflows or help inform their decisions. But that context is not part of their clinical workflow.

The result? Clinicians are forced to break their clinical workflow, searching manually across organizational resources, and mentally combining scattered data points, all while a patient is waiting. This isn't a knowledge problem. It's a retrieval problem. And it's costing time, focus, cognitive burden and clinical confidence every single day.

That's exactly the gap we're closing by bringing clinical intelligence and your organization's knowledge into one seamless, workflow-native experience.

Clinical workflow, now with your organizational context

Within Dragon Copilot, clinicians will be able to securely surface relevant information across Microsoft 365, without leaving the clinical workflow:

Email: retrieve relevant information that was exchanged with patients, colleagues or from specialist correspondence, referral communications, or care coordination threads.

find me the email from Dr. Ting that mentioned the latest research about this mutation.
In this example, the chat functionality in Dragon Copilot uses the patient and encounter context to resolve the referenced mutation, then leverages Microsoft 365 Copilot behind the scenes to locate the email from Dr. Ting that mentions it.
Microsoft Teams: surface information from Microsoft Teams chats that the clinician had with colleagues, discussions or group chat conversations.

The patient is traveling to Florida. Identify dialysis centers near the patient’s destination based on information shared by Dr. Salomon in Microsoft Teams and provide practical travel guidelines I can share with the patient.
In this example, Dragon Copilot uses trusted sources for travel guidelines and Microsoft 365 Copilot to retrieve relevant Microsoft Teams messages from Dr. Salomon, identifying nearby dialysis centers in Florida.
SharePoint and OneDrive: access organizational knowledge on demand: HR policies, facility procedures, compliance guidelines, shift schedules, and more

Who is on call for nephrology tonight and who is covering tomorrow morning?

In this example, Dragon Copilot leverages Microsoft 365 Copilot behind the scenes to locate the most up‑to‑date Excel file with upcoming shift and coverage information from the hospital’s SharePoint, and surfaces the answer directly in the conversation, without disrupting the clinician’s workflow.

With Microsoft 365 Copilot, work context is available directly inside Dragon Copilot, clinicians can choose if, and when to access their work information. Within Dragon Copilot, they can ask questions in natural language and receive the most relevant information, grounded in patient context, from trusted clinical sources and their Microsoft 365 data. One conversational flow. Full clinical and work context. No tab switching, no manual searching, no lost focus.

Trusted by design, built for healthcare

Security and privacy are built in from the ground up. Information is always accessed on behalf of the individual user, fully respecting existing Microsoft 365 identity and access management, compliance, and privacy controls, meaning clinicians see only what they're authorized to see, and that Dragon Copilot will only use their work context if the clinician consented to it. This also means no new security risks to manage, and no changes to how your organization governs access to information.

For healthcare organizations where data sensitivity, regulatory compliance, and patient privacy are non-negotiable, this better-together experience is designed to meet that bar from day one.

Join the Private Preview

If you're a Dragon Copilot customer, and your organization is using Microsoft 365 Copilot, we invite you to be among the first to experience this new capability. Register now for early access to the private preview and play a role in shaping the future of clinical workflow intelligence.

Register for private preview

Dragon Copilot centralizes trusted medical content and relevant contextual information in-workflow

hadasb — Fri, 06 Mar 2026 15:00:20 GMT

This blog is co-authored by Bert Hoorne, Principal Program Manager & Ksenya Kveler, Principle Medical Science Manager

Dragon Copilot delivers medical intelligence from trusted sources directly within clinical workflows for healthcare organizations in one solution.

We are pleased to announce that we are expanding those knowledge sources with additional best‑in‑class content providers and enabling broader access to your organization’s internal sources with Microsoft 365 Copilot integration.

Access information from new credible medical content providers

Dragon Copilot users will gain access to an additional robust collection of

trusted clinical content from leading evidence-based resources. We are partnering with renowned publishers to bring you the best, most trusted content, safely and securely, within clinician’s workflows while helping to reduce the use of unauthorized AI tools and applications, commonly referred to, as “shadow AI.”

Access content from Wolters Kluwer UpToDate

We’ve partnered with Wolters Kluwer UpToDate to bring trusted, evidence-based clinical guidance directly into Dragon Copilot. Customers with an active Wolters Kluwer UpToDate license will be able to access UpToDate content in Dragon Copilot, within the context of their clinical workflows.

This integration allows clinicians to ask both general questions and patient specific questions and receive answers grounded in UpToDate evidence, with clear references to supporting sources. Over time, it will also introduce contextual links to UpToDate concepts layered on top of Dragon Copilot–generated notes, further enhancing clinical insight at the point of care.

“Clinicians need reliable guidance that supports fast, confident decision-making without disrupting care delivery. We are excited to partner with Microsoft to bring UpToDate’s gold standard evidence and expertise-based clinical insights to Dragon Copilot, helping clinicians quickly access, actionable answers that reduce cognitive burden and support better patient care.”

Yaw Fellin, Senior Vice President and General Manager, UpToDate Clinical Decision Support and Provider Solutions
Wolters Kluwer Health

Here’s an example of UpToDate content embedded in the Dragon Copilot workflow:

Wolters Kluwer UpToDate powering Dragon Copilot Chat answers

Obtain trusted clinical evidence with Elsevier ClinicalKey AI

Elsevier’s ClinicalKey AI will be available in Dragon Copilot. This integration enables customers with an active Elsevier ClinicalKey AI license to surface trusted medical literature and clinical evidence directly within clinicians’ workflows.

“Clinicians are navigating a complex and rapidly changing healthcare landscape and need solutions they can trust. The ClinicalKey AI extension for Dragon Copilot transforms how clinicians interact with trusted medical literature and clinical answers. The conversational interface makes evidence discovery faster and more intuitive.”

Jukka Valimaki, SVP Clinical Solutions
Elsevier

Here’s an example of ClinicalKey AI content embedded in the Dragon Copilot workflow:

Support clinical decisions with EBMcalc

With the integration of EBMcalc medical calculators, Dragon Copilot enables clinicians to use evidence-based calculators directly within their workflows—applied in context to the patient they’re caring for.

“Clinicians need trusted, evidence-based insights exactly at the point of care. By integrating EBMcalc’s rigorously curated clinical calculators and references into Dragon Copilot, we’re helping make high quality medical evidence more accessible, more actionable, and easier to use within everyday clinical workflows”.

Louis Leff, MD, MACP, Founder and CEO
EBMcalc

Access independent evidence in Dragon Copilot with Wiley and Cochrane

Wiley and Microsoft are partnering to bring scientific literature and clinical evidence directly into the healthcare workflow, starting with the Cochrane Library. Through this integration, customers with an active Cochrane Library AI license will be able to access Cochrane’s high-quality, independent evidence, systematic reviews, and clinical answers, to inform more reliable and efficient decision-making. This includes the Cochrane Database of Systematic Reviews (CDSR), the home of gold-standard evidence syntheses, widely used to inform clinical guidelines worldwide.

"Working with Microsoft to bring the Cochrane Library into Dragon Copilot reflects a shared commitment to meeting researchers and clinicians where they are. Healthcare Institutions can now access independent, peer-reviewed evidence— right within their clinical workflow”

Josh Jarrett, SVP & GM of AI Growth
Wiley

Access work context with Microsoft 365 Copilot in Dragon Copilot

With the Microsoft 365 Copilot integration, Dragon Copilot enables clinicians to seamlessly access information from their emails, chats, OneDrive and SharePoint, within the flow of their clinical work. Clinicians can combine this information with additional questions and actions, all governed by existing organizational and user access controls. Use of this data within Dragon Copilot workflow remains fully at the user’s discretion.

Here’s an example of content from an email surfaced by Microsoft 365 Copilot accessible through the Dragon Copilot workflow:

Microsoft 365 organizational context in Dragon Copilot

Read more for a deeper dive on how Dragon Copilot enables work context access with Microsoft 365 Copilot integration.

Safe web search

Dragon Copilot safe web search delivers trusted, evidence linked answers when curated sources are unavailable—ensuring clinicians continue to receive timely support without disrupting their workflow.

The goal of safe web search is to prevent broken workflows and eliminate unsafe external browsing. Clinicians remain within their clinical context, focused on the patient—without tab hopping or the risk of landing on unreliable or unverified websites.

Safe web search eliminates “no response” dead ends by maintaining a seamless conversational experience in Dragon Copilot and reducing unanswered prompts.

This capability is enabled by using verified, secure, and responsible mechanisms designed for safe clinical experiences. It enforces multilayer protection through evidence validation, provenance linked responses, content filtering, and regulated search with built in safeguards.

Here’s an example of content from a safe web search in the Dragon Copilot workflow:

Conclusion

These advancements represent an important step forward in how Dragon Copilot delivers trusted medical intelligence - bringing together best‑in‑class clinical evidence, organizational knowledge, and safe web access in one governed, in‑workflow experience. We will continue to expand our partner ecosystem, deepen integrations with leading evidence providers, and evolve Dragon Copilot conversational extensibility to meet clinicians where they work.

Why nursing needs a different kind of AI—and how Dragon Copilot delivers

Allison_Novick — Thu, 05 Mar 2026 14:30:00 GMT

The Dragon Copilot experience for nurses was made generally available (GA) in December 2025 with a clear mission: help nursing staff focus on care, not the computer.

From the start, the goal was to create a comprehensive AI clinical assistant—one that works alongside nurses throughout their shift, reduces cognitive load, captures the full scope of care delivered, and translates real clinical work into automated next steps, including documentation—fundamentally transforming workflows to keep patient care at the center.

Microsoft has continued to execute on that vision. Recent enhancements include extended mobile access with Android support—enabling nurses to record care in Epic Rover on Android devices—as well as significant expansion in ambient documentation coverage. Together, these advances reflect a consistent approach: adoption follows when technology aligns with how nurses work.

Expansive nursing documentation coverage

Nursing work spans multiple flowsheet templates, assessments, state changes, and, at times, narrative notes. When solutions support only a subset of this work, nurses are left filling gaps after the fact—reintroducing cognitive load and eroding the value of this technology.

Microsoft has expanded Dragon Copilot’s ambient documentation capabilities by broadening the range of supported nursing value types—and by extending it to deliver complete coverage across all flowsheet templates in supported departments and settings. The result is comprehensive documentation generated from each recording including:

Lines, Drains, Airways, and Wounds (LDAs) documentation, including assessments, additions, and removals
Nurse notes, automatically generated from natural nurse-patient conversations and voice memos captured on the go
Full flowsheet template coverage—not just a subset—including admission and discharge flowsheets, blood administration, CIWA-Ar, and care plan-related flowsheets
Adaptations to each organizations charting philosophy, including macros support, chart-by-exception, pertinent positives, and more

This breadth matters because nursing work is rarely captured within only a narrow set of flowsheets—nor does it typically result in just narrative notes. Yet many solutions labeled “for nurses” prioritize what is easiest to automate, rather than what nurses need. The result can be a false sense of completeness, with nurses still managing gaps across their shift.

Why nursing ambient documentation is hard—and what makes Dragon Copilot unique

Achieving comprehensive, high‑quality nursing documentation has required specialized technology designed to address the structural, workflow, and feedback challenges unique to nursing—challenges that general narrative ambient models and physician‑oriented solutions are not built to solve:

Flowsheets are messy, complex, and frequently changing
Flowsheets are large, hospital-specific, internally ambiguous, and constantly evolving under governance. Complex logic—such as cascading rows, documentation‑by‑exception patterns, and duplicative or overlapping rows—makes it far from straightforward to accurately map a clinical observation to the correct field and value. Microsoft works directly with real hospital schemas, handling hierarchy, ambiguity, and multiple valid documentation destinations—without requiring flowsheet redesign or sacrificing quality.
Nurses don’t speak for documentation
Bedside language is optimized for care delivery, not chart completeness. Critical details are often implied or never spoken aloud. Microsoft’s technology translates natural nursing communication into accurate documentation without changing nurse behavior. Built on industry‑leading transcription accuracy and decades of speech recognition expertise, Dragon Copilot is informed by real‑world integration across diverse EHR environments, preserving accurate translation and clinical intent that directly impact downstream documentation accuracy.
Nursing audio is diverse
Recordings mix shorthand, dialogue, monologue, and unit-specific language. Dragon Copilot accounts for mixed speaking modes instead of flattening audio through a generic pipeline or requiring nurses to speak in constrained ways.
Feedback loops are noisy
Nurse corrections to AI output often reflect hindsight or personal preferences rather than model error. Microsoft’s approach analyzes correction patterns with clinical context, enabling calibration at the institution, department, and even individual user level.
Bedside workflows demand predictability
Baseline LLMs are not suited for real-world nursing accuracy, latency, and cost requirements — especially with tens-of-thousands of possible flowsheet values. Dragon Copilot is optimized for consistent performance across real nursing environments, exceeding the reliability and latency characteristics of baseline models.

Beyond specialized nursing architecture, Dragon Copilot enforces strict quality and safety gates for new documentation outputs—including oversight by Microsoft’s internal, nurse-led Clinical Integrity team, phased validation, and Responsible AI review—ensuring new documentation covered meets defined nursing standards before being introduced at scale.

Dragon Copilot represents a fundamental shift in how nursing work is supported by AI by meeting the full complexity of bedside care head-on. By delivering comprehensive ambient documentation across live inpatient care environments, Dragon Copilot helps ensure that the care nurses provide is accurately captured, trusted, and usable downstream. The result is an AI clinical assistant that keeps nurses focused on what matters most: their patients.

From dictation to intelligence at the cursor with Dragon Copilot Desktop

James_Jeffries — Mon, 09 Mar 2026 13:24:33 GMT

This blog is co-authored by Dr. David Ting, Chief Clinical Product Lead, and Sarah Grey, Senior Product Manager.

Today’s patient encounters are packed with friction that breaks clinicians’ flow, clouds clinical thinking, and drains the joy from practice. Instead of focusing on patients, listening, assessing, and collaborating on care, clinicians spend visits staring at a screen, wrestling with bloated EHRs, and acting as data entry clerks. They type into endless fields, check boxes, click buttons, and memorize arcane text shortcuts and key sequences designed to satisfy computer hard-stops, regulatory tests, and payer demands, not to deliver compassionate, high-quality care.

And it doesn’t stop with the EHR. Clinicians bounce between imaging systems, referral portals, messaging apps, mobile devices, specialty tools, and the nonstop demands of email, policies, training, credentialing, and CME. Managing care across dozens of disconnected systems every day makes one thing clear: clinicians and healthcare organizations are desperate for relief.

Can artificial intelligence help? Of course it can. But hope in AI is too often disappointed by the reality of AI solutions that are poorly integrated with each other and with clinicians’ holistic workflow. AI risks being trapped in a single application like the EHR, leading to disjointed and suboptimal experiences.

But if AI lives in a separate window from where clinicians actually work, it risks adding another source of friction, forcing context switching and manual copying and pasting across the EHR and other systems.

What clinicians need is a way to access a seamless clinical intelligence everywhere they work. This is precisely what Dragon Copilot brings to the beleaguered physician, nurse, and advanced practice provider.

Extending the power of AI to remove friction

The reality for most clinicians is that their patients’ information and care coordination is managed in an EHR and via complementary apps living on the Windows desktop. If an AI clinical assistant is to be helpful, it must be available anywhere the clinician is working, regardless of which of the many desktop applications happen to be in focus. As the clinician moves from the EHR to a PACS viewer, web browser, email client, or Teams meeting, the AI should be able to understand the underlying context and offer ways to streamline the clinician’s flow.

Dragon Copilot’s AI changes the dynamic. When editing text anywhere – in the EHR, in Outlook, Word, a web browser app, or within Dragon Copilot’s own editor – a clinician simply selects desired text, speaks or types a natural language instruction (e.g., “Expand this paragraph to reflect more of the patient’s description of the car accident”), and receives an inline rewrite or insertion directly in the target application.

A clinician searching for information can simply place their cursor over the text, and Dragon Copilot understands the context, allowing the clinician to make multi-part requests: “Given this diagnosis, what is the recommended treatment, and are any of those covered by the patient’s insurance plan?” Dragon Copilot gathers context, searches through approved, trusted knowledge sources, and provides the answer within the Dragon Copilot workflow.

Here’s another example: while reviewing an online guideline or internal protocol in the organization’s SharePoint, a clinician can select a passage and ask, “Give me the three key takeaways from this reading and summarize them in patient-friendly language to include in the after-visit summary.” The clinician receives the in-context summary directly in workflow.

Dragon Copilot overcomes EHR-constrained AI and disconnected tools by unifying and delivering intelligence in one centralized workspace.

Dragon Copilot is the AI clinical assistant connecting fragmented systems

Care doesn’t happen in a single app. Even though the EHR is the system of record, clinicians still do significant work outside of it. Yet EHR-based AI cannot reach outside the EHR itself. And most third-party tools don’t (and won’t) ship deep AI integrations quickly.

The result is a frustrating array of AI-powered experiences that can only perform a part of the required task – such as an EHR ordering agent that cannot read the hospital formulary in SharePoint, or a third-party coding solution that cannot automatically map the provider’s visit diagnoses. That leaves the clinician in an unhappy position of needing to be data courier, manually copying information from one application to the other, rather than spending time taking care of the patient.

Dragon Copilot acts as the connective tissue across that fragmentation. By working with standard text controls, it can bring a consistent interaction model, including dictation, commands, and cursor-native AI, across the clinical workflow.

Shifting from speech dictation to natural language editing

Over 650,000 providers worldwide have benefited from computerized speech-to-text dictation using Dragon Medical One. Traditional speech-to-text systems, like Dragon Medical One convert spoken audio into text word for word, like a courtroom transcript. But Dragon Copilot provides a new way to turn language into clinical content, in addition to capturing patient encounters ambiently.

With Dragon Copilot, clinicians can just say what they want by speaking naturally. They can instruct Dragon Copilot to perform targeted edits (“Summarize the HPI in two sentences”) and issue high-leverage whole-document edits that used to be tedious. Clinicians can even talk naturally to create documentation from scratch in new ways:

“Using only the details already documented, draft an HPI and list a few clarifying questions to ask.”

“Update the patient’s pronouns throughout the note (don’t change clinical facts).”

“Rewrite the entire note in a more concise style, preserving meaning and keeping all facts the same.”

“Write the A&P using my standard knee template with a conservative treatment plan.”

These are new use cases where a short instruction can yield large, efficient edits. And for Dragon Medical One users who have benefitted from Dragon Medical One voice macros (called Step-By-Step commands) and custom vocabularies, these are all transferrable to Dragon Copilot.

The cursor becomes a reusable (and extensible) primitive

The cursor provides a dependable, ubiquitous interface for AI. While EHRs and complementary clinical and non-clinical applications may present vastly different UIs, they generally rely on the underlying operating system to provide the same cursor. Hence, users have come to expect a common behavior and access to a core set of functions associated with the cursor, regardless of the underlying application. Thus, the cursor defines scope (selected text vs. current field), intent (summarize, rewrite, extract), and placement (results appear where the clinician expects them).

In-workflow, cursor-anchored AI like Dragon Copilot’s turn that repeatable pattern into a platform primitive—a foundational, reusable building block that developers can rely on to deliver consistent experiences across applications. For Microsoft developers as well as third-party extension partners, this platform primitive provides experience extensibility: app teams and agent developers will be able to deliver new skills at the cursor through Dragon Copilot. Rapid expansion of Dragon Copilot capabilities becomes possible, because internal and partner developers can focus on the capabilities themselves instead of reinventing the UI. And end-users benefit from a coherent, always-discoverable, always-available experience.

Cursor-anchored AI can be safer and more trustworthy, especially with integrations

Healthcare demands trust. Cursor-anchored AI keeps actions tied to what the clinician can see and edit, and it makes output reviewable, editable, and reversible.

That doesn’t mean ignoring backend-integrated context. The goal is to combine the value of integrations with an interaction model that keeps scope clear: what the AI acted on, what it used, and what it produced, so clinicians stay in control.

Conclusion: Dragon Copilot on the desktop unifies fragmented systems and delivers a seamless cross-application intelligence in one solution

Dragon Copilot represents the next and necessary step in the evolution of healthcare AI. With today’s status quo, clinical use of generative AI is largely restricted to algorithmic clinical decision support or ambient documentation creation. Clinicians are still faced with disparate AI systems that are either tightly bound to EHRs but limited in scope beyond the EHR, or poorly integrated with the EHR, such that the AI systems lack sufficient visibility into patient context or require manual data extract, transform and load. In either case, clinicians are stuck with the role of being human data couriers; and that is still a far cry from the patient-clinician relationship they went to medical school or nursing school to practice.

With Dragon Copilot, AI isn’t “somewhere else.” It is intelligence embedded in the act of information gathering, analyzing, synthesizing, writing, revising, deciding, and executing. It is AI assistance right where you work. It represents less context switching, better notes, and stronger knowledge work support.