The readiness work nobody talks about: scaling AI in healthcare
Who this is for: leaders deciding whether and how to put agents into clinical and operational workflows.
A health system built an agent that passed the demo and failed the shift. It was back-office automation. No diagnosis, no treatment recommendation, no patient conversation — it should have been safe. It pulled information together for a clinical team and saved real time on paper. The numbers were good. Then it landed on the floor and the clinicians pushed back.
When they looked at why, it wasn’t the automation. The way it surfaced its output added a step to their day instead of removing one. It saved time for the organization and cost time for the people using it. That is a governance failure, not an engineering failure. Nobody asked the question that governance exists to ask: should this agent work this way for these people in this workflow?
This is the readiness work that decides whether an agent becomes part of care delivery or another pilot people route around. And it echoes Part 1, one layer up: the people who block AI are usually naming a real readiness gap.
Governance as enablement, not restriction
In Part 1 we mapped the people who block healthcare AI and what they worry about. Here, we govern the things they were right to worry about. Most teams hear “governance” and picture a brake, or a committee whose default answer is no.
Run it that way and it becomes a blocker. Run it right and it’s the opposite. Governance is the structure that lets good ideas move fast and stops risky ones early. It’s how you put an agent into a clinical workflow and have the CMO, the CISO, and Legal already comfortable, because the questions they would've raised were answered before the agent reached them. The systems that scale agents aren’t the ones with the loosest rules. They’re the ones that can quickly tell a safe idea from a risky one.
Healthcare-specific agent risks
A generic agent-governance framework misses what makes clinical environments different. Four risks stand out.
Clinical workflow disruption. An agent that saves time in aggregate can still cost time at the point of care, exactly like the back-office example above. In a clinical setting, that extra time is not a productivity footnote. A workflow that adds friction during a patient encounter gets abandoned. An abandoned tool that still runs in the background is a governance liability with no upside.
Patient-facing versus internal. Whether an agent touches a patient is the most important call you’ll make. The test is simple: can the patient see it, hear it, rely on it, or be affected by it without knowing it exists? A patient-facing agent inherits the full weight of consent, disclosure, and algorithmic-harm scrutiny that Part 1 mapped to the patient advocacy and ethics stakeholders. An internal agent carries real risk too, but a different kind. Treating the two the same is how you either over-govern an internal tool into uselessness or under-govern a patient-facing one into an incident.
Physician trust and the monitoring perception. An agent that observes or summarizes clinical activity can read as surveillance, upholding the same dynamic that makes works councils and clinicians wary of usage analytics. If clinicians believe an agent is grading their judgment, they will not use it honestly. They may click through it, work around it, or stop documenting the way the workflow expects. Trust is essential. How an agent is framed and who it reports to is a governance decision, not a marketing one.
Liability and accountability. When an agent contributes to a clinical or operational decision, someone owns the outcome. Governance requires a decision about who the owner is before the agent goes live. An agent with no named accountable owner is not ready for production, however good the demo looks.
The agent use case vetting framework
Vet before you govern. Decide whether an agent should exist in that form before you decide how to control it. This framework answers two questions: Should we build it? And if so, at what risk tier?
|
Dimension |
Question to answer |
Why it sets the controls |
|
Clinical consequence |
Could this agent change, delay, accelerate, or shape care, even from a back-office workflow? |
Clinical leadership should be involved even when the workflow looks administrative |
|
Patient-facing or internal |
Does a patient interact with this agent directly or indirectly? |
Patient-facing agents trigger consent, disclosure, and ethics review as an essential gate |
|
Governance owner |
Who is accountable for this agent’s behavior in production? |
No named owner, no production |
|
Liability boundary |
If this agent contributes to a bad outcome, who owns it and how is it caught? |
Decided here, in design, so it isn't discovered later in an incident |
Score your use cases against the above framework before a line of code is written. A clinical, patient-facing agent with no clear owner and no liability boundary is not a project — it is an incident waiting for a budget code. An internal, operational agent with a named owner, a measurable outcome, and a clear handle on what data it touches is a fast yes.
The governance triad
No single function can govern an agent well by itself. IT could build guardrails that ignore clinical reality, Legal could write boundaries nobody can operate inside, and the business could chase an outcome with no controls. The governance triad answers a different question than the vetting framework: not whether to build, but who decides and signs off. The work is in running all three together rather than letting one dominate.
- IT sets the technical guardrails: What the agent can access, which identity it runs as, what is logged, and how it is monitored and revoked. This is where Part 2’s identity foundation pays off: you can only scope an agent if you can govern who it is.
- Legal sets the risk boundaries: What the agent may and may not do, where consent or disclosure is required, and where the liability line sits. Not a veto at the end, but a boundary set at the start.
- The business sets the success criteria: The measurable outcome the agent exists to produce, owned by the function that benefits. Without this, you cannot tell a working agent from a running one.
IT, Legal, and the business sign off together and early on the same agent, so it reaches production having already cleared the three objections that would otherwise stop it. A triad that meets and designs together leads to shipped agents.
Walkthrough: a prior-authorization agent from prototype to governed production
Prior authorization is a well-known source of administrative burden: fax machines, phone tag, and lots of waiting. The path from prototype to governed production runs through every step listed below.
Step 1: Build. The team builds a prototype in Microsoft Copilot Studio. The tool is low-code, so the idea becomes something people can try in days instead of a planning cycle.
Step 2: Vet. The framework classifies this agent as internal-facing and operational — with real clinical consequences, since a delayed or misrouted authorization can hold up or redirect care. It stays out of the heaviest consent gates yet still gets clinical oversight, a clear escalation path, and a leadership review. Because prior authorization touches PHI, HIPAA minimum-necessary and role-based access apply regardless of patient interaction, so the agent retrieves only the requested records.
Step 3: Triad sign-off. IT sets the guardrails. The agent runs under an agent identity in Microsoft Entra, governed through Microsoft Entra Agent ID, with access scoped to only the systems it needs. All actions are logged. Legal sets the liability boundary — the agent assembles the packet, flags missing evidence, and routes the request. An accountable human role approves what gets submitted with escalation to the ordering clinician when evidence is missing or ambiguous. Finance, the business owner, sets two success metrics: cycle time on the request and rework caused by missing or wrong documentation.
Step 4: Govern in production. Microsoft Agent 365 is the chosen control plane for governing and securing the agent. In the Microsoft 365 admin center, the Agent Registry gives you one place to see it, block or restrict what it connects to, and review its deployment approval. Its identity and lifecycle run through Microsoft Entra Agent ID. Microsoft Purview adds audit, data-loss-prevention in supported scenarios, and data-risk signals, alongside the runtime and system-of-record logs. In plain terms: the agent has its own name badge, so you can always see what it touches and shut it off if needed.
Step 5: Measure and reframe. In this scenario, the outcome to aim for is a measurable reduction in cycle time. What makes it stick is framing. Pitched as a tool that “monitors your authorization decisions,” it reads as surveillance and stalls. Pitched as the thing that “removes the fax-and-phone-tag,” clinical and operational sentiment flips. The frame is a governance decision about trust, not a tagline.
The one-page agent readiness checklist
It all comes down to a short set of yes/no gates. When the vetting framework decides whether to build and the triad decides who signs off, the below checklist answers the last question: can the agent go live safely? If you cannot answer yes to every question, write the missing owner, control, metric, review, or stop condition into the launch plan before production, no matter how good the prototype is.
- Governance clarity. Is the agent classified as clinical or operational, patient-facing or internal, and governed to the right level for that class?
- Operational ownership. Is there a single named owner accountable for the agent’s behavior in production?
- Data quality and access. Does the agent reach only what it needs, as a scoped identity, with the data behind it good enough to trust?
- Measurable outcome. Is there a defined metric the agent exists to affect, owned by the function that benefits?
- Shutdown trigger. Is there a named condition that takes the agent out of production? Does someone have both the authority and the access to act on it? This is the gate most often skipped and the one that matters most when an agent starts behaving in a way no one anticipated.
Answering “yes” across the board does not make the agent perfect. It makes it governable enough for a controlled production rollout with a way to watch it and a way to stop it. Answering “no” anywhere is not a reason to abandon the idea. It is the next piece of readiness work to do before go-live.
Get the people right, get identity right, and govern well. That’s the readiness work. That’s the work that decides whether AI lands.
The pattern held across all three parts of this series: the technology was rarely the hard part. Part 1 was the people who decide whether AI ships. Part 2 was identity, the gate that turns Copilot on. Part 3 is governance, the structure that lets you put agents into real workflows and have the organization say yes. None of it shows up in a Copilot demo, but all of it decides whether the demo becomes a deployment.
Charles Wallace is a Senior FastTrack Architect at Microsoft. He works on Microsoft 365 Copilot and agent adoption across healthcare and life sciences and on the security, identity, and governance foundations that decide whether AI deployments actually succeed.