The healthcare industry’s relationship to agentic artificial intelligence has reached an inflection point. The technology is advancing faster than most organizations can adopt it — and in healthcare that gap carries consequences for administrators, clinicians, payers and patients. Unlike other industries where delayed adoption means lost efficiency, in healthcare it can mean delayed care, compromised privacy and erosion of clinical trust.
The organizations that lead aren’t necessarily the ones moving fastest, but those who possess a clear understanding of how agentic AI actually works — and where responsible use can break down
Understanding the Layers
Agentic AI is often thought of as a single system. Functionally, however, it operates across five interdependent layers:
- Power requirement: like all cloud-computing platforms, a strong grid is needed to power the computing of every AI agent
- Infrastructure/hardware: the data centers that are springing up around the world are where the data every AI agent uses “lives”
- Network layer: sophisticated digital safeguards must be in place to ensure private data remains private
- Data layer: digital data delivery requires a complex chain of protocols
- Application layer: the way a user interfaces with an AI agent can take a variety of forms
Every agentic AI user depends on all five. To troubleshoot a problem, or improve a process, users must know which layer to target. And to use these systems responsibly, we need to understand where each layer is vulnerable
What ‘Responsible Use’ Really Means
Most conversations about responsible AI focus on the top three layers — network, data and application — because end users interact with these layers most often. Malicious actors can infiltrate an insecure network. But irresponsible use doesn’t require bad intent. A well-secured organization can still fail at the data layer through hallucinations that seem harmless until they aren’t
Consider a concrete example: A patient rates their pain a 10 on a 10-point scale. An LLM might interpret that rating as “severe pain,” and insert that phrase into a clinical summary, but the physician never used the phrase himself. That single interpretive leap, multiplied across thousands of cases, can create documentation that misrepresents clinical reality, exposes organizations to legal risk, and erodes the trust of the clinicians the system was built to support
Responsible use requires more than secure infrastructure. In particular, three principles must undergird any AI initiative:
- Bias mitigation. The history of medicine offers hard lessons about what happens when treatments and protocols embed the biases of their era. AI systems trained on that same historical data can perpetuate those patterns at scale. Healthcare organizations deploying agentic AI must actively audit for bias across age, race, gender and other factors — not as a compliance exercise, but as a clinical imperative.
- Observability. Every person whose work is touched by an AI agent needs visibility into what that agent is designed to do and how it’s performing. Without it, there’s no way to confirm the system is operating within its guardrails — and no way to catch it when it isn’t.
- Explainability: The ability to articulate what an AI agent does, and why it does it, shouldn’t be optional. Third-party audits are a necessary safeguard. The capacity to explain a system’s purpose to stakeholders outside the immediate team ensures both internal and external accountability.
Responsible governance in this space — for now, at least — requires more than holding a single membership or certification
Begin with a multi-pronged approach. Industry frameworks like the National Institute of Standards and Technology’s (NIST) Risk Management Framework and the Open Worldwide Application Security Project (OWASP) offer actionable insights for managing AI risk and security. HIPAA compliance remains non-negotiable for any healthcare AI deployment. Organizations like Coalition for Health AI (whose Agentic AI work group I belong to) are working toward codifying agentic AI principles into industry-wide best practices.
Frameworks alone aren’t enough. The organizations already getting this right are integrating these standards into their own internal governance frameworks — contextualizing them to their specific workflows, patient populations and risk profiles — rather than treating compliance as a box to check. The goal is to build the kind of trust that makes AI a durable asset — not a liability
Deliberate Adoption Beats Moving Quickly
Healthcare organizations can’t afford to ignore agentic AI, but they also can’t afford to deploy it without the infrastructure to use it responsibly. The gap between these two failure modes — overanalyzing to the point of paralysis, and underanalyzing to the point of recklessness — is where the real strategic work lives
Agentic AI represents a bridge between predictive, generative, and autonomous systems. Building that bridge to last requires keeping a human meaningfully in the loop — the mechanism by which guardrails are set, monitored, and enforced over time
Health systems struggle less with model accuracy than deciding who is responsible when the model is wrong. The most successful adopters build the kind of clinical and organizational trust that makes agentic AI transformational, both for their teams and for their patients.
