The Short Version
- Most healthcare organizations are accumulating AI tools without an integration strategy, creating expensive silos that do not compound
- The shift from "tool buying" to "architecture design" is the difference between incremental improvement and systemic transformation
- Healthcare leaders need to think in layers (data, orchestration, governance, interface) rather than in vendor names
The Problem With the Current Approach
I talk to health system CIOs every week. The pattern is consistent.
They have an ambient scribe from one vendor. A prior auth bot from another. A patient engagement platform from a third. A coding assistant from a fourth. None of these tools share data. None of them share a governance framework. None of them learn from each other's mistakes.
So when the ambient scribe misinterprets a medication name, the coding assistant does not know about it. When the prior auth bot discovers a payer policy change, the patient engagement platform keeps sending outdated information. When the coding assistant flags an unusual pattern, nobody else in the stack benefits.
This is not an AI strategy. It is a procurement list with a buzzword attached.
And it is expensive. Not just in licensing fees, but in the hidden costs: integration maintenance, duplicate data pipelines, conflicting audit trails, and the staff hours spent reconciling outputs across systems that should be talking to each other.
What Architecture Looks Like
Think of it as four layers:
Layer 1: Data Foundation
A unified clinical and operational data layer. Not a data lake (those are where data goes to drown). A curated, governed, FHIR-aligned data service that every AI agent can query. One source of truth for patient records, payer rules, scheduling constraints, and clinical protocols.
Layer 2: Orchestration
A workflow engine that routes tasks to the right agent. Patient calls in? The orchestrator checks identity, determines intent, and dispatches to scheduling, triage, or billing. No hardcoded if/then trees. Dynamic routing based on context, urgency, and available capacity.
Layer 3: Governance
A shared policy layer. Every agent follows the same rules for PHI handling, clinical escalation, audit logging, and human override triggers. When CMS changes a billing rule, you update the governance layer once, not six vendor dashboards.
Layer 4: Interface
The thin layer where humans interact with the system. For clinicians: an EHR-embedded panel. For patients: voice, chat, or portal. For administrators: a dashboard showing what every agent is doing, what it decided, and why.
Most organizations are buying Layer 4 tools and hoping they will magically produce Layers 1 through 3. That is not how infrastructure works.
Why This Matters Now
Three forces are converging:
Model commoditization. GPT-5, Claude, Gemini, Llama, and their successors will all be "good enough" for most healthcare tasks within 18 months. The model is no longer the differentiator. The architecture around it is.
Regulatory pressure. CMS, ONC, and state regulators are moving toward requiring explainability and auditability for AI-assisted clinical decisions. A patchwork of vendor-specific audit logs will not pass muster.
Cost reality. The average health system spends $1.2M annually on AI tools (Bain, 2025). Most cannot demonstrate ROI because they cannot measure cross-tool impact. Architecture enables measurement. Tool collections do not.
The Pharmacy Analogy
I come from pharmacy. Here is what I know about medication management: a patient on 12 medications from 4 specialists with no pharmacist reconciling the list is a patient heading for an adverse event.
The same principle applies to AI in healthcare operations. Twelve AI tools from four vendors with no architecture reconciling the outputs is an organization heading for a compliance incident, a patient safety event, or both.
Medication therapy management works because a pharmacist looks at the whole picture. AI architecture works because an orchestration layer looks at the whole operation.
The Bottom Line
- Audit your stack before your next purchase. List every AI tool in your organization. Map the data flows between them. Identify where data is duplicated, where audit trails diverge, and where a failure in one tool creates a blind spot in another. If you cannot draw the diagram, you do not have architecture. You have accumulation.
- Invest in the boring layers. Data normalization, API governance, shared authentication, centralized logging. These do not make great demo videos. They make great organizations. Every dollar you spend on orchestration and governance saves three dollars in integration maintenance and incident response.
- Design for composability, not commitment. Your architecture should let you swap any individual AI agent without rebuilding the system. That means standard interfaces, clean data contracts, and agent-agnostic orchestration. The vendor landscape will look different in two years. Your architecture should survive the churn.