All Insights
    Healthcare IT Nov 11, 2026 24 min read

    The AI-Native EHR Has a Permission Problem

    When an AI assistant can change the chart, trigger work and move patient data, the organization must prove who authorized each action. BCP's architecture for enforcing AI permissions outside the model, and six failure demonstrations to require from vendors.

    Brandywine Consulting Partners
    Healthcare AI & Interoperability Practice
    The AI-Native EHR Has a Permission Problem
    25 min read 4,889 words

    When AI can change the chart, trigger work, and move patient data, healthcare needs an architecture that can prove who authorized each action.

    Consider an illustrative referral. An AI assistant drafts a request and a clinician approves it. Before submission, the clinician withdraws the planned referral. The assistant still sends the older request, the response times out, and a retry creates a second request. The summary may have been accurate, yet the organization now has an obsolete instruction, a duplicate transaction, and a hard question: what, exactly, did the clinician authorize?

    In The AI-Native EHR Imperative, Brandywine Consulting Partners argued that integration readiness determines whether an AI investment delivers value. The next challenge begins when an assistant gains authority to act on that data, and an approval button cannot compensate for an execution path that ignores the conditions of approval.

    BCP's position is direct: an AI system should earn permission to execute a defined action by satisfying explicit evidence, identity, and operational requirements, defined per workflow and enforced outside the language model. Neither the label on the EHR nor the fluency of its answers establishes them.

    Key takeaways

    • ▸Adoption is not authorization. The AMA's 81% "awareness or use" figure includes 9% of physicians unsure which AI tools their practice offered.
    • ▸A correct answer can reach the wrong patient. In controlled fault injection, GPT-4.1 withheld writes in only 17.4% of full patient-header-swap runs; enforce patient binding outside the model.
    • ▸Grant separate permissions for separate effects. Read, draft, recommend, submit and cross-system coordination each need their own boundary, evidence and owner.
    • ▸Approval expires when its conditions change. The action executed must be the action reviewed, under conditions that still justify it.
    • ▸A timeout is an unknown outcome, not a failure. Reconcile with a durable request identifier before retrying; every unresolved outcome gets an owner.
    • ▸Ask for the failure demonstration. Six negative tests show whether a vendor's architecture can be trusted with write access.

    The adoption numbers do not authorize the next action

    In the American Medical Association's 2026 survey, 1,342 respondents answered the practice-use question: 72% reported at least one named AI use case, 9% were uncertain which AI tools their practice offered, and 19% reported none. The report combines the first two into an 81% awareness-or-use figure, which does not establish readiness for autonomous execution.

    Figure 1. Stacked bar: 72 percent of physicians incorporate at least one named AI use case, 9 percent are uncertain which tools the practice offers, and 19 percent report none; a bracket marks the first two as 81 percent awareness or use. Source: AMA 2026 Physician Survey on Augmented Intelligence, p. 5.

    Figure 1. Physician-reported AI use in practice. AMA 2026 survey, question n=1,342; fielded January 15–February 2, 2026. Self-report does not establish deployment quality or safety.

    Response categoryPercent
    Explicitly incorporates one or more named AI use cases72%
    Uncertain which AI use cases the practice offers9%
    Reports no AI use cases present in the practice19%
    Total "awareness or use" (first two categories)81%

    Documentation evidence is narrower than the promises attached to it. A randomized UCLA trial assigned 238 outpatient physicians to DAX Copilot v2.0, Nabla v1.5, or usual care. Relative EHR time-in-note declined 9.5% with Nabla versus control (95% CI −17.2% to −1.8%); DAX's estimate was −1.7% (95% CI −9.4% to +5.9%).

    Figure 2. Forest plot: Nabla version 1.5 at minus 9.5 percent change in EHR note time, 95 percent confidence interval minus 17.2 to minus 1.8; DAX Copilot version 2.0 at minus 1.7 percent, interval minus 9.4 to plus 5.9. A vertical line marks no change. Source: Lukac et al., NEJM AI, 2025.

    Figure 2. Relative change in EHR time-in-note versus control, single-system randomized trial, November 2024–January 2025; 79 physicians per product, 80 controls. The metric excluded editing within vendor applications. Results concern the studied versions and are not a current product ranking.

    Product (studied version)Relative change vs. control95% confidence interval
    DAX Copilot v2.0−1.7%−9.4% to +5.9%
    Nabla v1.5−9.5%−17.2% to −1.8%

    Both products integrated into Epic, showing that effective integration can deliver benefit without replacing the record system, and that organizations must test the deployed workflow, including work outside the EHR's time metric. Documentation efficiency is a reason to invest. It does not show that the same system can safely submit an authorization, change a medication record, or close a clinical task. Expanding autonomy should be a deliberate operational decision, never a side effect of enabling another product feature.

    A correct answer can still reach the wrong patient

    A 2026 study in the International Journal of Medical Informatics tested six models across 1.2 million tool calls in a controlled record environment built from MIMIC-IV emergency-department data, with swapped patient headers and smaller identity inconsistencies. GPT-4.1 withheld writes in only 17.4% of full-header-swap runs; detection of subtle medical-record-number or age changes was near zero. This was experimental fault injection, not a production error rate, but it shows that a model can copy the correct code while binding it to the wrong record. Scoring the code alone misses the failure that matters to the patient.

    Conceptual illustration of two patient chart panels, one cyan and one amber. An AI tool call carrying a correct medical code is aimed at the wrong amber chart and is stopped by a locked verification barrier, while a green identity-binding path links the code to the correct patient. Patient and encounter binding is enforced outside the model.

    BCP recommends treating patient and encounter binding as an enforced condition of every consequential action. The execution service should establish identifiers from trusted application context, validate cross-system mappings, and reject conflicts, never accepting an authoritative identifier because it appeared in a retrieved paragraph.

    Incomplete access is another trap. SMART App Launch specifies that underlying permissions can omit resources from an otherwise successful search, so a 200 OK does not prove the agent received every relevant record. A missing result must not become an assertion that the patient has no allergy, medication, or prior event; retrieval should carry known access restrictions and missing inputs into review. Identity resolution, terminology mapping, status semantics and source freshness belong in the action path: a well-formed FHIR resource can still describe a preliminary result, an inactive order, or the wrong encounter.

    Define what the agent may do before selecting how it thinks

    Reading records, drafting, and sending a request have different consequences, and a system may suit one but not another. Grant separate permissions for separate effects; greater autonomy is not the destination for every workflow.

    Conceptual illustration of five rising glass platforms, each a separate AI permission level: read, draft, recommend, submit and coordinate across systems. A lock gate stands between every step, and the higher, consequential steps glow amber. Each capability needs its own boundary.

    CapabilityRequired boundary
    Read and summarizeApproved patients, purposes, data sources and access limits; visible evidence gaps.
    DraftA distinct proposed state; clear authorship; no silent promotion into a signed record.
    RecommendReviewable evidence and limitations; intended use and an accountable clinical owner.
    Submit or updateApproval for the exact action, current authority, valid patient context and execution preconditions.
    Coordinate across systemsDurable request identity, verified outcomes and ownership of unresolved or partial work.

    SMART scopes delegate rights within underlying user or client policies; they do not model every permission in the clinical organization. The Model Context Protocol's authorization specification requires audience validation and prohibits passing the client's token straight through to an upstream API, which protects the agent-to-tool connection, not the permitted business action. The application still decides whether this actor may perform this operation for this patient and purpose under current conditions.

    Prefer narrow tools: one that submits a validated, approved referral is easier to govern than one that executes arbitrary API requests, and a payer-workflow assistant should not inherit prescribing authority because both share an integration platform. Model confidence never grants authority; it needs validation for a defined task and population, while authorization comes from policy and accountable people.

    Put the enforcement path outside the model

    The model produces a structured proposal; a separate execution path decides whether it may proceed. That separation should be inspectable in the design and provable in a negative test.

    Figure 3. Govern every AI action: BCP proposed reference architecture. Six stages run top to bottom: 1 evidence snapshot of patient, encounter, source versions and freshness; 2 the model produces a proposal with no direct destination write credential; 3 a policy gateway checks role, purpose, patient and current state, with a branch to reject or request review; 4 an approval check binds human review to the exact payload, source versions and approval expiry; 5 a constrained execution adapter rechecks authority and freshness with least privilege, idempotency and concurrency controls, branching to re-evaluate on conflict or reconcile on unknown outcome; 6 verify the destination outcome. Evidence and audit span every stage in a client-controlled record.

    Figure 3. BCP proposed reference architecture: policy and execution stay outside the model, and runtime checks can still stop an approved request. An engineering recommendation, not a design mandated by one standard.

    Preserve the evidence the action depends on

    The proposal should cite the source records, versions and timestamps it used, plus unresolved contradictions. Preserve units, terminology and clinical status so a preliminary laboratory result stays distinguishable from a corrected final one, and carry uncertain mappings forward rather than substituting a plausible value. FHIR supplies shared resource models and interfaces, not a universal AI-native storage engine, so governed AI can sit around an incumbent EHR, a new platform, or both, provided it behaves reliably across the interfaces it actually uses.

    Keep model routing inside the approved boundary

    A router should select only from models approved for the task, data sensitivity and intended use; cost or latency may decide within that set. If no eligible model is available, defer or use an approved manual path, never an unapproved fallback. Chart text, uploaded documents, external messages and retrieved passages are evidence, not instructions: they cannot authorize a disclosure, broaden a patient's scope, or turn a draft into a committed action. Test that boundary with deliberately hostile content before release.

    Make the proposed effect visible to the reviewer

    Show the reviewer the destination, intended change, supporting evidence, unresolved issues and source age; high acceptance rates mean little if reviewers cannot see what they approve. Record each approval against the specific patient, proposed payload or its digest, source versions, approver, policy version and expiry. A regenerated proposal needs new approval when its effect or relevant evidence changes.

    Separate origin, access and completion evidence

    FHIR Provenance describes the entities and activities that produced a resource; AuditEvent records security-relevant interactions. Neither proves that a referral was accepted or that an authorization workflow reached its intended state. BCP's proposed evidence record joins source references, observable model output, policy decisions, approval, tool and model versions, execution responses and confirmed destination state: an auditable account of what the system did, not a generated explanation of what the model claims it was thinking.

    Approval expires when its conditions change

    Photograph of a physician at a hospital workstation reviewing an approval on a tablet, with a patient chart open on the monitor behind her.A sign-off holds only while the conditions behind it still hold.

    In the referral, approval applied to a specific request built from a specific view of the patient; reusing it after the care plan changed defeats the purpose of review. FHIR supports version-aware updates using ETag and If-Match; where the endpoint supports them, a mismatched version returns 412 Precondition Failed instead of overwriting the newer resource. These controls address resource contention only; they do not make upstream evidence or downstream systems part of one transaction. The execution service must also recheck the conditions the action depends on, such as a care plan held elsewhere or payer coverage that changed after drafting. If one changes, stop and return the material change for review, using freshness tolerances agreed with the workflow owner so the system avoids both unsafe execution and needless interruptions.

    The practical test for human oversight: can the organization prove that the action executed was the action reviewed, under conditions that still justified it? A checkbox with an employee's name is weak evidence if the application can alter the payload or destination afterward.

    A timeout is an unknown outcome

    The receiving platform may have committed the referral before the connection failed, so blindly retrying a create can produce a duplicate. The workflow needs a durable request identifier and a destination-aware way to confirm whether the original operation completed.

    Figure 4. A timeout is an unknown outcome: BCP proposed reconciliation workflow. After executing the approved request and persisting a durable request ID and attempt state, the response is classified. Success leads to confirming the destination state; a conflict leads to re-reading and reviewing the changed state; a timeout or lost reply enters an explicit outcome-unknown state. From there, query before retrying using the durable request ID and destination state, then mark confirmed completed when evidence confirms, retry only when duplicate-safe, or route to a manual exception when unresolved. No blind retries, and no assumption that every action can be rolled back.

    Figure 4. BCP proposed reconciliation of uncertain execution: no blind retries, and an accountable owner for every unresolved outcome.

    A FHIR transaction succeeds or fails as a whole on one server; a batch is a set of independent operations whose individual results must each be inspected. Neither creates a distributed transaction across an EHR, payer platform, pharmacy and scheduler. Record each stage's verified outcome (submitted, accepted, pending, rejected, partially completed, or unknown), and let the business owner define completion; an HTTP response and a completed referral are different events.

    A client-generated idempotency key helps only if the destination honors it or the integration layer enforces equivalent behavior; otherwise route uncertain outcomes to reconciliation, with bounded retry policies. Recovery needs clinical judgment too: a sent message cannot be unsent by rolling back a row, and a canceled order may require a new authorized action. BCP recommends a remediation procedure for each consequential effect, with an owner and evidence of completion.

    Zero retention in a pass-through integration service fits this design when the client controls durable workflow state and clinical evidence. BCP's Transaction Layer Integration approach keeps necessary evidence in approved client systems and limits intermediary telemetry to approved operational metadata; correlation identifiers still need protection.

    The certification baseline is changing

    Photograph of a compliance officer and a health system executive reviewing a regulatory binder and a laptop at a conference table.When a requirement leaves the rulebook, write the evidence you still need into the contract.

    HHS published the HTI-5 final rule on October 28, 2026. Effective December 28, 2026, it removes the source-attribute, access and modification, and intervention risk management requirements in 45 CFR 170.315(b)(11)(iv)–(vi), with related maintenance requirements in 170.402(b)(4). As of November 11, 2026, these removals are not yet effective, and a revised DSI criterion remains. The rule reduces burden; it does not eliminate oversight of healthcare AI, and the DSI provisions were always scoped rather than covering every AI feature.

    Specify the assurance evidence you need in procurement and operating agreements: intended uses, validation results, material limitations, change notifications, monitoring access, incident responsibilities and exit provisions. A changing certification requirement should not silently change your standard for accepting a consequential action.

    FDA's January 29, 2026 CDS guidance sets out which software functions meet the statutory criteria for exclusion from the device definition, including enabling a professional to independently review the basis of recommendations; adding a review button does not resolve classification. NIST's voluntary AI Risk Management Framework and Generative AI Profile help organize accountable decisions and evidence, but confer no regulatory clearance.

    Measure completed work and the cost of supervision

    Generated notes, tokens and tool calls do not show whether operations improved. Count the workflow outcomes your team needs, including the labor to review, correct and reconcile. Define each denominator before a pilot, keep simulated safety tests separate from production observations, and report severity and unresolved cases alongside rates so a common low-risk action cannot hide a rare high-consequence failure.

    MeasureWhat to establish
    Authorization integrityValid authorization records divided by consequential actions; distinguish prevented test violations from production incidents.
    Identity and state protectionWrong-patient attempts blocked and stale-state conflicts detected in defined negative tests; investigate every production escape.
    Verified completionEligible initiated workflows reaching the agreed destination state; display pending, failed and unknown outcomes separately.
    Reconciliation burdenUnresolved outcomes by age and severity, plus staff minutes spent resolving them.
    Net work reductionBaseline labor minus review, correction, exception handling and other labor in the AI-assisted workflow.
    Cost per verified outcomeAllocated operating costs, including supervision and support, divided by confirmed completed workflows.

    Photograph of a nurse in scrubs at a hospital workstation carefully reviewing and correcting a draft clinical note across two monitors.Time saved counts only after review, correction and follow-up are on the scale.

    False stops matter too: a control that blocks valid requests or floods an exception queue can erase the benefit. Tune thresholds against both unsafe approvals and unnecessary escalation using representative populations, specialties, languages and operating conditions, and document what the evaluation does not cover. Model, prompt, mapping and tool-version changes should trigger targeted regression evaluation; last month's passing test is not permanent permission.

    How BCP would take one workflow into production

    BCP brings healthcare integration, Azure engineering, analytics and AI governance into one delivery process. Our published methodology moves through Assess, Design, Implement, Optimize and Govern, applied to one bounded operational outcome with explicit release criteria.

    StagePractical deliverable and release decision
    AssessMap the workflow, systems, evidence sources, identity joins and failure paths. Establish the baseline and select a bounded first use case.
    DesignDefine the action-permission matrix, evidence requirements, review experience and execution contract. Name the accountable owners.
    ImplementBuild scoped adapters, protected evidence records, version checks and reconciliation. Run historical replay and negative tests in an isolated environment before enabling production writes.
    OptimizeCompare verified outcomes and supervision costs with the baseline. Address recurring exceptions and validate changes on the affected tasks.
    GovernMaintain release approvals, evaluation evidence, access reviews and incident procedures. Reassess permissions when the workflow or model changes.

    Photograph of a physician, an engineer and two operations managers mapping a single workflow on a whiteboard wall of sticky notes.One bounded workflow, with clinical, technical and operations owners in the same room.

    Take prior-authorization packet preparation as an illustrative start. The agent gathers permitted sources, flags missing evidence and drafts a packet for staff review, so the team measures completeness and time before any submission; coverage verification, clinical recommendations and payer submission stay separate permissions. If submission is later enabled, the packet stays bound to the patient, payer, destination and supporting records, pending work is held until the agreed acknowledgment arrives, and coverage or evidence changes trigger a recheck. This is a recommended pattern, not a reported client outcome.

    In Azure, BCP maps these responsibilities to workload identities, protected APIs, approved model endpoints, durable workflow services and client-controlled evidence storage. Start in shadow mode with write tools disabled, then enable a limited scope under defined review, expanding only after controls and benefit are demonstrated, with a tested kill switch and a staffed manual workflow for continuity. BCP's agentic AI and integration services turn a proposed capability into a workflow with defined authority, reliable interfaces and inspectable outcomes, and the same assessment can show that improving the integration layer should come before any EHR replacement.

    Ask for the failure demonstration

    Photograph of a physician leader and an IT director watching a vendor engineer present a workflow dashboard showing an alert during a demonstration.The most useful vendor demo is the one where something goes wrong on purpose.

    At the next vendor review, ask the implementation team to demonstrate the following in an isolated test environment:

    • ▸Change the source record after approval. Show how the system detects the change and decides whether review is required again.
    • ▸Introduce a mismatched patient identifier. Show which component blocks the write and what evidence it records.
    • ▸Drop the response after the destination accepts a request. Show how the system avoids duplicate execution.
    • ▸Revoke the initiating user's authority before execution. Show that queued work cannot rely on stale permission.
    • ▸Place a hostile instruction in a retrieved document. Show that it cannot expand tool permissions or change the destination.
    • ▸Disable the model or connector. Show how staff identify unfinished work and resume the approved manual process.

    A vendor that completes these and explains its remaining limitations gives you evidence to evaluate; one that returns to model benchmarks has left the operational question unanswered.

    Questions health system leaders are asking

    What is the permission problem in AI-native EHRs?

    When an AI assistant can change the chart, submit work or move patient data, the organization must prove who authorized each action and under which conditions. Model accuracy or a generic approval button does not establish that; authority must be defined per workflow and enforced outside the language model.

    Do SMART on FHIR scopes or MCP authorization solve AI action permissions?

    No. SMART scopes delegate rights within underlying policies, and MCP authorization protects the agent-to-tool connection. The application must still decide whether this actor may perform this operation for this patient and purpose under current conditions.

    When should a human approval be considered invalid?

    When the conditions it relied on change. Bind each approval to the patient, payload or its digest, source versions, approver, policy version and expiry, and recheck them immediately before execution.

    How should an AI workflow handle a timeout?

    As an unknown outcome. Query the destination with a durable request identifier before retrying, retry only when duplicate-safe, and route anything unresolved to an owned manual exception.

    Does the HTI-5 final rule remove oversight of healthcare AI?

    No. HTI-5 removes specified decision-support certification requirements effective December 28, 2026, but a revised DSI criterion remains, and FDA CDS guidance and organizational assurance obligations still apply.

    Bring BCP the workflow your AI is about to change

    AI becomes more valuable when leaders can explain the authority they delegated and verify the work completed under it. Before enabling an agent to change records or submit work, bring Brandywine Consulting Partners one workflow, the systems it touches, and the outcome you want to improve. We will assess integration gaps, define action boundaries, and develop a staged implementation and support plan. Reach us through the project inquiry form, BCP's Agentic AI for Healthcare page, or contact@brandywine.consulting; we will arrange a secure review of any sensitive material.

    Give the next AI project a measurable operational outcome and an owner who can account for every consequential action, and require the implementation to demonstrate both before expanding its authority.

    Start a project inquiry · Book a discovery call with BCP · Explore AI Governance & Compliance

    References

    Sources reviewed November 11, 2026. Numbered citations support the associated factual claims. The referral and prior-authorization scenarios are illustrative, and the reference architecture, reconciliation workflow and measures are BCP recommendations. Study findings, standards and BCP recommendations retain their stated scope.

    Share this article

    Ready to put this into practice?

    BCP partners with healthcare and life sciences leaders to translate strategy into shipped, secure systems. Let's talk about your next initiative.

    Talk to BCP

    Related Insights

    Sorted by tag overlap