In a recent webinar hosted by MedCity News Editor-in-Chief Arundhati Parmar, experts explored the insights from the newly released Healthcare AI Readiness Index. The discussion, sponsored by Cotiviti, featured contributions from Cotiviti’s Chief Information Officer, Sean Warren, and Thomasina Anane, Associate Vice President of Enterprise Analytics at the Alliance of Community Health Plans. The panel focused on effective strategies for integrating AI into healthcare settings, addressing governance challenges, and scaling solutions within organizations.

Cotiviti specializes in transforming disjointed healthcare data into practical insights, aiding organizations in minimizing waste while combating fraud and enhancing operational efficiency. Warren highlighted the role of Cotiviti as a vital link that promotes smoother interactions among payers, providers, and various stakeholders. 

Anane shared insights from a recent ACHP annual meeting that centered on AI governance, emphasizing that proper management needs to begin well before AI technologies are put into operation and should adapt as use cases develop. She noted that health plans face numerous questions, such as the regulatory landscape surrounding AI, what data is utilized, and how decisions are influenced by AI outputs.

Both panelists identified ongoing vigilance as a significant challenge in AI governance within clinical and payer environments. “Deployment is really not the finish line,” Warren remarked, stressing that continuous follow-up, security assessments, and human oversight are crucial. Monitoring the operations of AI systems is vital, raising questions about the adequacy of existing safeguards and the capacity to reverse decisions or track actions taken by AI.

The conversation also ventured into several other critical topics, including:

- Assessments leaders should conduct before transitioning an AI project from pilot testing to full-scale implementation.
- The effectiveness of traditional vendor questionnaires and single-point assessments in modern contexts.
- The factors contributing to unauthorized AI usage within organizations.
- Specific considerations regarding AI models developed in China and emerging frontier AI models.
- The challenges that conventional cybersecurity measures may face in addressing contemporary threats in the healthcare industry.

For those interested in accessing the full webinar, a recording is available via the provided form.

Ongoing Watchfulness is Essential for Governing Healthcare AI

A bright, sunlit modern hospital control room where a determined doctor in scrubs stands alert before a large transparent touchscreen console displaying a 3D neural‐network “brain” with flowing data streams. She holds a policy manual under one arm as medical monitors and patient charts line the walls, symbolizing unwavering oversight and safety in AI‐driven healthcare governance.

Constant Vigilance is the Watchword for Healthcare AI Governance - MedCity News

- A webinar explored the Healthcare AI Readiness Index and governance issues in healthcare AI.  
- Cotiviti's role is to turn data into insights for reducing healthcare waste and fraud.  
- Constant vigilance and human oversight are essential for effective AI governance in healthcare.  

A webinar moderated by MedCity News Editor-in-Chief Arundhati Parmar delved into findings of the recently published Healthcare AI Readiness Index. Panelists discussed how best to balance AI adoption, governance, how to scale AI at healthcare institutions and best practices for doing so. Sponsored by Cotiviti, the webinar included perspectives from Cotiviti CIO Sean Warren and Thomasina Anane, Associate Vice President, Enterprise Analytics, with the Alliance of Community Health Plans.

Cotiviti turns fragmented data into actionable insights. It helps healthcare organizations reduce waste, fraud, and abuse, and works with them to simplify their operations.

“In many ways, what we’ve become is the connective tissue that helps healthcare work more efficiently across all of the payers, providers, and other stakeholders,“ Warren said.

Anane referenced a recent annual meeting of ACHP, which included a candid discussion on AI governance.

“One theme that really came through clearly is that AI governance has to start well before deployment and evolve along the use case. Many of our health plans are grappling with issues like: What is my state saying about AI use? What does that look like federally and what does that look like as a health plan?….What data are actually involved? What decision is being influenced? What happens when the output is wrong?”

The biggest challenge with AI governance in a clinical and payer setting is the need for constant vigilance, a point made by both Anane and Warren.

“Deployment is really not the finish line,” Warren said. “It’s constant follow-up, constant security checks, and we have a lot of human-in-the-loop and human oversight with observability. It’s ten times more important when we add AI to the mix to be able to know what that AI is doing. Do you have the guardrails around what the AI is doing in your environment? Do you have the ability to undo it? Do you have the ability to follow and trace every transaction?”

Additionally, the webinar discussion also touched on other issues such as:

What should leaders assess before moving an AI use case from pilot to enterprise deployment?

Are traditional vendor questionnaires and point-in-time assessments still sufficient?

What is driving shadow AI use?

Important considerations for Chinese AI models and frontier AI models.

Why might traditional cybersecurity controls be necessary, but insufficient in the new threat environment healthcare organizations face?

To access a recording of the webinar, fill in the form below:

Photo: Sandwish, Getty Images

Constant Vigilance is the Watchword for Healthcare AI Governance

In the rapidly evolving landscape of healthcare, the dilemma of whether to build or buy technology is becoming increasingly relevant, especially in the age of artificial intelligence (AI). A recent study from Define Ventures, a venture capital firm, indicates a strong lean towards acquiring technology rather than developing it internally, reflecting the fast-paced advancements in the sector.

The report highlights feedback gathered from 62 senior leaders within healthcare organizations, revealing that immediate AI implementation is a top priority for a significant majority of providers (75%) and payers (69%).

Intriguingly, payer organizations have notably increased their AI budgets, with 31% indicating substantial growth, a significant rise from just 6% reported in 2024. Providers are also expanding their budgets, with 35% anticipating increases in 2026 compared to 31% in the previous year.

With these evolving budgetary trends, there is a clear preference among both providers and payers to purchase AI technology rather than build it themselves. The survey found that 54% of payers expect to buy AI solutions in most instances, a jump from 31% in 2024. Conversely, 46% are considering developing AI tools internally in at least half of their cases.

While there's a majority inclination towards buying—71% of respondents expect to typically acquire AI technology—this figure has seen a slight decrease from 78% in 2024. Additionally, the proportion of leaders who commit to buying every time has dropped from 26% to 15% since 2024, with around 29% still favoring self-built solutions for half or more of their needs.

Despite the interest in AI technology, both providers and payers are facing significant challenges related to its integration. Notably, change management has emerged as a primary concern, with 50% of payer respondents highlighting this issue in 2026, up from just 6% in 2024. Additionally, 48% of providers expressed similar worries, a stark rise from 24% two years ago.

A staggering increase in concerns about embedding AI into existing workflows was reported among providers, soaring from 31% in 2024 to 82% in 2026. Nevertheless, a large majority of both groups remain amenable to reevaluating their workflows to incorporate AI, with 96% of providers and 92% of payers open to this transition.

The report emphasizes the tension between the desire for change and the actual management of that change, stating, "While there is a strong appetite for organizational transformation, managing that change effectively remains a crucial obstacle." This suggests that while there is faith in the transformative potential of AI solutions, the complexities of their implementation are substantial.

Return on investment (ROI) has surfaced as a pivotal factor influencing vendor selection as both providers and payers seek tangible benefits from their AI investments. A substantial 83% of payers and 65% of providers expect to see measurable ROI within one year; however, their expectations differ. Specifically, 69% of payers prioritize cost savings, while 56% of providers focus on revenue generation through AI technologies.

This pressure to validate ROI is shaping pricing strategies, with over half of payers (62%) predicting that performance-based pricing will become the leading model within the next year. Additionally, 38% anticipate an increase in enterprise Software as a Service (SaaS) models, which rely on subscription fees for cloud access.

The majority of providers (85%) also foresee enterprise SaaS models as the dominant pricing structure in the coming years, with 42% expecting performance-based pricing to take the lead.

The report concludes with a clear message: “The opportunity for partnerships is vast, and healthcare organizations must collaborate to keep pace with the rapid advancements in AI.” It encourages innovators to arrive with enterprise-ready solutions, domain expertise, and a strong focus on ROI from the outset. 

Anuja Vaidya, a seasoned healthcare journalist since 2012, covers the intersection of healthcare and technology, including innovations in AI, digital health, electronic health records, and interoperability.

Survey: Providers and payers favor purchasing AI instead of developing it in-house.

A bright, sunlit corporate boardroom scene: on one side, a group of healthcare executives in white coats and stethoscopes; on the other, suited insurance leaders. Between them sits a table piled with sealed software boxes stamped “AI Solution.” Nearby, untouched blueprints and technical drawings labeled “In-House AI” lie rolled up and gathering dust. Floor-to-ceiling windows flood the room with natural light, hinting at a clear preference to buy rather than build.

Survey: Providers, payers prefer to buy rather than build AI - TechTarget

- Majority of healthcare providers and payers prioritize buying AI technology over building internally.  
- Concerns about AI integration and change management are rising among healthcare organizations.  
- Most expect measurable ROI within a year, influencing pricing models for AI solutions.

The build-versus-buy equation is a common conundrum in healthcare. In the AI age, providers and payers are grappling with whether to purchase a piece of technology or build it internally as the technology advances rapidly, but new research shows a tilt toward buy.

A report from venture capital firm Define Ventures, which detailed surveys and conversations with 62 senior leaders across healthcare organizations, shows that AI implementation is an immediate priority for both providers (75%) and payers (69%).

In particular, there has been a jump in AI budgets among payers, with 31% saying their budgets would increase significantly, up from 6% in 2024. Providers' AI budgets have also increased, albeit more modestly, with 35% saying their budgets will increase in 2026 compared to 31% in 2024.

With these increases, providers and payers appear to favor buying new AI technology instead of building it in most cases, according to the report.

Among payers, 54% said they would buy AI technology in most cases, up from 31% in 2024. Meanwhile, 46% plan to build AI tools internally at least half of the time in 2026.

A majority (71%) expect to buy AI technology most or all the time, down slightly from 78% in 2024. Notably, the share of leaders who said they would buy the technology every time declined from 26% in 2024 to 15% in 2026. And about 29% still plan to build their tools in half or more cases this year.

However, concerns about AI integration abound among both groups.

Change management is a key integration challenge, with 50% of payer respondents citing it as a concern this year compared to only 6% in 2024. Meanwhile, 48% of provider respondents also noted it as a challenge in 2026, up from 24% in 2024.

Among providers, concerns around integrating AI into workflows soared from 31% in 2024 to 82% in 2026.

Still, both providers (96%) and payers (92%) are open to redesigning workflows around AI.

"Taken together, buyers' appetite for organizational change is high, yet change management remains a key challenge to success," the report stated. "This could mean that buyers believe in the value of these solutions and are therefore willing to invest in change management. Alternatively, the complexity of implementation cannot be underestimated, even with AI solutions intended to ease workflow burden."

As providers and payers emphasize buying their AI technologies, return on investment has emerged as the driving factor in vendor selection, the report shows.

Most payers (83%) and providers (65%) want measurable ROI within a year, but the ROI metrics differ between the groups. For instance, 69% of payers want cost savings from their AI tools, while 56% of providers want AI that drives revenue.

The pressure to demonstrate ROI is also influencing vendor pricing models. More than half of payers (62%) expect performance-based pricing to be dominant within the next year or so. Another 38% expect enterprise Software as a Service (SaaS) models, which depend on recurring subscriptions to access software in the cloud, to be the most common pricing model.

Most providers (85%) indicate that enterprise SaaS models will be the primary pricing model in one or two years, and 42% expect performance-based models to dominate.

"The window to partner is wide open, and healthcare organizations cannot keep up with the pace of change in AI on their own," the report stated. "Innovators need to come prepared: arriving enterprise-grade, bringing a new perspective alongside domain depth, and keeping a keen eye on ROI from the beginning."

Anuja Vaidya has covered the healthcare industry since 2012. She currently covers healthcare IT and innovation, including artificial intelligence, digital healthcare, EHRs and interoperability.

Survey: Providers, payers prefer to buy rather than build AI

As healthcare organizations increasingly integrate AI-based tools into their vendor offerings, a pressing question arises: who has ownership over the data, and what rights does the vendor have regarding its use? Under HIPAA, data ownership is not explicitly defined. It primarily outlines how protected health information (PHI) can be used and shared, leaving ownership issues and data usage rights to be clarified in contractual agreements. This discussion emphasizes the vital considerations healthcare entities must take into account when partnering with vendors that utilize AI technologies.

In the context of HIPAA, vendors who “create, receive, maintain, or transmit” PHI on behalf of a covered entity are labeled as "business associates." This broad definition includes health IT companies, cloud service providers, and data analytics platforms, as well as AI-powered tools that manage patient data or clinical records. Notably, even a vendor handling encrypted electronic PHI (ePHI) qualifies as a business associate, a point highlighted by the HHS’s Office for Civil Rights in December 2022 guidance regarding HIPAA and cloud computing. Therefore, simply hosting or processing PHI—irrespective of encryption—triggers HIPAA compliance requirements.

Once deemed a business associate, a vendor must sign a HIPAA-compliant Business Associate Agreement (BAA) before accessing PHI. This agreement outlines acceptable uses and disclosures of PHI, imposing direct HIPAA obligations on the vendor.

Many organizations might be surprised to find that just because a vendor has access to PHI does not mean it can use that data for AI training or product enhancement. According to HIPAA, a business associate is permitted to use or disclose PHI only as outlined in its contract or as legally required. If the BAA does not specifically allow such uses, the vendor must refrain from engaging in them, no matter the vendor's internal justifications for those activities.

The BAA may allow for limited uses of PHI for the vendor's internal management purposes and data aggregation services related to the covered entity’s operations, but these terms should be narrowly interpreted. Vendors sometimes attempt to categorize AI model training under these provisions, making it crucial for the BAA to explicitly define whether AI utilization falls within these limited categories.

Federal regulations around interoperability do not alter this dynamic; vendors cannot assume they have a blanket right to use any received data simply because it relates to interoperability. Those regulations focus on the access and sharing of ePHI for permitted activities and do not grant a new right for vendors to use PHI for AI model training purposes outside the BAA.

Organizations should be cautious not to take a vendor's generic agreement at face value, as these documents may contain broad data usage provisions that conflict with HIPAA requirements. The BAA must clearly articulate what the vendor is allowed to do with PHI, ensuring those uses do not exceed the permissions available to the covered entity under HIPAA.

De-identification of PHI presents an alternative pathway for vendors wishing to utilize healthcare data for AI development. The HIPAA Privacy Rule permits covered entities to share PHI to generate information that is not identifiable, freeing that data from HIPAA restrictions. However, proper de-identification is not straightforward; HIPAA stipulates that either a specific set of identifiers must be removed or a qualified expert must verify that the risk of re-identification is minimal. Organizations should remain vigilant regarding vendor claims of data anonymization to ensure they comply with HIPAA standards.

When establishing agreements with AI-focused vendors, healthcare organizations need to incorporate several practical measures to safeguard their data and comply with regulatory standards. 

1. Explicitly clarify whether model training using PHI is permitted or prohibited, avoiding ambiguous language.
2. Ensure that any restrictions on data use extend to any third-party vendors or subcontractors associated with the primary vendor.
3. Incorporate terms that allow for audits of the vendor's data handling practices and require clear communication about any data breaches or unauthorized use of PHI.
4. Address how PHI will be handled at the termination of the BAA, ensuring provisions for the return or destruction of data, which should comply with recognized standards.
5. Define the ownership of any outputs or derivatives generated from the use of the data to prevent disputes over intellectual property rights.

In addition to HIPAA, other regulations, such as state privacy laws and emerging AI legislation, may also apply to personal data outside of HIPAA's reach.

In summary, if the terms related to the use of PHI within an AI vendor's contract do not permit it, the vendor is prohibited from such use. For organizations navigating these intricacies, expert guidance can help refine vendor relationships and contractual practices.

Understanding HIPAA Data Ownership When Vendors Implement AI: Key Information for Healthcare Organizations

A sunlit hospital compliance office at dawn: on a polished wooden desk lies an open contract stamped “HIPAA,” overlaid with a softly glowing padlock icon. Nearby, a sleek server rack bearing a subtle AI chip emblem emits circuit-like streams that snake toward a locked medical records cabinet. In warm natural light, a lone healthcare executive’s silhouette stands surveying the scene, capturing the tension of vendor-driven AI and patient data ownership.

HIPAA Data Ownership When Your Vendor Uses AI: What Healthcare Organizations Should Know - Dentons

- Ownership of data in healthcare under HIPAA is determined by contractual agreements, not regulation.  
- AI vendors processing PHI must enter into a HIPAA-compliant Business Associate Agreement (BAA).  
- Properly de-identified data can be used for AI training without HIPAA restrictions, if verified.

As AI-enabled tools become embedded in vendor offerings across the healthcare sector, organizations must confront a critical but often overlooked question: who owns the data, and what can the vendor do with it? HIPAA does not assign ownership of data. Apart from individuals’ right to access their own protected health information (“PHI”), it governs how data may be used and disclosed, so ownership and data-use rights must be set by contract. This article highlights key guardrails under HIPAA that healthcare organizations should consider when engaging vendors whose products involve AI usage or model training.

Your AI Vendor Is A Business Associate

Under HIPAA, any vendor that “creates, receives, maintains, or transmits” PHI on behalf of a covered entity qualifies as a “business associate.” This classification is broad. It encompasses traditional health IT companies, cloud service providers, and data analytics platforms. Importantly, it also encompasses AI-enabled tools that process patient data, claims information, or clinical records. Even vendors that process only encrypted electronic PHI (“ePHI”) may meet this threshold. HHS’s Office for Civil Rights made this point in its December 2022 guidance on HIPAA and cloud computing. A cloud provider that stores ePHI is a business associate even if the data is encrypted and the provider does not hold the decryption key. The same logic applies to AI vendors. Processing or merely hosting PHI is enough to trigger HIPAA requirements. It does not matter if the vendor can “see” that data.

Once a vendor is classified as a business associate, the covered entity must enter into a HIPAA-compliant Business Associate Agreement (“BAA”) before permitting the vendor to access PHI. The BAA establishes the permitted uses and disclosures of PHI. The BAA establishes the permitted uses and disclosures of PHI and subjects the vendor to direct HIPAA obligations.

AI Training on PHI Is Not a Given

This is a point that may catch organizations off guard. Under HIPAA, a business associate “may use or disclose protected health information only as permitted or required by its business associate contract … or as required by law.” The BAA defines those contractual boundaries. If the BAA does not expressly authorize a specific use of data, the vendor cannot do it.

A business associate’s use of PHI, including AI model training, product improvement, or benchmarking, is not a presumed right that comes with access to the data. If the BAA does not expressly authorize such use, the vendor cannot engage in it, regardless of how the vendor characterizes the activity internally. “Product improvement” or “service enhancement” are not magic words that automatically unlock permission to train models on PHI.

A BAA may permit limited uses for the vendor’s own proper management and administration and for data aggregation services. HIPAA does not define “management and administration,” and the phrase has not received substantive regulatory interpretation, but the limited commentary available suggests that it should be construed narrowly. Additionally, the “data aggregation” services must be related to the “health care operations” of the covered entity, not the business associate’s own “management and administration.” Vendors sometimes try to fit model training into these buckets, so the BAA should clearly state whether those limited uses cover AI usage.

HIPAA’s interoperability rules do not change this. Vendors may argue that federal interoperability and information blocking requirements let them use whatever data they receive. They do not. Those rules address access, exchange, and use of ePHI for permitted purposes. They do not create a new right for a business associate to use PHI for its own AI training outside the BAA.

Organizations should not assume that a vendor’s standard agreement adequately addresses this issue. Standard vendor agreements may include broad, vendor-friendly data usage provisions that may conflict with HIPAA requirements. Or the agreement may simply not contemplate the restrictions that apply to PHI. The BAA must provide what the vendor may and may not do with the data. Moreover, permitted uses cannot exceed what the covered entity itself could perform under HIPAA.

De-Identification as an Alternative Pathway

The Privacy Rule permits, but does not require, a covered entity to disclose PHI to a business associate “to create information that is not individually identifiable health information… whether or not the de-identified information is to be used by the covered entity.” HIPAA’s data restrictions apply only to PHI. Data that has been properly de-identified is no longer PHI, and HIPAA’s restrictions do not apply. This may create a potential avenue for vendors that want to use healthcare data for AI training. If the data is de-identified before it enters the vendor’s data pipeline, HIPAA does not restrict that downstream use. The BAA should determine when and how PHI will be de-identified. The BAA should also prohibit the vendor and any other parties from attempting to re-identify the data or link it to other datasets.

However, de-identification under HIPAA may not be a routine exercise. HIPAA requires either (a) removal of 18 specified categories of identifiers under its “Safe Harbor” method, or (b) a formal determination by a qualified expert that the risk of re-identification is “very small.” Organizations should be cautious about vendor claims that data has been “anonymized” or “de-identified” without verifying that the process meets HIPAA’s requirements.

Practical Considerations for Vendor Agreements

When negotiating agreements with AI-enabled vendors, healthcare organizations should consider some practical steps to protect their data and satisfy their regulatory obligations.

Be explicit about whether restricted training is permitted or prohibited. The BAA should clearly permit or prohibit use of PHI for model training, product improvement, or any purpose beyond the contracted services. Do not rely on general language or vendor assurances about what is “standard.” Any secondary use of the organization’s data by the vendor should be carefully drafted in the BAA.

The BAA should clearly permit or prohibit use of PHI for model training, product improvement, or any purpose beyond the contracted services. Do not rely on general language or vendor assurances about what is “standard.” Any secondary use of the organization’s data by the vendor should be carefully drafted in the BAA. Require subcontractor coverage. HIPAA requires that business associates ensure their subcontractors agree to the same restrictions and conditions that apply to the business associate itself. The BAA should require the vendor to extend data-use restrictions to any sub-processors, cloud hosts, or other technology providers involved in delivering the services.

HIPAA requires that business associates ensure their subcontractors agree to the same restrictions and conditions that apply to the business associate itself. The BAA should require the vendor to extend data-use restrictions to any sub-processors, cloud hosts, or other technology providers involved in delivering the services. Include audit rights and data breach reporting. Organizations should reserve the right to audit the vendor’s data handling practices and AI training pipelines to verify compliance with the BAA. The agreement should also include clear data breach notification obligations, including notification of any unauthorized use of PHI for AI training or other impermissible purposes.

Organizations should reserve the right to audit the vendor’s data handling practices and AI training pipelines to verify compliance with the BAA. The agreement should also include clear data breach notification obligations, including notification of any unauthorized use of PHI for AI training or other impermissible purposes. Address data return and destruction. HIPAA requires that at termination of the BAA, the business associate must return or destroy all PHI received from or created on behalf of the covered entity. In the AI context, organizations should address AI models, training artifacts, input, output, or derived datasets that use PHI. Data return or destruction must be provided for in the agreement. Data destruction should follow applicable NIST or ISO standards.

HIPAA requires that at termination of the BAA, the business associate must return or destroy all PHI received from or created on behalf of the covered entity. In the AI context, organizations should address AI models, training artifacts, input, output, or derived datasets that use PHI. Data return or destruction must be provided for in the agreement. Data destruction should follow applicable NIST or ISO standards. Clarify ownership of outputs and derivatives. The BAA should determine which party owns AI models, usage data, input, output, derived data, or synthetic data created using the organization’s data. While the vendor retains ownership of its underlying models, the organization should understand whether its data contributed to the vendor’s intellectual property and any rights it may have to that property.

Beyond HIPAA

Other consumer data laws, such as Washington’s My Health My Data Act, comprehensive state privacy laws, the FTC Health Breach Notification Rule, and emerging state AI laws may also apply, especially to personal data outside HIPAA’s scope.

Bottom line: If use of PHI with a vendor’s AI is not allowed in the BAA, the vendor cannot do it.

Our team is available to discuss how these requirements may affect your organization’s AI vendor relationships and contracting practices.

HIPAA Data Ownership When Your Vendor Uses AI: What Healthcare Organizations Should Know

Picture yourself in a doctor's office, recounting your discomfort while your physician's gaze shifts from you to a nearby screen. That screen indicates a low-risk status, yet your body feels different. In this scenario, a crucial question arises: whose perspective should be trusted—the technology or your own lived experience?

Across the globe, healthcare facilities are rapidly integrating algorithms into patient care, utilizing everything from symptom checkers that help prioritize cases to AI tools aimed at enhancing diagnostic accuracy. The primary selling points emphasize improved efficiency, streamlined coordination, and enhanced precision in medical assessments.

However, these digital tools often overlook an important reality. They implicitly determine which sources of information are deemed credible, which symptoms warrant attention, and how patients' narratives should be interpreted.

I characterize this phenomenon as algorithmic gaslighting.

Redefining Medical Gaslighting

The term “medical gaslighting” refers to scenarios where patients’ genuine symptoms are undermined or misinterpreted by medical professionals—sometimes influenced by factors like gender, race, or chronic illness. Critics argue that the term is sometimes overused, as gaslighting suggests intentional deception, whereas many dismissive healthcare providers operate based on unconscious biases and longstanding institutional norms.

Regardless of the underlying intentions, the result is troubling: patients are left doubting their own experiences, feeling disconnected from the language necessary to articulate their conditions.

Algorithms exacerbate this dynamic. A computer program lacks intent and cannot perform deception; however, its decisions can overshadow patient accounts with a veneer of impartiality. These systems can constrain personal experiences into rigid schemas, leading individuals to question the validity of their own symptoms.

Key Issues in Algorithmic Healthcare

Firstly, there's what can be termed “data violence.” Medical data often skews towards populations that are predominantly white, male, and economically advantaged. This bias renders patients who do not fit that profile statistically invisible. For instance, symptoms like chest pain manifest differently in women than men, yet much cardiovascular research predominantly reflects male experiences.

Secondly, there's a troubling lack of transparency. While patients can challenge a physician's judgment, they find it difficult to dispute the results of a process whose inner workings are often opaque, even to the healthcare providers themselves. Take the Epic Sepsis Model, for example—a proprietary alert system used across many U.S. hospitals. Research revealed it performed significantly worse than advertised, identifying only about one-third of sepsis cases while generating frequent false alarms, all without clinicians or patients understanding the rationale behind any specific alert.

The third issue pertains to automated authority. When algorithms generate conclusions that contradict a patient's description, it can create a climate where individuals doubt their own experiences, and medical professionals feel reluctant to disregard what seems like an unbiased recommendation.

This authority can sometimes extend beyond safety. Internal documents later disclosed serious flaws in IBM’s Watson for Oncology, a tool promoted as a reliable decision-making aid. The tool often provided unsafe and incorrect treatment suggestions, having been largely trained on hypothetical scenarios rather than actual patient cases.

Lastly, there's the feedback loop effect. Patients and clinicians quickly learn which symptoms are likely to be categorized as "valid cases," leading them to adjust their descriptions accordingly. These new formulations then enter the data stream, reinforcing the limitations of the original categorizations that failed to accurately capture a patient's experience.

Over time, patients may find themselves framing their symptoms based on how they believe algorithms will interpret them rather than expressing their genuine feelings.

Understanding the Underlying Reality

Many may think these issues are mere technical glitches that superior design could remedy. Yet, the underlying analysis suggests something more systemic at play, challenging the notion that better engineering alone can resolve the complexities of algorithmic healthcare.

Algorithmic manipulation: The concealed dangers of artificial intelligence in healthcare

A dimly lit hospital room at night, a worried doctor in scrubs stands before a patient monitor whose warm amber glow illuminates streams of opaque algorithmic code wrapping around the bed’s metal rails like chains, deep shadows pooling in corners to suggest hidden manipulation and harm.

Algorithmic gaslighting: The hidden harm of AI in healthcare - TRT World

- Hospitals are rapidly implementing algorithms for patient triage and diagnosis to enhance efficiency.  
- Algorithms can prioritize certain demographics, making others statistically invisible in medical datasets.  
- Patients may conform their symptoms to fit algorithmic categories, distorting their own experiences.

Imagine describing your pain to a doctor and watching her eyes move from you to a screen. The screen says you are low risk. Your body says otherwise. Between the two, someone has to be believed.

Hospitals around the world are racing to put algorithms into the clinic, from symptom checkers that triage patients to AI tools that support diagnosis. The pitch is efficiency, coordination and better accuracy.

But these systems do something the brochures rarely mention. They quietly decide whose knowledge is credible, which symptoms matter and how a patient’s words should be interpreted.

I call this dynamic algorithmic gaslighting .

Gaslighting without a gaslighter

“Medical gaslighting” describes patients whose symptoms are dismissed or reinterpreted by clinicians, often along lines of gender, race or chronic illness.

Critics say the word is misused because gaslighting implies deliberate deception, and most dismissive doctors are not scheming. They are acting out of implicit bias, institutional routine and ingrained hierarchies of expertise.

But the harm is not defined by motive. It is defined by what happens to the patient, who is left doubting their own perceptions and cut off from the words they need to make sense of their experience.

Algorithms make this point impossible to avoid. A piece of software has no intentions. It cannot plot or deceive.

Yet it can override patient testimony with the authority of apparent neutrality. It can squeeze a lived narrative into rigid categories. And it can leave a patient feeling that their own experience must be unreliable.

Related TRT World - How ageist AI could affect the health of the elderly

That is gaslighting without a gaslighter.

The first is data violence. Medical datasets often skew towards white, male and socioeconomically advantaged populations. Patients outside that mould become statistically invisible.

Chest pain, for example, often presents differently in women than in men, yet cardiovascular data disproportionately encode male profiles.

Pulse oximeters, calibrated mostly on lighter-skinned patients, can overestimate blood oxygen in Black patients and hide dangerously low levels. If triage tools treat those readings as ground truth, the error travels downstream.

The second is opacity. You can argue with a doctor’s reasoning. You cannot argue with a score whose derivation is closed to you and, often, to your doctor too.

Consider the Epic Sepsis Model, a proprietary alert system used in hundreds of US hospitals. An external study found it performed far worse than its developer claimed, catching only about a third of sepsis cases while producing frequent false alarms. Neither clinicians nor patients could see the logic behind any single alert.

The third is automated authority. When the algorithm “says otherwise”, patients feel pressure to doubt their own account, and clinicians hesitate to override a supposedly objective recommendation.

That authority can outrun the evidence. Internal documents later showed that IBM’s Watson for Oncology, marketed worldwide as an authoritative decision-support tool, produced unsafe and incorrect treatment recommendations and had been trained substantially on hypothetical cases.

The fourth is the feedback loop. Patients and clinicians learn which descriptions register as legible “cases” and adjust how they speak. Those adjusted descriptions then become new data, entrenching the very categories that failed to capture the patient in the first place.

Over time, people stop describing their symptoms as they experience them. They describe them as the machine can hear them.

Why this is not a glitch

It is tempting to treat all this as a bug that better engineering will fix. The paper argues otherwise.

Algorithmic gaslighting: The hidden harm of AI in healthcare

In a significant move for healthcare innovation, Oracle has announced a collaborative agreement with AstraZeneca aimed at harnessing artificial intelligence (AI) and real-world data to enhance the identification of eligible patients for clinical trials. This partnership is set to leverage AstraZeneca's extensive scientific and therapeutic knowledge, combined with Oracle's robust capabilities in health data, cloud computing, AI, analytics, and clinical workflow.

Under this non-binding agreement, the two companies plan to investigate specific use cases designed to streamline the clinical trial process. By employing cutting-edge applications and AI, they aim to help healthcare teams identify potential trial candidates more efficiently, ultimately improving patient accessibility to crucial clinical trials and addressing one of the major challenges in this field—patient recruitment. Initial areas of focus may encompass oncology, cardiovascular and metabolic diseases, respiratory conditions, immunology, and rare diseases.

Pascal Soriot, CEO of AstraZeneca, emphasized the significance of this collaboration, stating, “This partnership merges leaders in healthcare and AI on a global scale to fast-track the transition from medical discovery to patient care. By fusing AstraZeneca’s exceptional scientific prowess with Oracle’s extensive data and technological tools, we can help healthcare professionals identify patients more promptly, optimize clinical trials, and facilitate ongoing evidence generation throughout a drug's lifecycle.”

Mike Sicilia, Oracle's CEO, highlighted the immense potential for AI within the healthcare sector, noting that improved data utilization can transform scientific advances into tangible patient benefits. He expressed optimism that this union of AstraZeneca's scientific strengths and Oracle's data and AI capabilities will expedite research, increase patient enrollment in clinical trials, and expedite the integration of relevant insights into clinical practices.

AstraZeneca's Chief Digital, Data, and AI Officer, Anne-Claire Gerbaldi, remarked on the pressing need to improve patient care pathways, acknowledging that many patients experience delays in diagnoses or face lengthy journeys to receive needed treatments. Gerbaldi stated that their goal is to utilize AI to link insights from scientific research, clinical trials, and routine medical care. This approach aims to generate data that enhances research, fosters innovation, and ultimately improves patient outcomes. She expressed confidence that the synergy between AstraZeneca and Oracle can lead to more effective clinical trials and ensure timely diagnoses.

Seema Verma, Oracle’s executive vice president and general manager for Health and Life Sciences, pointed out that advancements in healthcare hinge on bridging the gap between scientific knowledge and everyday clinical decisions. She conveyed that AstraZeneca's expertise, combined with Oracle's secure data infrastructure and advanced capabilities, could help healthcare systems more swiftly identify eligible patients, promote access to clinical trials, and generate supportive evidence for informed decision-making within health systems.

Looking ahead, both companies plan to explore a continuous learning model that integrates findings from clinical research and routine care to enhance future study designs, safety protocols, and overall patient care. They anticipate establishing shared governance, reusable technology components, and standardized outcome measures to evaluate effectiveness and determine if successful strategies can be applied across different diseases and geographical locations.

For further inquiries, please contact Kris Reeves at Oracle Corporate Communications via email at kris.reeves@oracle.com.

About Oracle:  
Oracle delivers comprehensive application suites coupled with secure and autonomous infrastructure within the Oracle Cloud ecosystem. To learn more about Oracle (NYSE: ORCL), visit www.oracle.com.

Trademarks:  
Oracle, Java, MySQL, and NetSuite are registered trademarks of Oracle Corporation. NetSuite pioneered the cloud computing era.

Oracle and AstraZeneca Collaborate to Enhance AI-driven Healthcare Innovations

Sunlit high-tech laboratory with warm natural light flooding through large windows. Two adult scientists in crisp white lab coats shake hands over a sleek digital tablet showing a 3D AI-generated molecular model. The tablet bears a prominent Oracle logo, while a large AstraZeneca logo is displayed on a wall panel behind them. Around them, modern lab equipment—test tubes with softly glowing solutions, pipettes, and digital readouts—hint at groundbreaking AI-driven healthcare innovation.

Oracle and AstraZeneca Partner to Advance AI-enabled Healthcare Innovation - Oracle

- Oracle and AstraZeneca collaborate using AI to enhance clinical trial patient recruitment and access.  
- The partnership aims to improve research efficiency in oncology, cardiovascular, and rare disease areas.  
- Insights from clinical research will inform future study design and patient care strategies.  

Austin, Texas and Abu Dhabi—Oct 20, 2026

Oracle today announced an agreement with AstraZeneca to explore how AI and real-world data can help clinicians identify eligible patients for clinical trials, accelerate evidence generation, and bring relevant research insights into everyday care. The collaboration is expected to combine AstraZeneca’s scientific and therapeutic expertise with Oracle’s health data, cloud, AI, analytics, and clinical workflow capabilities.

Through a non-binding agreement, the companies plan to explore selected use cases that could improve clinical-trial efficiency and success by leveraging modern applications and AI to help care teams identify eligible patients earlier and easier, resulting in greater access to clinical trials for patients in need and helping to overcome one of the largest obstacles in clinical trials - patient recruitment. Initial disease areas to be explored could include oncology; cardiovascular, renal and metabolic diseases; respiratory and immunology; and rare diseases.

Pascal Soriot, Chief Executive Officer, AstraZeneca said: “This unique collaboration brings leaders in healthcare and AI together at a global scale to accelerate the journey from the discovery of medicine to patient care. Combining AstraZeneca’s world class science with Oracle’s deep data and technology expertise, we can help clinicians identify patients sooner, optimize clinical trials, and generate evidence continuously across the lifecycle of a medicine.”

“Some of the most important opportunities for AI are in healthcare, where better use of data can help turn scientific innovation into better outcomes for patients,” noted Mike Sicilia, chief executive officer, Oracle. “By bringing AstraZeneca’s scientific expertise together with Oracle’s data, cloud, AI, and healthcare capabilities, we have an opportunity to help accelerate research, connect more patients to clinical trials, and ultimately bring relevant insights into care faster.”

Connecting Research and Care

Anne-Claire Gerbaldi, Chief Digital, Data and AI Officer, AstraZeneca, added: “Too many patients are diagnosed late or face a long journey to get the care they need. To build the next decade of medicine, we aim to harness AI to connect learnings across science, clinical trials, and everyday care, generating insights that continuously strengthen research, accelerate innovation, and enhance patient outcomes. Bringing together the unique capabilities of Oracle and AstraZeneca, we believe we can support smarter clinical trials, strengthen safety monitoring, and help ensure patients are diagnosed sooner and receive the right care.”

Seema Verma, executive vice president and general manager, Oracle Health and Life Sciences, said: “Healthcare progress depends on shortening the distance between scientific insight and the decisions clinicians make every day. AstraZeneca brings deep scientific and therapeutic expertise, and Oracle will deliver the secure data, cloud, AI, and clinical workflow capabilities. This collaboration has the potential to help health systems identify eligible patients sooner, broaden access to clinical trials, and generate evidence that supports the next decision, all under the governance of the health systems delivering care.”

AstraZeneca and Oracle intend to explore a continuous learning model in which insights from clinical research and routine care help inform future study design, safety monitoring, and patient care. The companies plan to establish shared governance, reusable technical components, and agreed outcome measures to assess results and determine whether successful approaches could extend across disease areas and geographies.

Contact Info Kris Reeves Oracle Corporate Communications

kris.reeves@oracle.com



About Oracle

Oracle offers integrated suites of applications plus secure, autonomous infrastructure in the Oracle Cloud. For more information about Oracle (NYSE: ORCL), please visit us at www.oracle.com.

Trademarks

Oracle, Java, MySQL, and NetSuite are registered trademarks of Oracle Corporation. NetSuite was the first cloud company—ushering in the new era of cloud computing.

Oracle and AstraZeneca Partner to Advance AI-enabled Healthcare Innovation

AI platforms designed to address clinical inquiries are gaining traction by helping healthcare professionals navigate extensive data and access pertinent information quickly. Evidence suggests that these technologies are indeed fulfilling their potential, with increasing adoption rates among practitioners. However, insights into which groups of clinicians are most engaged with these tools and their usage patterns have remained largely unexplored—until now.

A recent research paper published in JAMA provides clarity on this matter, revealing a significant rise in the utilization of OpenEvidence over the past two years, particularly among trainees and early-career doctors.

Anupam Jena, M.D., Ph.D., a prominent figure in healthcare policy at Harvard Medical School, highlighted the study's main findings: “A remarkable number of physicians are leveraging AI for their clinical duties, often in fields where they lack extensive expertise,” Jena noted in a press statement.

The research team from Harvard Medical School and Brigham and Women's Hospital investigated the adoption of OpenEvidence, the leading AI-based clinical information tool in the United States. They analyzed an impressive dataset comprising 271 million queries from 952,008 clinicians between January 1, 2024, and June 30, 2026, including 556,200 physicians.

Although OpenEvidence did not sponsor the research, three of the authors are employees of the company. The data utilized for the analysis was anonymized to maintain privacy.

The findings detail the demographics of OpenEvidence users, revealing a significant surge in monthly active participants—from merely 355 in January 2024 to an astonishing 703,265 by June 2026. The average number of queries per active user escalated from 18 to 39.5 over the same timeframe. Additionally, the proportion of clinicians returning to query the platform from one month to the next increased substantially from around 70% in early 2024 to 90% by 2026. Notably, nurse practitioners and physician assistants submitted a higher volume of queries per user compared to their physician counterparts. 

Experience in the field emerged as the most significant factor influencing platform usage: clinicians with fewer than ten years of experience averaged 43.4 queries per month, in contrast to 29.9 from those with two decades or more of practice. Among physicians specifically, nearly two-thirds (61.8%) of queries were generated by those within ten years of graduation from medical school. Resident and fellowship members accounted for 27.2% of the queries, while veteran physicians with over 20 years of experience contributed 16.5%. Oncologists and hematologists were the most frequent users, followed by physicians practicing internal and family medicine. Conversely, usage was less pronounced in fields focused on procedural and diagnostic work, such as ophthalmology, radiology, and orthopedics.

The popularity of AI-driven clinical intelligence is skyrocketing, but who is adopting it?

A sunlit modern hospital corridor where three doctors in white coats lean over a sleek tablet. Above its screen, glowing 3D patient vitals—heartbeat waves, neural‐network synapse maps, DNA helixes—hover in midair like floating data clusters. Warm natural light pours through tall windows, illuminating their focused expressions as they discuss the promises of AI‐powered clinical insights.

AI-based clinical intelligence is all the rage. But who is using it? - TechTarget

- The usage of the AI platform OpenEvidence among clinicians surged significantly over two years.  
- Trainees and early-career physicians constitute the majority of OpenEvidence's active users.  
- Clinical experience influences platform use; newer clinicians submit more queries than experienced ones.  

AI platforms that answer clinical questions promise to make clinicians' jobs easier by sifting through large volumes of evidence and pulling up relevant information. And it appears the technology is making good on that promise as these platforms become more popular. But which clinicians are using them most, and how, have remained open questions -- until now.

A new study published in JAMA shows that the number of clinicians using OpenEvidence increased sharply over the last two years, with trainees and early-career physicians using the platform the most.

"Two key takeaways are that an enormous number of physicians are using AI for clinical care and that they are often using it in areas where they don't have specific expertise," said the study's corresponding author Anupam Jena, M.D., Ph.D., the Joseph P. Newhouse Professor of Health Care Policy at Harvard Medical School and HMS professor of medicine at Massachusetts General Hospital, in the press release.

Researchers from Harvard Medical School and Brigham and Women's Hospital assessed clinicians' use of OpenEvidence, as it is the most widely used AI-driven clinical information platform in the country. They examined 271 million queries from 952,008 clinicians between Jan. 1, 2024, and June 30, 2026. Of the clinicians, 556,200 were physicians.

While OpenEvidence did not fund the study, three study co-authors are OpenEvidence employees. The company provided the data, but it was de-identified data for analysis.

Who is using the OpenEvidence platform? The study shows that the number of active users per month on the platform jumped from 355 in January 2024 to 703,265 in June 2026. The average number of monthly queries per active user rose from 18 to 39.5 in the same period. The share of clinicians who were active users in one month and who queried again the next rose from approximately 70% in early 2024 to 90% in 2026. Nurse practitioners and physician assistants submitted more queries per active user than physicians in one month. But clinical experience was the biggest driver of platform use. Clinicians with fewer than 10 years in practice submitted 43.4 queries per user per month compared with 29.9 among those with 20 or more years in practice. This trend could also be seen among physicians. Close to two-thirds of the queries (61.8%) were from physicians within a decade of medical school graduation, while residents and fellows submitted 27.2% of the queries and physicians with 20 or more years of experience submitted 16.5%. Oncologists and hematologists used the platform most, followed by internal medicine physicians and family medicine physicians. Use of the platform was lower in procedural and diagnostic specialties, such as ophthalmology, radiology and orthopedics.

AI-based clinical intelligence is all the rage. But who is using it?

Artificial intelligence is rapidly becoming an integral part of electronic health records (EHRs). As healthcare organizations transition from testing isolated AI solutions to broader implementation, a critical question arises: is the AI merely integrated with the EHR, or is it inherently part of the platform itself?

The distinction between standalone AI tools and native AI capabilities significantly influences clinical workflows. Separate AI applications can create challenges such as data silos, additional governance requirements, integration issues, and disruptions in workflow. In contrast, AI that is embedded within the EHR platform operates seamlessly, utilizing the same data, permissions, workflows, and clinical context as the rest of the system.

This differentiation is particularly vital as organizations scale their AI applications. Here are three key considerations for healthcare leaders:

1. AI is most effective when it accompanies the clinician through the workflow.

Clinical interactions begin well before a patient steps into the examination room. Clinicians must often sift through extensive records, including notes, medications, lab results, treatments, and past interactions, to gain a comprehensive understanding of the patient’s history. EHR-integrated AI can effectively consolidate this longitudinal data and maintain pertinent context throughout the appointment.

Before the appointment, AI tools designed to work within the EHR can generate concise, problem-focused summaries of a patient’s relevant history. Additionally, chart-related AI can provide clinicians with vital information during patient encounters by processing natural-language queries directly within the workflow.

During the appointment, ambient note-taking technologies can discreetly and accurately document the discussion between patient and provider, eliminating the need for separate applications or logins.

After the appointment, since documentation is recorded in real time, clinicians can refine the notes rather than spend time recreating them afterward.

The benefits are quantifiable. For example, one customer of athenahealth utilizing AI-native athenaOne saw a reduction of nearly six minutes in chart preparation and documentation time per visit¹. Clinicians employing this technology also achieved same-day chart completion rates surpassing 80%².

2. AI adds more value when workflows are interconnected.

The potential of AI stretches beyond simply speeding up tasks. When AI capabilities are unified on the same platform, the results of one task can enhance the next step. This interconnectedness cultivates a seamless experience throughout the clinical and operational processes:

Preparation → Documentation → Coding → Quality → Operations

More thorough documentation can lead to more precise coding, which in turn supports quality measurement and helps identify care gaps, influencing broader clinical and operational strategies.

This is where an AI-native framework distinguishes itself from an assortment of disparate solutions. The real advantage arises not only from what each tool can achieve alone but from how they collaborate across the entire healthcare experience.

3. Scaling AI requires careful architectural evaluation.

As organizations expand their AI capabilities, the underlying architecture becomes crucial for facilitating data sharing, clinical context, and workflow continuity across the institution.

An AI-native platform offers a cohesive foundation for integrating AI into both clinical and operational functions, rather than treating each capability as an isolated experience.

This foundation grows increasingly critical as AI applications extend beyond patient encounters to encompass areas such as quality management, care-event summaries, inbox management, and practice operations.

For Chief Medical Information Officers (CMIOs) and other healthcare executives, the evaluation criteria are evolving. While feature lists can outline AI capabilities, it’s the underlying architecture that reveals whether these abilities are designed to function cohesively and scale as usage expands.

Assess the architecture, not just the AI features.

When assessing EHR AI, healthcare organizations must delve into the mechanisms that drive these capabilities.

Does the AI sustain clinical context throughout various workflows? Is information generated in one phase of care utilized in subsequent phases? Can the organization broaden its AI applications without the constant need for new systems, integrations, and governance protocols?

The responses to these questions can assist leaders in differentiating between AI systems that merely automate standalone tasks and those that integrate intelligence across the entirety of the care experience. Understanding this distinction is crucial for progressing from simply enhancing AI functionalities to establishing a holistic, AI-native clinical environment.

Footnotes

1 Based on athenahealth data for Q2 2026 compared to Q2 2025 encounter documentation time; results reflect topline customer data. Individual results may vary, and not every clinician should expect the same result; M327

2 Based on athenahealth data for Q2 2026 compared to Q2 2025 same-day encounter close rates; results reflective of topline customer data and individual results may vary; M326

Three key considerations for healthcare leaders regarding AI-native EHRs

A sunlit modern hospital command center: a senior healthcare leader in a white coat stands at a large semi-transparent touchscreen wall, bathed in warm natural light. Three floating numbered panels hover before them—1 shows patient vitals morphing into an AI-driven waveform, 2 displays workflow automation charts, 3 features a data-security lock icon. The scene feels optimistic and realistic, with soft shadows and a clean, professional atmosphere.

3 things healthcare leaders should know about AI-native EHRs - Healthcare Dive

- AI-native tools integrated into EHR workflows enhance clinical context and efficiency during visits.  
- Shared platform context allows smoother patient care processes and improves documentation and coding.  
- Evaluating AI architecture is crucial for scaling and integrating capabilities across healthcare organizations.  

AI is quickly becoming standard across EHRs. But as healthcare organizations move from experimenting with individual AI tools to deploying them more broadly, the question is shifting: Is the AI simply connected to the EHR, or is it built into the platform itself?

Whether AI sits alongside the EHR or is built natively has real implications for clinical workflows. Standalone tools can introduce new data boundaries, governance processes, integrations, and workflow interruptions. AI-native capabilities can instead work from the same data, permissions, workflows, and clinical context as the rest of the platform.

That difference becomes increasingly important as organizations scale AI. Here are three things healthcare leaders should consider:

1. AI is more useful when context follows the clinician

A clinical encounter starts well before the patient enters the exam room. Clinicians may need to review years of notes, medications, labs, treatments, and previous encounters to understand the full picture. AI native to the EHR can help synthesize that longitudinal record and carry relevant context from preparation into the visit itself.

Before the visit: AI-native tools built inside the workflow can automatically generate problem-oriented summaries of relevant patient history, while other chart-related tools empower clinicians with the information needed at point-of-care by asking natural-language questions directly within the encounter workflow.

During the visit: Ambient note tools can discretely, safely, and accurately capture visit notes, proactively reflecting the patient-provider conversation without requiring a separate application or login.

After the visit: Because documentation is captured during the encounter, clinicians can review and refine the record rather than spend valuable time reconstructing the visit afterward.

The potential payoff is measurable. One athenahealth customer using AI-native athenaOne across the clinical workflow reduced chart preparation and documentation time by nearly six minutes per visit.¹ Clinicians using AI-native athenaOne® have also increased same-day chart completion to more than 80%.²

2. AI creates more value when one workflow informs the next

The opportunity extends beyond making individual tasks faster. When AI capabilities share the same platform and clinical context, the output of one workflow can inform what happens next. Shared context can create continuity across the clinical and operational journey:

Preparation → Documentation → Coding → Quality → Operations

More complete documentation, for example, can support more accurate coding. That coding can inform quality measurement and care-gap identification, which can ultimately support broader clinical and operational decisions.

This is where an AI-native approach begins to differ from a collection of point solutions. The value isn't limited to what each capability can accomplish independently; it comes from how those capabilities work together across the care experience.

3. Scaling AI means evaluating the architecture behind it

As AI adoption scales, architecture determines whether individual capabilities can share data, clinical context, and workflows across the organization.

An AI-native platform provides a common foundation for extending AI across clinical and operational use cases rather than managing each capability as a separate experience.

That foundation becomes increasingly important as AI moves beyond the encounter into areas such as quality management, care-event summaries, inbox management, and practice operations.

For CMIOs and other healthcare leaders, the evaluation criteria are shifting. A feature list can show what AI can do. Architecture can show whether those capabilities are designed to work together — and scale — as adoption grows.

Evaluate the architecture, not just the AI

As healthcare organizations evaluate EHR AI, they also need to understand what sits underneath those capabilities.

Does the AI share clinical context across workflows? Can information generated in one part of the care journey inform the next? Can the organization expand its use of AI without continually adding new systems, integrations, and governance processes?

The answers can help leaders distinguish between AI that automates individual tasks and AI designed to connect intelligence across the care experience. And that’s the difference between adding AI capabilities and building an AI-native clinical experience.

Footnotes

1 Based on athenahealth data for Q2 2026 compared to Q2 2025 encounter documentation time; results reflect topline customer data. Individual results may vary, and not every clinician should expect the same result; M327

2 Based on athenahealth data for Q2 2026 compared to Q2 2025 same day encounter close rates; results reflective of topline customer data and individual results may vary; M326

3 things healthcare leaders should know about AI-native EHRs

Healthcare operators can now engage in conversational queries about their automation processes, gaining insights into what is working, identifying potential risks, and receiving actionable recommendations tailored to their live workflows.

In a recent announcement from San Mateo, California, Notable, a frontrunner in healthcare AI aimed at boosting workforce efficiency, unveiled Flow Compass—an innovative intelligence layer within the Notable Platform. This new feature empowers healthcare systems to construct, evaluate, and implement reliable AI automations, while also providing visibility into operational results and enabling growth with governance and human oversight integrated into the process.

Faced with increasing demand, limited personnel, and soaring costs, many healthcare organizations are leaning into AI solutions to enhance operational capacity. While the initial creation of an AI agent may seem manageable, maintaining the reliability of potentially dozens or even hundreds of agents as systems and workflows evolve presents a much larger challenge. Operators must be able to ascertain where further automation can be applied, track the effectiveness of existing agents, and analyze their output. This challenge amplifies as additional workflows are introduced.

Flow Compass forms a crucial part of Notable's System of Work, which synthesizes human effort and AI capabilities across various applications to ensure tasks are completed efficiently. Unlike traditional systems that merely record what has occurred, the System of Work is designed to execute tasks effectively. With Flow Compass, healthcare operators can simply ask questions like, "How can I improve referral turnaround time?" or "What issues are affecting my appointment outreach?" and obtain clear, actionable insights relevant to their specific operational context.

The capabilities of Flow Compass enable health systems to:

- Identify and resolve bottlenecks: Evaluate workflow issues, define success metrics, and outline project timelines. Each initiative contributes to the system's operational knowledge, expediting the transition from concept to execution.
  
- Leverage established frameworks and resources: Build automations using tested templates and step-by-step instructions. Pre-launch, teams can conduct readiness assessments, formulate test scenarios, and document the ownership and intricate steps of each workflow.

- Assess effectiveness and impact: By querying aspects related to workflows, execution rates, and operational costs, healthcare providers can create dashboards that reflect both financial gains and cost reductions alongside resource usage.

- Continuously enhance workflows: Analyzing unsuccessful runs allows teams to trace issues to their source and understand the broader implications of potential changes on interconnected workflows.

Healthcare teams can optimize their automation strategies with greater control, while Notable's support remains accessible whenever it’s needed.

Pranay Kapadia, co-founder and CEO of Notable, emphasized the critical nature of operational capacity in healthcare. "The industry is not lacking in technology; it struggles with the capacity to implement it effectively. Our existing solutions already save healthcare teams over 3.2 million hours annually and drive more than $130 million in yearly savings and revenue. The challenge now is not merely about deploying more AI agents; it’s about equipping operators to manage an entire workforce of them effectively across all departments, ensuring they have the visibility, accountability, and ongoing improvement necessary to succeed. Flow Compass facilitates this vision."

Designed to address the complexities inherent in healthcare environments, Flow Compass utilizes:

- A decade of specialized knowledge: Drawing on extensive insights and proven practices, along with reusable capabilities available throughout the Notable network, preventing individual health systems from needing to tackle the same operational issues repeatedly. Notable complies with rigorous standards, including HIPAA, HITRUST CSF, SOC 2 Type II, and ISO 27001.

- Tailored data integration: Utilizing data from Notable's Connector Hub—comprising EHRs, clearinghouses, and knowledge databases—combined with the context established within the existing Notable Platform.

- Strong governance protocols: Flow Compass operates within each user's designated access levels, maintaining an audit log of actions and citations for transparency. It includes built-in safety features such as test suites and voice simulation, ensuring that only authorized personnel within the healthcare team can publish workflows or modifications.

A real-world example of Flow Compass' utility comes from NKC Health, where the team was preparing to implement a patient questionnaire process for headache assessments. Prior to launch, Flow Compass identified a critical configuration setting that ensured patient responses would be properly integrated into their medical records, thereby avoiding substantial manual effort down the line. This proactive adjustment allowed patients to fill out the questionnaire at their convenience, with their input seamlessly available during physician consultations.

Dr. Todd Beardman, CMIO at NKC Health, noted, "When developing agent-driven workflows, we prioritize patient impact. This meticulous scrutiny of our processes is essential and enables us to move faster, innovate more, and positively influence our organization."

At Beacon Health System, Digital Transformation Strategist Jen Hopper employs Flow Compass to address operational challenges collaboratively with her team, leveraging the platform to structure workflow development and testing more effectively.

"There are business challenges that teams have wrestled with for some time," Hopper remarked. "Now, when faced with a problem, I can propose a potential solution and collaborate with the agent to explore feasible options."

As Notable pioneers a new System of Work, the vision aligns people and AI agents to jointly navigate workflows, including oversight of the automations themselves. Future iterations promise agents that will actively monitor performance, flag persistent issues, and suggest enhancements before complications escalate. Operators will manage a synchronized AI workforce, prioritizing tasks, approving adjustments, and maintaining accountability. The true success metric will be not just the number of agents deployed, but the speed of patient care, the restored capacity for caregivers, and the overall efficiency of healthcare delivery.

Flow Compass is currently accessible in Flow Studio, enabling teams to create and oversee their Notable workflows effectively.

Notable stands at the forefront of transforming healthcare operations with its innovative System of Work, where human expertise converges with AI capabilities. The platform aids healthcare organizations in redesigning and automating critical areas such as patient access, operational processes, and revenue management through multimodal agents that seamlessly integrate various technological functions. With over 380 million patient interactions to date, Notable is committed to saving its partners millions of hours of operational work annually while generating significant financial benefits. The company boasts rapid deployment success, with 75% of implementations going live within three months. Through the Notable Network, organizations can accelerate their innovation by leveraging proven workflows and shared expertise. Notable receives support from prominent investors including ICONIQ Growth, Greylock Partners, F-Prime, Oak HC/FT, Maverick Ventures, and 8VC. Additional information can be found on notablehealth.com.

Notable launches Flow Compass, empowering healthcare operators to manage their AI workforce.

A bright hospital command center flooded with warm natural light. A doctor in scrubs and a nurse stand over a sleek digital console featuring a glowing compass dial labeled “Flow Compass.” On large wall screens, stylized icons of AI assistants organize patient charts and schedules. The Notable logo is boldly displayed above the workstation, conveying empowered healthcare operators confidently directing their AI workforce.

Notable introduces Flow Compass, giving healthcare operators control of their AI workforce - PR Newswire

- Notable launched Flow Compass to enhance AI automation in healthcare workflows.  
- Flow Compass enables operators to assess automation effectiveness and identify improvement opportunities.  
- Health systems can create, test, and deploy AI agents using proven templates and guidance.

Healthcare operators can now conversationally ask what's working in their automations, what's at risk, and what to do next, and get answers grounded in their own live workflows.

SAN MATEO, Calif., Oct. 19, 2026 /PRNewswire/ -- Notable, the leading healthcare AI platform for transforming workforce productivity, today announced Flow Compass, a new intelligence layer of the Notable Platform. Flow Compass helps health systems build, test and deploy dependable AI automation, see what it's doing and what it returns, and scale it with governance and human control built in.

Health systems face growing demand, constrained staffing, and rising costs, and many are turning to AI agents to add capacity. Building an agent is becoming the easy part. The harder part is keeping dozens or hundreds of agents dependable as workflows, systems and models change. Operators also need to know where to automate next, whether their agents are performing as intended, and what those agents return. That work grows with every workflow a health system adds.

Flow Compass is part of Notable's System of Work, where people and AI agents coordinate across existing applications to complete work from start to finish. While systems of record capture what happened, a System of Work gets the work done. With Flow Compass, operators simply ask, "How can I reduce turnaround time on my referrals?" or "What's causing my appointment outreach workflows to fail?" in a conversational chat. They get clear answers and next steps grounded in their own workflows.

With Flow Compass, health systems can:

Find and fix what's slowing them down: Scope a workflow's problem, success metrics and phasing. Each plan adds to the health system's operating context, so every new project moves from idea to impact faster.

Scope a workflow's problem, success metrics and phasing. Each plan adds to the health system's operating context, so every new project moves from idea to impact faster. Turn operational expertise into working agents: Build with proven templates, skills and connectors, with step-by-step guidance. Before go-live, run readiness reviews, generate test cases, and document each workflow's ownership, steps and subflows.

Build with proven templates, skills and connectors, with step-by-step guidance. Before go-live, run readiness reviews, generate test cases, and document each workflow's ownership, steps and subflows. Know what's working and what it's worth: Ask questions about flows, runs and costs, and build dashboards that show revenue generated, cost saved and utilization.

Ask questions about flows, runs and costs, and build dashboards that show revenue generated, cost saved and utilization. Make every workflow better over time: Analyze failed runs, trace a problem across complex configurations to its root cause, and see every workflow a change could affect.

Teams gain more capability and control over their automation, and Notable's experts remain available when they need them.

"Healthcare doesn't have a shortage of technology. It has a shortage of capacity to get the work done," said Pranay Kapadia, co-founder and CEO of Notable. "We've already demonstrated what is possible, returning more than 3.2 million hours annually to healthcare teams and generating over $130 million in annual revenue and cost savings for our partners. The next challenge isn't deploying another AI agent. It's giving operators the ability to run an entire workforce of them, across every department, with the visibility, accountability and continuous improvement they expect from their people. Flow Compass makes this possible."

Built for the complexity of healthcare

Unlike general-purpose AI tools, Flow Compass draws on:

A decade of healthcare automation expertise: Best practices, lessons learned and vetted skills, plus reusable capabilities shared across the Notable network, so no health system has to solve the same operational problem twice. Notable maintains certified compliance across HIPAA, HITRUST CSF, SOC 2 Type II, ISO 27001 and more.

Best practices, lessons learned and vetted skills, plus reusable capabilities shared across the Notable network, so no health system has to solve the same operational problem twice. Notable maintains certified compliance across HIPAA, HITRUST CSF, SOC 2 Type II, ISO 27001 and more. Each health system's unique context: Data from Notable's Connector Hub, including the EHR, clearinghouses and knowledge bases, paired with the flow definitions, runs, phone calls, faxes and other context already on the Notable Platform.

Data from Notable's Connector Hub, including the EHR, clearinghouses and knowledge bases, paired with the flow definitions, runs, phone calls, faxes and other context already on the Notable Platform. Governance and safety: Flow Compass works within each user's role-based access, keeps an audit trail of its actions and cites its sources. Guardrails, test suites and voice simulation are built in, and only the health system's team can publish workflows or flow changes.

Real-world impact: Finding problems before they reach patients

NKC Health was preparing to launch a workflow that sends a headache questionnaire to about 250 patients a week before their visits. During the team's pre-launch review, Flow Compass flagged a configuration detail that needed adjusting so patients' answers would be written back into the chart, preventing significant manual rework from staff. The team made the update before go-live, and now patients complete the questionnaire at home on their own time, and their answers are in the EHR when the physician sees them.

"When we build agentic workflows, we're very mindful about how they impact our patients," said Dr. Todd Beardman, CMIO at NKC Health. "That level of scrutiny of our flows is vital. It gives us confidence to move quicker, build more, and create more impact for our organization."

At Beacon Health System, Jen Hopper, Digital Transformation Strategist, uses Flow Compass to work through operational problems with her teams and understand what the platform can do. It also lays out the steps to build and test a workflow, which turns ad hoc tinkering into a structured project.

"There are business problems where people have struggled to find a solution," Hopper said. "Now I can be presented with that problem and say, 'I might have a solution for you. Let me go brainstorm with the agent and see what we can do.'"

Toward a continuously improving System of Work

Notable is building a new System of Work, where people and AI agents work side by side, including on running the automation itself. Next, agents will proactively monitor workflow performance, flag recurring failures and emerging risks, and recommend improvements before problems compound. Operators will direct a coordinated AI workforce, with people setting priorities, approving changes and staying accountable. The measure of success isn't how many agents a health system deploys. It's whether patients get care sooner, caregivers regain capacity and healthcare runs better.

Availability

Flow Compass is available now in Flow Studio, where teams can build and manage their Notable workflows.

About Notable

Notable is pioneering healthcare's new System of Work, where people and AI agents work together to transform how healthcare operates. Its AI Platform helps organizations redesign and automate patient access, care operations, revenue cycle management, and more, using multimodal agents that combine voice, language, document intelligence, and computer use across healthcare's fragmented systems. Notable has interacted with over 380 million patients. Each year it saves partners more than 3.2 million hours of work and generates over $130 million in revenue and cost savings, and 75% of deployments go live in under three months. Through the Notable Network, organizations build on proven workflows and shared expertise to accelerate innovation. Notable is backed by ICONIQ Growth, Greylock Partners, F-Prime, Oak HC/FT, Maverick Ventures, and 8VC. Learn more at notablehealth.com.

SOURCE Notable

Notable introduces Flow Compass, giving healthcare operators control of their AI workforce

Several months ago, I had the opportunity to assess a sepsis prediction model for a client, and while the model itself demonstrated a respectable Area Under the Curve (AUC) and appeared calibrated adequately, one offhand remark from an ICU nurse during a workflow discussion was particularly telling: "we don't really pay attention to that score anymore; it triggers too frequently." That singular comment revealed more about the credibility of AI in healthcare than any performance metric might. Despite the model's technical precision, it was essentially ineffective, as there had been no thoughtful integration between the model's outputs and actionable insights for clinicians.

This disconnect—between a model's predictions and the practical actions a clinician can undertake—represents the major reason many AI initiatives in healthcare falter. It's not due to inaccuracy; rather, it boils down to a lack of trust. Trust is not simply an afterthought that can be added later on; it must be carefully woven into the architecture, starting from the moment patient data is collected all the way to when a physician evaluates the information presented to them.

Let’s delve into what this architectural framework looks like in real-world settings, drawing on patterns I’ve observed across various healthcare and related data platforms. This discussion is not an abstract academic model but rather a pragmatic guide, with genuine caveats included.

Navigating the Healthcare Data Landscape

For those accustomed to developing data pipelines in e-commerce or financial technology, healthcare data may offer a humbling revelation. The truth is, healthcare data is messier than many acknowledge. A single patient's record often resides across multiple systems—like an electronic health record (EHR), a different radiology PACS, an outdated lab information system, and perhaps even a faxed PDF from another facility. Timestamp inconsistencies and varied measurement units (like mg/dL versus mmol/L) frequently trip up pipelines. Importantly, a misidentified data point in this domain can have serious consequences for patient safety.

The issue of consent also warrants more engineering focus than it currently receives. Data gathered for treatment doesn’t automatically qualify for use in training algorithms without a robust consent framework. I’ve observed teams meticulously crafting feature pipelines only to later find that a substantial portion of their training data lacked consent for secondary uses, especially for AI applications. This issue is not merely a data science oversight; it's fundamentally an architectural oversight—consent should be a primary consideration, not an afterthought added just before compliance checks.

Here’s an outline of my proposed framework from start to finish.

At the foundation, notice the feedback loop. It’s not just an aesthetic addition. Many healthcare AI frameworks mistakenly treat deployment as the final stage, when it’s actually just a checkpoint. Continuous feedback, including clinician overrides and signals of model drift, must be incorporated into governance and retraining processes. Otherwise, a model that was suitable upon launch may lose reliability unnoticed until an audit reveals discrepancies or, worse yet, leads to negative patient outcomes.

Understanding Governance and Its Role

I used to perceive governance as the bureaucratic element that bogs projects down, but that perspective was somewhat unfair. Processes like data de-identification, FHIR normalization, tracking data lineage, and binding consent aren’t merely compliance checkboxes; they form the essential infrastructure determining if your model can function in production and whether you can accurately trace its data sources months later.

FHIR (Fast Healthcare Interoperability Resources) significantly aids in this regard, as it enforces a standardized vocabulary across disparate systems that often communicate in entirely different languages. While it won’t clean up a flawed source system, FHIR provides a consistent standard to normalize against, which is crucial when attempting to replicate training processes long after data sources may have changed.

Lineage tracking deserves special attention too. In most sectors, it serves as a useful tool for troubleshooting. However, in healthcare AI, it can be viewed as a hidden legal obligation. When regulators or ethics boards inquire about which patients’ data informed a model and whether those patients consented, you need a definitive answer—not just a vague gesture of acknowledgment.

Missing the Mark on Explainability

I feel particularly strongly about the perceptions surrounding explainable AI, so I will respectfully challenge some conventional narratives. Tools like SHAP values and attention maps can offer insights by revealing which factors shaped a specific prediction. However, an explanation devoid of an understanding of the model's uncertainty can be misleading. It can foster an illusion of confidence while disguising the truth.

From a clinician's perspective, when a model indicates an "82% sepsis risk, attributed primarily to lactate and heart rate trends," this sounds authoritative. But if that 82% prediction is poorly calibrated and the model is only accurate 55% of the time at such levels, the explanation is just embellishing a flawed figure. Calibration must come hand in hand with explanations rather than being treated as a separate pursuit that often falls by the wayside.

Here’s a proposed trust pipeline laid out in sequential steps.

The abstention stage is often overlooked, and I understand the hesitation. Crafting a model that sometimes opts to say “I don't know” can be uncomfortable. Nevertheless, models that abstain in genuinely uncertain situations tend to foster more trust than those that provide overly confident output for every case. Many clinicians prefer seeing “uncertain, recommend manual review” rather than resting on an unfounded precision.

Post-Deployment Responsibilities

When a model goes live, the focus shifts from building to monitoring. Drift detection becomes critically important in healthcare, especially as clinical practices evolve. For instance, a model trained on data from before the pandemic may behave inconsistently when updated care pathways for respiratory illness are instituted. Changes in population dynamics, hospital admission protocols, or even variations introduced by a new vendor can subtly compromise a model’s validity, going unnoticed until an apparent failure occurs.

Here, immutable audit trails become essential. I refer to genuinely tamper-proof records, not simple logs that can be altered at will. When issues arise (and they inevitably will), the ability to accurately reconstruct what data the model processed, the predictions it made, the interpretations generated, and the decisions rendered by clinicians will differentiate a thorough incident analysis from a potential legal predicament.

The Compliance Aspect Must Be Integrated

Previously mentioned governance shouldn’t be an afterthought, and the same goes for compliance. Regulations such as HIPAA and GDPR govern the data, while frameworks like the FDA's Software as a Medical Device guidelines and the EU AI Act’s high-risk category rules apply to the models once they begin influencing clinical decisions. These considerations should not be seen as sequential hurdles to leap over; rather, they run concurrently at every stage.

I’ve witnessed teams mistakenly consider FDA submissions or internal ethics reviews as final barriers before a launch. This approach is backward and costly. Retrofitting compliance documentation after the architectural choices are made can prove challenging, and sometimes it's nearly impossible without significant revalidation.

Recognizing the Limitations of This Framework

It’s vital to clarify that while a well-structured pipeline reduces risks, it cannot entirely eradicate them. Embedded biases in historical clinical data (such as dosing recommendations based on non-representative populations or pulse oximetry measuring inaccurately on people with darker skin tones) won’t be resolved solely through enhanced MLOps tools. Such issues demand genuine clinical and epidemiological knowledge integrated with engineering teams from the start.

Moreover, there’s a practical aspect to consider: smaller healthcare organizations and startups might not have the resources to support an expansive governance and explainability framework like I've outlined. In such cases, it’s preferable to be transparent about the compromises made rather than pretending they don’t exist. Recognizing and documenting gaps can be safer than leaving them unaddressed.

Final Thoughts

The challenge of creating trustworthy AI in healthcare is not primarily about refining algorithms anymore; rather, it hinges on the ecosystems surrounding them. While the modeling techniques are sufficiently advanced for many clinical scenarios, the supporting architecture—the governance that regulates data use, the explainability that makes predictions actionable, and the monitoring that identifies issues before they affect patients—remains in its early stages in many organizations.

Above all, I want to emphasize the importance of the feedback loop as much as the forward data pipeline. Developing the infrastructure to channel data into a model is only half the battle. The real challenge lies in creating mechanisms that track outcomes, clinician responses, and drift signals to ensure systems remain trustworthy not just at launch but throughout their lifecycle.

Could you please provide the news headline you’d like an image prompt for?

Engineering Trustworthy AI for Healthcare: From Medical Data to Clinical Intelligence - HackerNoon

- Many healthcare AI models lack trust due to poor integration between predictions and clinical decisions.  
- Governance, consent, and lineage tracking are crucial for lawful and effective AI deployment in healthcare.  
- Continuous monitoring and feedback loops are essential to maintain AI system reliability and safety.  

A few months back I was reviewing a sepsis prediction model for a client, and the model itself was fine. Decent AUC, reasonable calibration on paper. What stopped me was a throwaway comment from one of the ICU nurses during a workflow walkthrough: "we don't really look at that score anymore, it fires too often." That single sentence told me more about the state of trustworthy AI in healthcare than any benchmark table could. The model was technically accurate and practically useless, because nobody had engineered the layer between prediction and decision.

That gap, between what a model outputs and what a clinician can actually act on, is where most healthcare AI projects quietly fail. Not in accuracy. In trust. And trust, it turns out, is not a soft add-on you sprinkle at the end. It has to be architected in from the moment a patient's data is captured, all the way through to the moment a doctor decides whether to believe what the screen is telling them.

I want to walk through what that architecture actually looks like in practice, based on patterns I've seen across a few healthcare and health-adjacent data platforms. This isn't a theoretical framework pulled from a whitepaper. It's closer to a field guide, with the caveats left in.

Why healthcare data breaks the usual playbook

If you've built data pipelines for e-commerce or fintech, healthcare will humble you a little. The data is messier than people admit. A single patient record might be split across an EHR system, a separate radiology PACS, a lab information system that hasn't been upgraded since 2011, and a scanned PDF someone faxed over from another hospital. Timestamps drift. Units of measurement aren't standardized (mg/dL versus mmol/L trips up more pipelines than it should). And unlike a dropped shopping cart, a mislabeled data point here can genuinely hurt someone.

There's also the consent problem, which honestly doesn't get enough engineering attention. Data collected for treatment isn't automatically usable for training a model, at least not without a defensible chain of consent and purpose limitation. I've seen teams build beautiful feature pipelines only to discover, late in the project, that half their training cohort's consent didn't cover secondary use for AI development. That's not a data science mistake. It's an architecture mistake, because consent should be a first-class citizen in the pipeline, not an afterthought bolted on before a compliance review.

Here's roughly how I think the whole thing should be laid out, end to end.

Notice the feedback loop at the bottom. That's not decorative. A surprising number of the healthcare AI architectures I've reviewed treat deployment as the finish line, when it's really closer to a checkpoint. Outcomes, clinician overrides, and drift signals all need to flow back into governance and retraining, otherwise the model you shipped in January is quietly decaying by June and nobody notices until an audit or, worse, a bad outcome.

The governance layer isn't bureaucracy, it's plumbing

I'll admit I used to think of governance as the part of the project that slows everything down. Somewhat unfairly, in hindsight. De-identification, FHIR normalization, lineage tracking, consent binding, these aren't checkboxes for legal to tick. They're the plumbing that determines whether your model is even allowed to exist in production, and whether you can explain, six months later, exactly which records fed which version of the model.

FHIR (Fast Healthcare Interoperability Resources) helps here more than people expect, mainly because it forces a common vocabulary across systems that otherwise speak entirely different dialects. It won't fix a dirty source system, but it gives you a consistent target to normalize toward, which matters enormously once you're trying to reproduce a training run from eighteen months ago and half your source tables have changed schema since.

Lineage tracking deserves a special mention too. In most industries, lineage is a nice-to-have for debugging. In healthcare AI, it's closer to a legal requirement in disguise. If a regulator or an ethics board asks "which patients' data trained this model, and did they consent to this use," you need an answer that isn't a shrug.

Explainability is necessary, but it's not sufficient on its own

This is probably the part I feel most strongly about, and where I'll gently push back on how the industry talks about explainable AI. SHAP values and attention maps are genuinely useful. They tell a clinician which features pushed a prediction in a given direction. But an explanation without a sense of the model's uncertainty is, if anything, a bit dangerous. It gives false confidence dressed up as transparency.

Think about it from the clinician's side. If a model says "82% sepsis risk, driven mainly by lactate and heart rate trend," that sounds authoritative. But if that 82% is poorly calibrated, if the model is actually right only 55% of the time when it says 80%+, then the explanation is just decorating a bad number with plausible-sounding reasons. Calibration has to come before or alongside explanation, not as a separate research topic nobody gets around to.

Here's the trust pipeline I'd argue for, stage by stage.

The abstention step is the one most teams skip, and I understand why. It's uncomfortable to build a model that sometimes says "I don't know." Product managers don't love it. But a model that abstains on genuinely ambiguous cases is more trustworthy, not less, than one that forces a confident-sounding number on every input. Clinicians I've talked to actually prefer this. They'd rather see "uncertain, recommend manual review" than a false sense of precision.

Deployment doesn't end the responsibility, it starts a new one

Once a model is live, the job shifts from building to watching. Drift monitoring matters more in healthcare than almost anywhere else, partly because clinical practice itself shifts. A model trained on pre-pandemic respiratory data behaves differently once care pathways for respiratory illness change. Population drift, changes in hospital admission criteria, even a new lab equipment vendor changing measurement precision slightly, all of these can silently erode a model's validity without triggering an obvious error.

Audit trails matter here too, and I mean genuinely immutable ones, not a log table someone could theoretically edit. When something goes wrong (and eventually something will), the ability to reconstruct exactly what the model saw, what it predicted, what the explanation layer surfaced, and what the clinician decided, is the difference between a defensible incident review and a legal nightmare.

The compliance layer runs alongside everything, not after it

I mentioned earlier that governance shouldn't be an afterthought, and the same is true of regulatory compliance more broadly. HIPAA and GDPR govern the data. FDA's Software as a Medical Device framework, or the EU AI Act's high-risk category rules, govern the model itself once it starts influencing clinical decisions. These aren't sequential gates you pass through once. They run in parallel with every stage of the pipeline.

I've watched teams treat the FDA submission, or an internal ethics board review, as a final hurdle right before launch. That's backwards, and it's expensive. Retrofitting a risk management file after the model architecture is already locked in is painful, and honestly sometimes it's not even possible without redoing significant chunks of the validation work.

Where this framework has limits

I don't want to oversell this. A well-architected pipeline reduces risk, it doesn't eliminate it. Bias baked into historical clinical data (and there's plenty of it, in dosing guidelines calibrated on non-representative populations, in pulse oximetry that reads less accurately on darker skin) won't be fixed by better MLOps tooling alone. That requires actual clinical and epidemiological expertise sitting alongside the engineering team, not bolted on after the fact.

There's also a resourcing reality worth naming honestly. Smaller hospital systems and startups don't always have the budget for a full governance and explainability stack like the one I've sketched here. In those cases, the right move is usually to be explicit about which corners are being cut and why, rather than pretending the shortcuts don't exist. A documented, honest gap is far safer than an undocumented one.

Where I land on this

Trustworthy healthcare AI isn't really an algorithm problem anymore, or at least not primarily. The modeling techniques are mature enough for most clinical use cases. What's still immature, in a lot of organizations, is the architecture around the model: the governance that decides what data can be used, the explainability that makes a prediction actionable rather than just impressive, and the monitoring that catches decay before it becomes patient harm.

If there's one thing I'd want a reader to take from this, it's that the feedback loop matters as much as the forward pipeline. Building the pipeline that gets data into a model is the easier half of the job. Building the one that brings outcomes, overrides, and drift signals back into governance is what actually keeps a system trustworthy over time, not just on the day it launches.

During a recent technology conclave in Varanasi, the Secretary of the Ministry of Electronics and Information Technology (MeitY), S. Krishnan, emphasized the transformative potential of artificial intelligence (AI) in healthcare. He noted that AI could enhance disease screening, medical diagnostics, and remote patient monitoring, ultimately making healthcare more approachable and affordable.

The event, themed “AI for Health & Assistive Technologies,” was organized by the Software Technology Parks of India (STPI) in partnership with the Centre for Innovation, Incubation and Entrepreneurship (CIIE) at IIT BHU. It brought together a diverse group of stakeholders, including government officials, healthcare practitioners, researchers, and startups, to delve into the practical applications of AI in the healthcare sector.

In his keynote address, Krishnan underscored how AI can bolster clinical decision-making processes and elevate healthcare service delivery, particularly for underserved communities with limited access to medical resources. He discussed various applications, including disease screening, medical diagnostics, remote monitoring in intensive care units (ICUs), and AI-assisted surgical planning. He also advocated for the inclusion of AI education across various fields and the need for enhanced collaboration among technology sectors, medical researchers, and startups to develop large-scale solutions.

Additionally, Krishnan pointed out the supportive frameworks available through the IndiaAI Mission and other initiatives under MeitY aimed at fostering innovation within the healthcare landscape.

Arvind Kumar, Director General of STPI, reiterated the importance of leveraging India's existing capabilities in information technology and IT-enabled services to propel domestic AI innovation and encourage entrepreneurship. He highlighted STPI's extensive network of 27 Centres of Entrepreneurship, emphasizing the critical roles of translational research, intellectual property generation, and market access for startups to successfully bring their innovative concepts to market. Kumar also stressed the necessity of bolstering startup ecosystems outside traditional technology centers through synergistic collaborations involving academia, industry, and government.

The conclave featured dynamic panel discussions focusing on how AI can empower citizens in healthcare, promote inclusivity and well-being, and transition AI research into practical, scalable healthcare and assistive technology solutions. Esteemed experts across academia, healthcare, government, and startups participated in these discussions to explore how technological advancements could effectively address social challenges.

IIT BHU Director Amit Patra highlighted the institution's commitment to enhancing its startup ecosystem via CIIE, fostering entrepreneurial mindset among students and faculty, connecting innovators with funding opportunities, and creating a sustainable pipeline for upcoming startups.

MeitY-supported conference examines AI uses in healthcare and assistive technologies

A bright conference hall bathed in warm natural light, a large banner on stage reading “MeitY Conclave on AI in Healthcare & Assistive Technologies,” an Indian flag standing beside it. On the floor, a sleek medical-assistant robot hands a smart medicine tray to a doctor consulting an elderly patient using a sensor-equipped cane. Business-attired delegates mingle nearby, while screens behind the stage display heart-rate and mobility-aid icons.

MeitY-backed conclave explores AI applications in healthcare, assistive technologies - Livemint

- AI has the potential to enhance disease screening, diagnostics, and remote patient monitoring.  
- The conclave explored practical AI applications in healthcare and highlighted collaboration's importance.  
- STPI emphasized promoting indigenous AI innovation and strengthening startup ecosystems beyond tech hubs.  

New Delhi [India], October 18 (ANI): Artificial intelligence could improve disease screening, medical diagnostics and remote patient monitoring while making healthcare more accessible and affordable, Secretary of the Ministry of Electronics and Information Technology (MeitY) S. Krishnan said at a technology conclave in Varanasi, highlighting the potential of AI-driven solutions to address healthcare and accessibility needs.

The conclave on “AI for Health & Assistive Technologies” was organised by Software Technology Parks of India (STPI), under MeitY, and the Centre for Innovation, Incubation and Entrepreneurship (CIIE) at IIT BHU, bringing together government representatives, healthcare professionals, researchers and startups to explore practical applications of artificial intelligence.

Delivering the keynote address, Krishnan emphasised the role of AI in strengthening clinical decision-making and improving healthcare service delivery, particularly for communities with limited access to medical facilities.

He highlighted potential applications ranging from disease screening and medical diagnostics to remote intensive care unit (ICU) monitoring and AI-assisted surgical planning. He also stressed the need to integrate AI education across disciplines and strengthen collaboration between technology institutions, medical sciences, researchers and startups to develop scalable solutions.

Krishnan further highlighted support available through the IndiaAI Mission and other MeitY initiatives to encourage innovation in the sector.

STPI Director General Arvind Kumar called for building on India's experience in developing its information technology and IT-enabled services industry to promote indigenous AI innovation and entrepreneurship.

Highlighting STPI's network of 27 Centres of Entrepreneurship, Kumar underlined the importance of translational research, intellectual property creation and market access in helping startups take innovative ideas to commercial applications. He also emphasised the need to strengthen startup ecosystems beyond established technology hubs through collaboration between academic institutions, industry and government.

The conclave featured panel discussions on empowering citizens through AI in healthcare, inclusion and well-being, and translating AI research into scalable healthcare and assistive technology solutions.

The discussions brought together experts from academia, healthcare institutions, government and startups to examine how research and technological innovation could be converted into practical solutions addressing societal needs.

IIT BHU Director Amit Patra highlighted the institution's efforts to strengthen its startup ecosystem through CIIE, with a focus on nurturing entrepreneurial thinking among students and faculty, connecting innovators with funding sources and building a sustainable pipeline of startups.

MeitY-backed conclave explores AI applications in healthcare, assistive technologies

Healthcare is undergoing a significant transformation. State-of-the-art AI-driven conversational technologies are fundamentally changing the ways patients engage with care, how clinicians record their interactions, and the operations of healthcare systems on a large scale. By 2024, the global market for conversational AI in healthcare hit an impressive $13.7 billion, with an expected compound annual growth rate (CAGR) of 25.7 percent. Despite this rapid growth, only 19 percent of medical practices have adopted conversational AI for patient interactions.

This article delves into successful implementations of these technologies in the Europe, Middle East, and Africa (EMEA) region, discusses the safety challenges that the sector faces, and highlights how Amazon Connect Health, a product of Amazon Web Services (AWS), is introducing specialized AI solutions tailored for healthcare.

What's fueling the push in EMEA

Three key trends are driving urgent demand for AI in healthcare, particularly in Europe and the Middle East:

The healthcare workforce shortage has become a pressing issue. Analysts predict that by 2030, there will be a global shortfall of 10 million healthcare professionals. Clinicians in Europe allocate twice as much time to administrative duties as they do to patient care, with documentation tasks consuming 57 percent of their work hours. This situation necessitates a fundamental reassessment of how administrative responsibilities are managed. Additionally, patients are growing frustrated with subpar care navigation. A significant 89 percent of individuals transitioning between providers cite challenges related to scheduling, long wait times, and difficulty accessing care as their main reasons for their decisions. In the UK, NHS waiting lists are at unprecedented levels. Furthermore, the construction of new healthcare facilities in the Middle East, backed by a $25 billion investment in over 100 Gulf Cooperation Council (GCC) hospitals, requires a robust digital patient engagement strategy from day one. To drive adoption, various policy initiatives are being rolled out, such as NHS England’s expectation for Integrated Care Boards (ICBs) to implement ambient voice technology by April 2026. Other legislative acts, including Germany's Hospital Future Act and Saudi Arabia's Vision 2030, are paving the way for AI-based solutions, with Amazon Web Services recently recognized by Forrester as a leader in AI infrastructure.

Key components of healthcare-oriented conversational AI

Conversational AI tailored for healthcare encompasses four primary areas: patient engagement, ambient clinical documentation, medical coding, and virtual care. Insights from current implementations highlight the following:

1. Enhancing patient engagement and automating contact centers

Patients typically wait an average of 18 minutes on hold just to schedule routine appointments, and 30 percent of AI-driven healthcare conversations occur outside regular business hours. According to Boston Consulting Group (BCG), transforming patient access centers can yield profit improvements 3 to 5 times greater than simple cost reductions.

Examples from both public and private sectors in Europe and the United States illustrate the power of conversational AI in managing patient interactions:

- In Portugal, the National Health Service (SNS24), with assistance from AWS partner UpHill Health, deployed AI-driven triage systems. This initiative has allowed for the management of over 40,000 calls monthly, matching nurse evaluations over 83 percent of the time, with more than 80 percent of cases requiring no follow-up consultations. 

- At UC San Diego Health, the introduction of Amazon Connect Health for patient verification led to a 25 percent decrease in call handling time, freeing up 630 staff hours weekly for direct patient care and decreasing call abandonment rates by 30 percent.

- NHS Midlands and Lancashire utilized Amazon Connect and Amazon Lex chat assistants for managing waiting lists, resulting in a 9 to 14 percent decline in patient wait times, an 80 percent contact rate with over 2 million patients, and 67 percent of engagements conducted through automated calls.

2. Ambient clinical documentation

This facet represents one of the first significant advancements in healthcare AI. By mid-2025, almost two-thirds of U.S. hospitals using Epic had incorporated ambient AI documentation, with similar advancements now emerging in Europe.

One notable initiative is NHS England Midlands' procurement of ambient AI documentation tools from Heidi Health, deemed the largest of its kind in NHS history. This project aims to serve 70,000 clinicians across 15 trusts and 1,239 general practitioner (GP) practices. Initial trials at Dudley Group NHS Foundation Trust resulted in an 80 percent decrease in emergency care documentation time and the resolution of a backlog in rheumatology letters from six months to just 14 days.

3. Automation in medical coding and revenue cycle management

Traditional coding processes can be time-consuming, often taking several days. With AI, coding can be performed concurrently as clinical notes are created, yielding accurate International Classification of Diseases (ICD-10) and Current Procedural Terminology (CPT) codes with confidence scores and traceability.

Companies like Veradigm are integrating ambient documentation with electronic health records (EHRs) and expanding into coding and patient insights. Similarly, Greenway Health is embedding AI-centric capabilities such as scheduling and coding directly into clinical workflows.

4. Virtual care and a digital front door

The Ask Heidi assistant from Heidi Health caters to clinicians across 200 specialties and 110 languages. In a trial among UK GPs, 47 general practitioners found that documentation time decreased by 51 percent, and after-hours administrative tasks dropped by 61 percent.

Transition from scripted chatbots to sophisticated agentic AI

2023 marks a pivotal year in healthcare AI with the advent of agentic AI, moving beyond the simple decision trees used by earlier systems. Today’s agentic AI can:

- Integrate and reason across multiple data sources, including EHRs, scheduling platforms, insurance data, and comprehensive patient records.

- Perform multistage workflows autonomously, from verifying patient identity and confirming insurance to scheduling appointments and sending confirmations.

- Seamlessly escalate complex issues to human staff when necessary, while retaining thorough context for better resolution.

As institutions embrace autonomous agents to handle routine tasks, healthcare professionals can concentrate on areas that demand empathy and expert judgment.

Challenges in safety and time recovery

Although the progress is encouraging, healthcare organizations must acknowledge the challenges ahead.

The safety paradox

Research from the University of Lynchburg indicates that ambient AI scribe technologies average 13.9 transcription errors per case, with some potentially leading to significant harm. While AI-generated notes might be easy to read, errors could linger undetected, creating a new verification challenge.

A Harvard Business School case study on Mass General Brigham highlighted this dilemma: a pilot program using AI-enhanced documentation saw promising results in reducing clinician burnout and improving efficiency, but fears regarding AI errors have hindered broader application. AI tools lacking transparency may struggle to gain the requisite trust.

The time recovery dilemma

An in-depth study by Sam Houston State University’s Institute for Homeland Security found that while ambient AI implementations can reclaim 1 to 3 hours of documentation time daily, the long-term effects on clinician burnout are unclear. If saved time gets consumed by increased patient loads, the overall benefit may be minimal. On the other hand, a multicenter study reported that burnout rates fell from 51.9 percent to 38.8 percent within 30 days of using these tools, and after-hours EHR engagement decreased by 23.2 percent.

Keys to successful implementation

Current deployments indicate several factors that distinguish successful initiatives from those that falter:

- Initiate with straightforward, high-volume processes such as appointment scheduling and patient verification. Once confidence is established, you can venture into ambient documentation and coding.

- Emphasize robust integration. Standalone chat assistants often lack significant impact. Amazon Connect Health effectively integrates with systems like Epic and Oracle Health (formerly Cerner), streamlining development efforts.

- Focus on meaningful metrics. Evaluate important statistics such as the percentage of inquiries handled autonomously, call deflection rates, wait time reductions, documentation time savings, and clinician adoption rates.

- Adopt a phased approach: 
  - Phase 1 focuses on efficiency, emphasizing documentation time reduction and automation. 
  - Phase 2 addresses safety, promoting traceability and review workflows 
  - Phase 3 prioritizes wellness, ensuring that recovered time contributes to lower burnout rates and improves overall staff wellbeing.

Critical questions for healthcare leaders contemplating conversational AI

Before selecting vendors for conversational AI solutions, leaders should scrutinize their preparedness:

- What is your most frequent, least complicated patient interaction? Begin there to demonstrate value before scaling up.

- How well are your systems integrated? Without EHR integration, conversational AI often resembles a basic chat assistant.

- What will you do with the time you regain? If this time is absorbed by increased patient volume, it may not effectively alleviate burnout.

- Who will govern AI initiatives? Lack of clear ownership can stall progress due to compliance hurdles.

- Are you measuring the right outcomes? Track not just efficiency metrics, but also clinician wellbeing and clinical safety.

Envision a scenario where a patient calls their healthcare provider at 10 PM on a Sunday. Instead of reaching a voicemail, an AI agent confirms their identity, verifies insurance details, finds the next available appointment, books it, and sends a confirmation—all in under two minutes. This is not a vision for the future but a reality already experienced by patients at facilities like UC San Diego Health and NHS Midlands.

Looking ahead

Future developments in conversational AI within healthcare will likely steer systems away from single assistants towards multi-agent coordination, where teams of AI agents continuously share patient information. Voice interaction is anticipated to become the standard, especially in multilingual markets like EMEA and mobile-focused regions in the Middle East and Africa.

Regulatory frameworks such as the EU AI Act and various national initiatives will clarify guidelines and expedite adoption processes. Additionally, ambient AI is expected to evolve from documentation assistance to offering real-time clinical insights and evidence-based treatment recommendations.

Getting started

For provider organizations, AWS offers seamless integration of patient engagement functionalities within Amazon Connect Customer, enabling deployment with real-time EHR access in a matter of days. For those building healthcare technology, point-of-care capabilities can be accessed through a unified software development kit, with a sample application available on GitHub.

The ultimate goal of healthcare innovation is not to replace human connections, but to alleviate the administrative burdens that hinder them. Agentic AI in conversational contexts is making significant strides toward achieving that vision now.

The growth of conversational AI in the healthcare sector

A sunlit modern clinic room with warm natural light: a doctor in a white coat and an adult patient seated at a round table, beside them a sleek tablet on a stand showing a conversational AI chat interface with floating medical icons (stethoscope, heart rate) above it. In the background, anatomical charts on the wall and a potted plant by the window. Photorealistic, warm tones, storytelling composition.

The rise of conversational AI in healthcare - Amazon Web Services (AWS)

- The global conversational AI in healthcare market is growing rapidly, reaching $13.7 billion.  
- Clinician shortages and administrative burdens drive urgent demand for AI solutions in healthcare.  
- Successful AI implementations focus on integrating systems and prioritizing high-volume, low-complexity workflows.  

Healthcare is at an inflection point. A new wave of AI-powered conversational systems is reshaping how patients access care, how clinicians document encounters, and how health systems operate at scale. The global conversational AI in healthcare market reached $13.7 billion in 2024 and is growing at a 25.7 percent compound annual growth rate (CAGR), yet only 19 percent of medical practices use conversational AI for patient communication.

This post examines what’s working in production with a focus on Europe, the Middle East, and Africa (EMEA), the safety challenges the industry must confront, and how Amazon Connect Health by Amazon Web Services (AWS) brings purpose-built agentic AI to healthcare.

What’s driving urgency in EMEA

Three structural forces are creating immediate demand, particularly in European and Middle Eastern healthcare systems:

The clinician shortage is no longer theoretical. Analysts project the global gap will reach 10 million by 2030. In Europe, clinicians spend twice as much time on administrative tasks as they do with patients. Fifty-seven percent of clinician time is consumed by documentation. This requires fundamentally rethinking how administrative work gets done. Patients are moving away from poor care navigation practices. Eighty-nine percent of patients cite care navigation challenges such as scheduling difficulties, long wait times, and access barriers as their primary reason for switching providers. In the United Kingdom (UK), National Health Service (NHS) waiting lists remain at historically high levels. In the Middle East, rapid hospital construction in the form of $25 billion poured into over 100 new Gulf Cooperation Council (GCC) facilities demands digital-first patient engagement from day one. Policy mandates are accelerating adoption. NHS England has set an expectation that Integrated Care Boards (ICBs) and providers will adopt ambient voice technology from April 2026. Germany’s Hospital Future Act, France’s Ma Santé 2022, Saudi Arabia’s Vision 2030, and the European Health Data Space Regulation (EHDS) are all creating procurement pathways for AI-powered solutions. Forrester recognized AWS as a Leader in the Forrester Wave: AI Infrastructure Solutions in Q4 2025, giving it the highest score in Strategy across 13 providers and reflecting the infrastructure readiness healthcare organizations need.

The four pillars of healthcare-focused conversational AI

Healthcare-focused conversational AI spans four capability areas: patient engagement, ambient clinical documentation, medical coding, and virtual care. Here’s what production deployments reveal.

1. Patient engagement and contact center automation

Patients spend an average of 18 minutes on hold for routine scheduling, and 30 percent of healthcare AI conversations arrive outside office hours. Analysis from Boston Consulting Group (BCG) suggests that the patient access center can do more than cut costs: When it’s redesigned to improve access and keep patients, the gain in profit margin can be 3–5 times larger than the benefits gained from cost reduction alone.

The following examples show how public and private healthcare organizations in Europe and the United States (US) are already using conversational AI to handle patient calls, triage, and waiting lists at scale:

Portugal’s National Health Service phone line (SNS24) –AWS partner UpHill Health deployed agentic AI triage on a Portuguese national health phone line, SNS24, which serves over 10 million citizens. It autonomously handles more than 40,000 triage calls with over 83 percent agreement with nurse assessments and more than 80 percent of episodes requiring no further clinical contact.

–AWS partner UpHill Health deployed agentic AI triage on a Portuguese national health phone line, SNS24, which serves over 10 million citizens. It autonomously handles more than 40,000 triage calls with over 83 percent agreement with nurse assessments and more than 80 percent of episodes requiring no further clinical contact. UC San Diego Health –After deploying Amazon Connect Health for patient verification, the health system achieved a 25 percent reduction in call handling time, redirected 630 staff hours per week to direct patient assistance, and reduced call abandonment rates by 30 percent.

–After deploying Amazon Connect Health for patient verification, the health system achieved a 25 percent reduction in call handling time, redirected 630 staff hours per week to direct patient assistance, and reduced call abandonment rates by 30 percent. NHS Midlands and Lancashire – These NHS systems deployed Amazon Connect Customer and Amazon Lex chat assistants for waiting list management, achieving a 9–14 percent reduction in patient waiting lists, an 80 percent contact rate across over 2 million patients, and 67 percent of patients served through automated calls.

2. Ambient clinical documentation

This category has become the first major healthcare AI race to reach scale. Nearly two-thirds of US hospitals running Epic had deployed an ambient AI documentation tool by mid-2025, covering roughly 1,744 hospitals. Europe is now matching this pace.

In July 2026, NHS England Midlands launched what it described as the largest ambient AI clinical documentation procurement in NHS history. NHS England Midlands selected Heidi Health as its sole supplier, covering 70,000 clinicians across 15 trusts, all Midlands ICBs, and 1,239 general practitioner (GP) practices. Pilots at Dudley Group NHS Foundation Trust delivered 80 percent reduction in emergency care documentation time and cut a 6-month rheumatology letter backlog to 14 days.

3. Medical coding and revenue cycle automation

Traditional coding takes days. Agentic AI generates International Classification of Diseases, 10th Revision (ICD-10) and Current Procedural Terminology (CPT) codes from clinical notes by the time each visit ends, with confidence scores and source traceability.

Veradigm has pre-integrated ambient documentation into its electronic health record (EHR) and is expanding to coding and patient insights. Greenway Health is embedding AI-powered scheduling, insights, and coding directly into clinical workflows.

4. Digital front door and virtual care

Heidi Health’s Ask Heidi assistant supports clinicians across more than 200 specialties and in over 110 languages. In a UK GP trial, 47 GPs across more than 2,800 consultations saw documentation time reduce by 51 percent and after-hours admin drop by 61 percent.

The leap from rule-based chat assistants to agentic AI

The most significant shift in healthcare AI this year is the emergence of agentic AI. Building on earlier chat assistant approaches that followed scripted decision trees, agentic AI systems can:

Reason across multiple data sources –Pulling from EHRs, scheduling systems, insurance databases, and patient histories simultaneously

–Pulling from EHRs, scheduling systems, insurance databases, and patient histories simultaneously Execute multistep workflows autonomously –Handling end-to-end processes such as verifying identity, checking insurance, finding open slots, booking the appointment, and sending confirmation

–Handling end-to-end processes such as verifying identity, checking insurance, finding open slots, booking the appointment, and sending confirmation Escalate intelligently – Recognizing when a situation requires human judgment and transferring seamlessly to staff with thorough context

Healthcare organizations are moving from deploying a chat assistant to deploying an AI workforce made of autonomous agents that handle routine tasks while humans focus on tasks that require their empathy, judgment, and expertise.

The safety paradox and the reclaimed time paradox

The momentum is real, but so are the challenges. The organizations that succeed must confront these honestly.

The safety paradox

A University of Lynchburg clinical review found ambient AI scribe platforms average 13.9 transcript errors per case, with 3.0 carrying moderate-to-severe harm potential. Because AI-generated notes read fluently, errors can hide in plain sight. The technology that saves documentation time can simultaneously create a new verification burden.

Harvard Business School profiled this tension in a case study on Mass General Brigham, where a pilot of AI-powered ambient documentation showed promising reductions in burnout and improved physician efficiency; however, AI hallucination concerns blocked wider deployment. Clinical AI without traceability is often clinical AI without trust.

The reclaimed time paradox

A sophisticated analysis by the Institute for Homeland Security (IHS) at Sam Houston State University found that while ambient AI reduces documentation time by 1–3 hours daily, the long-term impact on burnout is less certain. If reclaimed time is immediately absorbed by additional patient volume, the net effect on wellbeing might be minimal.

The counterpoint is one multicenter study, which found burnout decreased from 51.9 percent to 38.8 percent after 30 days of use, with after-hours EHR time dropping 23.2 percent.

Implementation and what works

Production deployments reveal what separates successful implementations from failed pilots:

Start with high-volume, low-complexity workflows. Appointment scheduling and patient verification are the most proven use cases. Expand to ambient documentation and coding after your organization establishes trust. As Joshua Glandorf, chief information officer of UC San Diego Health, put it, “It’s more effective to identify high-impact use cases and go deep.”

Appointment scheduling and patient verification are the most proven use cases. Expand to ambient documentation and coding after your organization establishes trust. As Joshua Glandorf, chief information officer of UC San Diego Health, put it, “It’s more effective to identify high-impact use cases and go deep.” Integration is everything. Standalone chat assistants deliver limited impact. Amazon Connect Health integrates with Epic, Oracle Health (formerly Cerner), MEDITECH, and over 100 systems through Redox, eliminating months of custom development.

Standalone chat assistants deliver limited impact. Amazon Connect Health integrates with Epic, Oracle Health (formerly Cerner), MEDITECH, and over 100 systems through Redox, eliminating months of custom development. Measure what matters. Production benchmarks include impactful statistics, such as more than 67 percent autonomous inquiry handling, more than 85 percent call deflection, wait times reduced from 18 minutes to under 30 seconds, more than 50 percent documentation time savings, and more than 90 percent clinician daily adoption.

Production benchmarks include impactful statistics, such as more than 67 percent autonomous inquiry handling, more than 85 percent call deflection, wait times reduced from 18 minutes to under 30 seconds, more than 50 percent documentation time savings, and more than 90 percent clinician daily adoption. Plan for three phases. Phase 1 is Efficiency: reduce documentation time, automate scheduling. Phase 2 is Safety: implement traceability, standardize review workflows, audit accuracy. Phase 3 is Wellness: deliberately protect reclaimed time, measure burnout alongside efficiency.

The three phases are illustrated in the following figure:

Figure 1: We recommend organizations progress through all three phases sequentially. Time savings are necessary but not sufficient

Five questions for healthcare leaders considering conversational AI

Before evaluating vendors for conversational AI in healthcare, stress-test your readiness:

Where’s your highest-volume, lowest-complexity patient interaction? Start there, prove value, then expand. How integrated are your systems? Conversational AI without EHR integration is just a chat assistant on a website. What will you do with reclaimed time? If saved time gets absorbed by additional volume, you’ve increased throughput, not reduced burnout. Who owns AI governance? If no single leader owns the framework, deployment stalls at compliance review. Are you measuring what matters? Track clinician wellbeing and clinical safety alongside call deflection and documentation time.

Imagine calling your hospital at 10 PM on a Sunday. Instead of voicemail, an AI agent verifies who you are, checks your insurance, finds the next available slot, books your appointment, and sends you a confirmation in under 2 minutes in your language. That’s not a vision for 2030. It’s what patients at UC San Diego Health, Portugal’s SNS24, and NHS Midlands are experiencing today.

What’s next

We predict that several trends will shape the next phase of conversational AI in healthcare. Health systems will move from single assistants to multi-agent orchestration, where coordinated teams of AI agents share patient context across one domain.

Voice-first interaction is likely to become the default, which especially matters in EMEA’s multilingual markets and in mobile-first populations across Africa and the Middle East. New contact center capabilities support this shift, including agentic voice, Agentic CX designer (a no-code canvas that combines deterministic and agentic AI steps), agent-to-agent orchestration, and Live Sync, a patented technology from AWS that combines voice and visuals.

At the same time, regulation is converging. The EU AI Act, EHDS, NHS England’s ambient voice technology (AVT) guidance, Germany’s Hospital Future Act, and Saudi Arabia’s Vision 2030 are creating clearer frameworks that can speed up adoption.

Finally, ambient AI is moving from documentation to decision support. Beyond capturing notes, it could surface real-time clinical insights and evidence-based treatment suggestions during encounters.

Getting started

For provider organizations, AWS pre-integrates patient engagement capabilities into Amazon Connect Customer. You can deploy with real-time EHR access in days. For healthcare tech builders, you can access point-of-care capabilities through the unified SDK. You can explore the sample application on GitHub.

The future of healthcare isn’t about replacing human connections. It’s about removing the administrative burden that prevents it. Conversational AI, and specifically agentic AI, is making that future possible today.

Further reading

Ongoing Watchfulness is Essential for Governing Healthcare AI

More from AI in healthcare

Survey: Providers and payers favor purchasing AI instead of developing it in-house.

Understanding HIPAA Data Ownership When Vendors Implement AI: Key Information for Healthcare Organizations

Algorithmic manipulation: The concealed dangers of artificial intelligence in healthcare

Oracle and AstraZeneca Collaborate to Enhance AI-driven Healthcare Innovations