Harvard Health Publishing extends a valuable service to our audience by granting access to a collection of archived materials. It's important to pay attention to the date indicated for the last review or update on each article.

Remember, the information available on this website, irrespective of its publication date, should never replace professional medical advice from your healthcare provider or any other certified medical expert.

AI in healthcare: Is a chatbot capable of responding to your medical inquiries?

A warm, sunlit modern living room: an adult sits on a light-gray sofa, holding a tablet that displays a medical chatbot interface labeled “AI Health Assistant.” Conversation bubbles read “My headache persists—what should I do?” and the chatbot’s reassuring response. On a nearby coffee table lie a stethoscope, prescription pad, and a cup of tea. A potted plant by the window adds a touch of calm, as soft natural light fills the scene.

AI in healthcare: Can a chatbot answer your medical questions? - Harvard Health

- Harvard Health Publishing offers access to a library of archived content for readers.  
- Articles include a date of last review or update for reference purposes.  
- Content on the site is not a substitute for professional medical advice.

As a service to our readers, Harvard Health Publishing provides access to our library of archived content. Please note the date of last review or update on all articles.

No content on this site, regardless of date, should ever be used as a substitute for direct medical advice from your doctor or other qualified clinician.

AI in healthcare: Can a chatbot answer your medical questions?

Presented by MongoDB

Creating AI solutions that are precise, secure, and dependable is a significant technical challenge for organizations operating under strict compliance mandates, particularly in the healthcare, financial services, and transportation sectors. These industries often lag behind in technological advancements due to regulations that necessitate a cautious approach to innovation. Additionally, many organizations are now faced with the task of modernizing their data infrastructures in order to meet the increasing demand for AI capabilities.

A notable example of successful modernization is Australian-founded AI Care Partner, Heidi. Its primary offering, Heidi Scribe, automates a substantial amount of administrative tasks that can consume clinicians' time, operating in over 190 countries and facilitating approximately 2.7 million patient interactions weekly. This growth is based on foundational infrastructure choices made long before the company expanded globally, according to Yu Liu, co-founder and CTO of Heidi.

“In most sectors, an AI feature with a two percent error rate may be merely an inconvenience, but in healthcare, this same error rate poses a serious risk to clinical safety,” Liu explains. “Therefore, our architecture is designed with the understanding that every output may be subject to scrutiny and auditing in patient care.”

Exploring the unique architectural challenges of deploying AI in healthcare

For Heidi, data residency is a fundamental requirement rather than an optional feature. Clinicians in different locations—whether in Sydney, London, Tokyo, or Denver—are governed by various regulatory frameworks such as the Australian Privacy Principles, GDPR, APPI, and HIPAA, necessitating that patient data remains within the respective regions.

Heidi employs fully logically isolated production environments worldwide, ensuring that data residency is an inherent characteristic of its architecture. Additionally, the system must be designed from the outset for auditability because organizations need to track the model's inputs, outputs, and clinician modifications for any session, even months later.

“The impact of change has to be minimized,” Liu emphasizes. “While less regulated industries allow for rapid deployment and subsequent fixes, in healthcare, we prioritize creating safe changes by default, implementing continuous integration gates on high-risk changes, utilizing canary releases, and treating database schema modifications with the same diligence as code reviews. Our speed is a result of this commitment to safety.”

Selecting a database for AI workflows

Heidi manages a wide range of medical data from diverse sources, including forms, referrals, and clinician notes, all of which must be unified into a consistent format and stored in a single location to align seamlessly with AI workflows. Traditional database structures were not suited for this requirement.

Consequently, a document database was the ideal solution for Heidi. By leveraging MongoDB, the team acquired the flexibility necessary to handle rapidly evolving AI data without the need for continual adjustments to the database architecture.

“The model constitutes about 20% of the overall system, and the data architecture dictates whether the remaining 80% performs effectively under actual clinical loads,” Liu states.

An AI Scribe session encompasses a variety of data types: transcripts, structured notes, templates, documents, patient context, and EHR integration states, all of which can change frequently. MongoDB enables this data to exist in forms that align with how clinicians function, allowing Heidi to adapt and evolve without being hampered by migration freezes as the product develops.

“MongoDB Atlas stood out for its combination of the document model's advantages—like seamless scalability, flexibility, and performance—with built-in AI-optimized features, including MongoDB Vector Search,” Liu notes. “Heidi does not need to integrate an additional vector database to enhance its platform.”

With more than 130 cloud regions worldwide, in addition to on-premises and hybrid options, MongoDB Atlas is the most accessible and globally distributed database solution available. Its unified query API allows developers to create full-text search capabilities, real-time analytics, and event-driven applications without complicating their architecture.

“Heidi Scribe translates large quantities of medical documents into vector embeddings using LangChain in Atlas, which facilitates semantic searches that connect transcribed medical terms directly to external knowledge bases,” Liu adds. “Switching to Atlas has cut latency on key APIs by almost 33%.”

Establishing a trustworthy clinical retrieval-augmented generation (RAG) system

“Retrieval is fundamentally a data architecture issue, rather than solely an AI problem,” Liu asserts. "In consumer applications, you collect data from the open web and hope for the best, whereas in healthcare, the focus is on ensuring compliance in what you retrieve."

Heidi Evidence draws from authorized clinical knowledge repositories, including collaborations with BMJ Best Practice, NICE CKS, and MIMS, and is designed to be jurisdiction-aware, ensuring that a clinician in the U.K. receives guidance relevant to the U.K. context, while an Australian clinician receives the Australian formulary—since what is accurate in one region may not be applicable in another.

Heidi’s embeddings and vector indexes reside within MongoDB Vector Search, situated in the same regionally isolated systems as the rest of its data. This ensures that retrieval processes cannot violate residency requirements, and obviates the need for a separate vector database with its own compliance and security protocols. Citations are treated as definitive contracts rather than suggestions, as the model only interacts with retrieved segments that are already linked to source records.

Region-specific isolation supports global compliance and scalability

“Each region operates a fully isolated production setup with its own MongoDB Atlas clusters, computing resources, and security keys,” Liu explains.

“This is what enables us to present a clear residency answer to U.S. health systems, NHS trusts, or Australian hospital groups, as this is enforced by our infrastructure rather than it being solely a contractual promise," he continues. "Managing multiple isolated regions efficiently relies on a consistent and well-maintained database infrastructure. Our multi-cloud framework means that launching a new region involves establishing another deployment using existing operational frameworks.”

This architectural approach has proved particularly effective in the U.S., where Beth Israel Lahey Health, among New England’s largest health systems, implemented Heidi’s AI Scribe after a pilot revealed that 74% of clinicians experienced reduced documentation time after hours, commonly referred to as “pajama time.” MaineGeneral Health, a non-profit health system, also selected Heidi as a strategic partner for its initiatives in rural healthcare.

"Establishing a presence in the U.S. meant we could create another regional setup using the infrastructure we had previously designed, rather than having to retroactively adjust for HIPAA compliance," Liu indicates.

Key takeaways and future plans

“Re-partitioning a large, continuously active data collection involves significant engineering efforts, whereas selecting a shard key upfront is part of initial design discussions,” Liu reflects. "We are currently engaging in this critical work alongside MongoDB partners, and the takeaway for those developing data-intensive AI products is that establishing horizontal scalability for fast-growing data is a foundational choice, much like ensuring data residency.”

Heidi plans to extend its offerings beyond consult notes to enhance the entire clinical workflow, encompassing pre-visit context, post-visit documentation, referrals, and workflow automation. The company is also investigating how MongoDB, large language models, and its own tools can create an integrated ecosystem for clinical processes.

“In the realm of healthcare AI, reliability engineering equates to trust engineering,” Liu states. “A clinician can lose trust just as quickly due to downtime, latency, or data inconsistencies as they would from an incorrect note. Much of our crucial work—such as canary releases with automatic rollbacks, continuous integration gates on changes to the database, and consistency checks across regions—is performed discreetly. Building clinician trust is the product we offer, and trust itself is deeply embedded in our architectural design.”

Heidi developed scalable, production-ready AI for the healthcare industry.

A confident woman in smart-casual attire standing in a sunlit, high-tech hospital operations room, floor-to-ceiling windows framing a softly glowing world map with data lines knitting together continents, nearby medical monitors display vital signs and AI-driven charts, warm natural light illuminating her focused expression as she taps a sleek touchscreen console, symbolizing production-ready healthcare AI deployed at global scale.

How Heidi built production-ready AI for healthcare at global scale - venturebeat.com

- AI in regulated industries like healthcare faces unique challenges due to compliance requirements.  
- Heidi Scribe automates clinical documentation, enhancing efficiency across 190 countries and 2.7 million interactions.  
- MongoDB's flexible architecture supports large-scale, compliant data handling essential for reliable healthcare AI.  

Presented by MongoDB

Building AI that is accurate, secure, and reliable is a major engineering feat for organizations subject to the compliance obligations that govern healthcare, financial services, and transportation. The challenge of delivering AI-driven products is compounded by the fact that technology in these industries has tended to lag behind other sectors because regulation requires organizations to move carefully — and slowly. Now, many are also confronting data infrastructure modernization projects as they try to catch up with today’s demand for AI.

Australian-founded AI Care Partner Heidi offers an example of successful modernization. Its flagship product, Heidi Scribe, now automates much of the administrative work that consumes clinicians’ days across more than 190 countries, supporting roughly 2.7 million patient interactions each week. That expansion rests on infrastructure decisions taken years before the company reached global scale, says Yu Liu, co-founder and chief technology officer at Heidi.

“In most industries, an AI feature that is wrong two percent of the time registers as an inconvenience, while in healthcare that same error rate becomes a clinical safety issue,” says Liu. “The architecture has to be built around the assumption that every output may be scrutinised, audited, and relied upon in a patient’s care.”

Why deploying production AI in healthcare is architecturally different

For Heidi, data residency is a precondition rather than a feature. A clinician in Sydney, London, Tokyo, or Denver is operating under different regulatory regimes, including the Australian Privacy Principles, GDPR, APPI, and HIPAA, and their patients’ data has to live in-region.

Heidi runs fully logically isolated production deployments across the world, so residency is enforced by architecture. Auditability also has to be built in from day one, because an organization needs to be able to answer what the model saw, what it produced, and what the clinician changed, for any session, months later, when called upon.

“The blast radius of change must be engineered down,” Liu says. “In less regulated industries you can ship fast and fix forward, but in healthcare we invest heavily in making change safe by default, with continuous integration gates on risky change classes, canary releases, and treating even database schema and index changes as code that goes through review. Our speed is a product of that safety rather than something we achieve in spite of it.”

Choosing a database to connect with AI workflows

Heidi handles a diverse set of medical data collected from multiple sources, including forms, referrals, and clinicians’ notes, all of which had to be consolidated into one consistent format and one location to connect seamlessly with AI workflows. Rigid rows and columns would have been ill-suited to that workload.

For Heidi, those requirements made a document database the natural choice.MongoDBgave the team the flexibility to accommodate rapidly changing AI data without constantly reshaping the underlying database.

“The model is maybe 20% of the system, and the data architecture is what determines whether the other 80% holds up under real clinical load,” Liu says.

An AI Scribe session isn’t a single piece of data. It’s a collection of transcripts, structured notes, templates, documents, patient context, EHR integration state, and dozens of other related artifacts that change from week to week. MongoDB lets a session’s data live together in shapes that match how clinicians actually work, and lets Heidi evolve those shapes without a migration freeze every time the product moves.

"MongoDB Atlas stood out because it combined the power of the document model, which allows seamless scale, flexibility, and high performance, with built-in AI-ready features such as MongoDB Vector Search,” Liu says. “This means that Heidi does not need another bolt-on vector database to augment its existing platform.”

With more than 130 cloud regions globally alongside on-premises and hybrid options, MongoDB Atlas is the most widely available, globally distributed database platform, and its unified query API lets developers build full-text search, real-time analytics, and event-driven experiences without complicating their architecture.

"Heidi Scribe converts large volumes of medical documents into vector embeddings via LangChain in Atlas, enabling semantic search that connects transcribed medical terms directly to corresponding external knowledge," Liu adds. "Migrating to Atlas reduced latency on key APIs by nearly 33%."

What a trustworthy clinical RAG system requires

“Retrieval is a data architecture problem before it is an AI problem,” Liu says. "In consumer RAG, you retrieve from the open web and hope, whereas in healthcare what you retrieve from is the compliance surface."

Heidi Evidence retrieves from licensed clinical knowledge bases, including partners like BMJ Best Practice, NICE CKS, and MIMS, and it is jurisdiction-aware, so a U.K. clinician gets U.K. guidance and an Australian clinician gets Australian formularies, because the right answer in one country can be the wrong answer in another.

Heidi’s embeddings and vector indexes live in MongoDB Vector Search, inside the same regionally isolated deployments as the rest of its data, which means retrieval physically cannot cross a residency boundary, and they are not operating a separate vector database with its own security and compliance story. Citations are a hard contract rather than a prompt suggestion, because the model only ever sees retrieved chunks that are already bound to source records.

Regional isolation enables global compliance and scale

“Each region is a full, isolated production deployment with its own MongoDB Atlas clusters, its own compute, and its own key,” Liu says.

“That is what lets us walk into a U.S. health system, an NHS trust, or an Australian hospital group and give a clean answer on residency, because it is enforced by infrastructure rather than promised by contract," he explains. "Running multiple isolated regions with a lean team only works because the database layer is managed and consistent. We are also multi-cloud, meaning a new region can stand up another deployment on rails we have already built."

That architecture has been most visible in the U.S., where Beth Israel Lahey Health, one of New England’s largest health systems, rolled out Heidi’s AI scribe following a pilot finding 74% of clinicians reported reduced after-hours documentation (“pajama time”), and where non-profit system MaineGeneral Health selected Heidi as a strategic partner in its rural healthcare work.

“Entering the U.S. market meant standing up another region on rails we had already built rather than re-engineering for HIPAA after the fact,” Liu says.

Lessons learned and the roadmap ahead

"Re-partitioning a large, hot, always-on collection is a serious engineering program, whereas choosing a shard key on day one is a design meeting," Liu says. "We are doing that work now in partnership with MongoDB, but the lesson for anyone building a data-heavy AI product is that horizontal scale for your fastest-growing data is a founding decision, just like residency."

Heidi is now extending beyond the consult note to support the full clinical workflow, from pre-visit context to post-visit documents, referrals, and workflow automation. The company is also exploring how MongoDB, large language models, and its own tooling can power an agentic ecosystem for clinical workflows.

“In healthcare AI, reliability engineering is trust engineering,” Liu says. “A clinician’s trust is lost just as fast by downtime, latency, or a data inconsistency as by a bad note, and some of our highest-leverage work is invisible, including canary releases with automatic rollback, CI gates on database changes, and cross-region consistency checks. Clinician trust is the product, and trust is architectural.”

How Heidi built production-ready AI for healthcare at global scale

Healthcare institutions are rapidly adopting AI technologies, yet many are not adequately preparing for the integration of AI as a colleague in the workplace.

Traditionally, the healthcare sector has viewed AI primarily as a tool to support human tasks. For instance, ambient documentation software aids doctors in completing notes, while virtual assistants field patient inquiries. Similarly, prior authorization tools streamline administrative processes. These technologies have been straightforward to understand, aligning with the conventional model where AI assists humans, who maintain responsibility for decision-making and governance.

However, a shift is occurring. Increasingly, healthcare organizations are entrusting AI agents with responsibilities that once necessitated human input. These agents are evolving beyond mere productivity boosters; they are becoming digital team members that need proper governance, oversight, and established operational boundaries. AI is no longer just an assistant; it is now an active participant in workflows.

This transformation marks a significant shift in how healthcare envisions AI's role.

The Transition

Many healthcare organizations are still evaluating AI on a case-by-case basis, asking whether these tools can alleviate documentation burdens, enhance call center performance, or accelerate prior authorizations. While these queries are vital, they underestimate the rapid evolution taking place. Historically, healthcare systems have centered around human actions, leading to governance frameworks and operational processes that cater to human interactions.

As AI agents take on more independent roles, healthcare organizations must adapt to a new operational model, different from anything they’ve managed before.

AI agents follow distinct operational rules. Unlike their human counterparts, these agents work continuously and can scale almost instantaneously. They manage information at machine speed, which may soon lead to a situation where digital agents outnumber human workers within healthcare systems. 

With capabilities such as simultaneous task execution and multitasking across a variety of systems, AI agents challenge longstanding assumptions about operational capacities, governance structures, and oversight.

This evolution transcends a mere technological challenge; it also presents a significant workforce management issue.

The Data Challenge

AI agents rely heavily on data to function effectively, whether for clinical documentation or care coordination. As organizations deploy numerous AI agents—potentially hundreds or even thousands—the volume of data accessed and processed will surge. However, the real challenge lies not just in managing increased data but in overseeing the significant operational activity occurring in systems originally designed for human users.

Each AI agent acts as a participant, requesting data, starting workflows, making decisions within set guidelines, and interfacing with enterprise applications. This influx creates new demands for governance, security, identity management, and data integrity across the organization.

Leaders in healthcare often ponder how many AI tools they can deploy. A more pertinent question is: How many digital agents can your organization effectively supervise? Organizations that start considering this question now will be in a better position than those still focused on deploying AI in isolated use cases.

The Governance Oversight

Healthcare possesses established methods to govern human users—employees have credentials, defined permissions, and training, and their actions can be audited. Yet, AI systems introduce a different set of governance considerations: 

- What information should each agent access?
- Who is accountable for the outputs of an AI agent?
- What processes are in place for granting or revoking permissions for agents?
- How can an organization monitor thousands of AI workers without overwhelming existing systems?

These questions pertain to governance but also represent broader leadership challenges. While models exist for managing human employees, healthcare organizations have yet to develop robust frameworks to govern AI agents that can make decisions at machine speeds.

Recognizing this gap allows organizations to begin formulating the policies, accountability guidelines, and operational practices necessary for appropriately managing their AI workforce.

As organizations transition from pilot programs to widespread adoption, these issues will become increasingly critical.

The Leadership Imperative

Those organizations that successfully navigate this evolution will not necessarily be the ones with the most extensive AI deployments. They will be the ones that understand AI agents as more than just technological implementations, viewing them as integral participants in daily operations. 

Instead of fearing these AI colleagues, healthcare must acknowledge that leading them requires a different mindset than simply deploying new technologies. Installing technology is manageable, but leading a workforce demands strategic vision and guidance.

Sagnik Bhattacharya, CEO of Rhapsody, emphasizes the importance of building the necessary infrastructure to support AI and connected care within healthcare organizations. This evolution is an opportunity for leaders to rethink governance and operational models in line with the changing landscape.

Healthcare Is Introducing AI Solutions — It’s Not Prepared for AI Partners

A dimly lit hospital corridor in warm, low-key shadows. A group of adult medical staff in scrubs clusters around a mobile cart topped with a softly glowing amber touchscreen. From the screen emerges a half-formed robotic hand with exposed wires, reaching toward a hesitant doctor whose face is etched with uncertainty. In the background, shadowy silhouettes of patients in beds and worried nurses underscore the uneasy arrival of AI colleagues in healthcare.

Healthcare Is Deploying AI Tools — It’s Not Ready for AI Colleagues - MedCity News

- Healthcare organizations are deploying AI agents, evolving from assisting to independently performing tasks.  
- AI agents require new governance and operational models to manage their integration effectively.  
- Successful transitions will depend on recognizing AI as participants, not just technologies to implement.  

Healthcare organizations are moving quickly to deploy AI tools, yet few are preparing for AI colleagues.

Until now, healthcare has largely viewed AI as technology that supports human work. Ambient documentation applications help physicians complete notes. Virtual assistants answer patient questions. Prior authorization tools help administrative teams process paperwork more efficiently.

Those technologies have been relatively easy to understand because they fit within an existing operating model: AI assists people, while people remain responsible for the work. Governance, accountability, and decision-making still center on human users.

Sponsored Post Closing the Quality-Affordability Gap in Workplace Mental Health In an interview, Kyan Health Co-Founder and Chief Commercial Officer Konstantin Struck discussed how Kyan gives mid-market and enterprise employers access to premium workforce mental healthcare, at a price point that is affordable.

Healthcare organizations are now beginning to rely on AI agents to perform tasks that previously required human intervention. Instead of acting as another productivity tool, these agents are beginning to resemble digital colleagues: participants in everyday work that require governance, oversight, and clear operational boundaries. AI agents aren’t simply supporting workflows. They’re participating in them.

That’s a fundamentally different role than healthcare has asked AI to play before.

The big shift

Most healthcare organizations are still measuring AI success one use case at a time. Can it reduce documentation burden? Improve call center efficiency? Speed up prior authorizations?

Those questions matter, but they assume a slower transition than what’s actually underway. Historically, healthcare systems were designed around human activity. Every login, workflow, transaction, and data request originated from a person. Governance frameworks, security policies, capacity planning models, and operational processes all evolved around that assumption.

That assumption is beginning to change. As AI agents move beyond assisting people to perform work independently, healthcare organizations will need to manage a fundamentally different operating model than they’ve ever managed before.

Agents operate by different rules.

Unlike humans, agents operate continuously, work in parallel, and can be deployed at scale almost overnight. They generate and consume information at machine speed. As a result, healthcare organizations may soon have more digital workers interacting with systems than human ones.

AI agents don’t have many of the constraints that humans do such as scheduled shifts, multitasking, quick context switching. They can execute tasks simultaneously, continuously, and across multiple systems. That changes long-held assumptions about capacity, governance, oversight, and operational design

This is no longer just a technology issue. It’s a workforce management issue.

The hidden consequence: A data explosion

Every AI agent depends on data to do its job. Whether it’s supporting clinical documentation, scheduling, prior authorization, or care coordination, agents require access to information across multiple systems and workflows.

As organizations deploy hundreds, or eventually thousands, of AI agents, the volume of data being accessed, exchanged, and acted upon will increase dramatically. But the bigger challenge isn’t simply managing more data. It’s managing far more activity across systems that were originally designed around human users.

Every new agent becomes another participant requesting information, initiating workflows, making decisions within defined parameters, and interacting with enterprise applications. That places new demands on governance, security, identity, auditability, and the infrastructure responsible for moving trusted data across the organization.

Healthcare leaders often ask how many AI tools they can deploy. A more important question is: How many digital workers can your organization realistically govern?

Organizations that begin answering that question now will be better prepared than those still evaluating AI one use case at a time.

The governance question nobody is asking

Healthcare has mature processes for governing human users. Employees receive credentials. They have defined permissions. They undergo training. Their activity can be monitored and audited.

Agentic systems raise a different set of questions:

Which agents should have access to what information? Who is accountable for actions taken by an agent? How are permissions granted, modified, and revoked? How is agent activity monitored at scale? How do organizations ensure thousands of digital workers don’t overwhelm existing systems and workflows?

These are governance questions, but they’re also leadership questions.

Healthcare already has mature models for managing human users. What it doesn’t yet have are mature operating models for managing non-human participants that can access information, trigger workflows, and take action at machine speed.

The sooner organizations recognize that gap, the sooner they can begin designing the policies, accountability frameworks, and operational practices needed to manage AI colleagues responsibly.

These matters will only become more pressing as organizations move from isolated pilots to enterprise-wide deployment.

The leadership challenge

The organizations that navigate this transition most successfully won’t necessarily be the ones that deploy the most AI. They’ll be the ones that recognize AI agents aren’t simply another technology to implement, they’re becoming participants in everyday work.

Healthcare doesn’t need to fear AI colleagues. It needs to recognize that leading them requires a different mindset than deploying them. Technology can be installed. A workforce has to be led.

Photo: Yuichiro Chino, Getty Images

Sagnik Bhattacharya Sagnik Bhattacharya is CEO of Rhapsody, a global leader in agent-ready interoperability, helping healthcare organizations build the data and integration infrastructure needed to power AI and connected care.

This post appears through the MedCity Influencers program. Anyone can publish their perspective on business and innovation in healthcare on MedCity News through MedCity Influencers. Click here to find out how.

Healthcare Is Deploying AI Tools — It’s Not Ready for AI Colleagues

Federal and state agencies are increasingly leveraging artificial intelligence (AI), and the healthcare sector is no exception. The U.S. Department of Health and Human Services (HHS) has reported significant rises in AI implementation among key health organizations from fiscal years 2024 to 2025. The U.S. Food and Drug Administration (FDA) saw a remarkable 148 percent increase, while the U.S. Centers for Disease Control and Prevention (CDC) experienced an 87 percent boost. The Centers for Medicare and Medicaid Services (CMS) and the National Institutes of Health (NIH) recorded increases of 78 percent and 51 percent, respectively. State governments are also embracing AI technologies.

While the drive towards AI is motivated by goals of enhancing efficiency, improving accuracy, and achieving cost reductions, this rapid integration into public health raises significant concerns regarding the appropriateness of AI in certain governmental decisions, particularly when human oversight may be required. In critical areas like healthcare, as AI systems become more sophisticated and less transparent, the risk of detrimental consequences increases. The complexity of these systems makes it more difficult to identify and correct errors, potentially leading to serious oversight issues where mistakes can go unnoticed, with even rare miscalculations capable of severely affecting patient care.

The types of AI employed in the healthcare sector have evolved noticeably. Previously, agencies primarily used rule-based algorithms, which were straightforward and transparent, allowing for easier audits. For instance, during the onset of the COVID-19 pandemic in 2020, states deployed these simple algorithms to prioritize access to limited monoclonal antibody supplies based on race and sex, thereby clarifying their decision-making processes and ensuring accountability under anti‑discrimination laws.

By 2023, during the Medicaid eligibility assessments known as “unwinding,” states also relied on rule-based algorithms. Unfortunately, due to programmer errors, data integrity issues, and insufficient oversight, many individuals were incorrectly disenrolled. A class action lawsuit against TennCare highlighted alleged violations of the Medicaid Act and the Americans with Disabilities Act, partly due to the shortcomings of its computerized Eligibility Determination System. Similar issues arose from Deloitte Consulting’s software in Texas, raising alarms about the dependability of automated eligibility evaluations used across various states. 

Since 2023, AI capabilities have advanced rapidly, marked by improvements in model performance and real-world applications. Federal agencies are progressively adopting machine-learning models and generative large language models (LLMs). Recent data from the U.S. Government Accountability Office (GAO) indicates a doubling of overall AI use cases across 11 federal entities between 2023 and 2024, with generative AI cases skyrocketing by a factor of nine, led by health agencies. States are also exploring generative AI applications and developing guidance on its utilization.

Although the recorded negatives linked to predictive machine learning and generative AI in healthcare remain relatively scarce, emerging evidence points to alarming risks. Studies have documented instances of diagnostic errors, unsafe triage recommendations, privacy violations, and concerning uses of these technologies in public decision-making. Notably, a class action suit from Medicare Advantage beneficiaries claimed that an AI system used by UnitedHealth Group incorrectly denied necessary medical care for post-acute treatments. A federal court later mandated the disclosure of the AI's error rates, which the plaintiffs contended were as high as 90 percent.

These cases demonstrate the critical need for healthcare agencies to conduct thorough evaluations prior to implementing AI systems that could lead to significant harm. Current federal policies necessitate AI impact assessments for high-stakes applications that could impact individual rights or safety. However, these frameworks primarily address risk mitigation rather than questioning the initial appropriateness of AI in specific decisions. They require agencies to clarify the system's objectives, evaluate data quality, assess privacy implications, analyze costs, and ensure independent review, but they do not confront the essential question of whether AI should be employed at all.

Agencies currently lack a structured approach to determine when it's appropriate to utilize AI, how much human judgment should be involved, and how the specific governmental function should inform the design and distribution of automated systems. This absence of a consistent framework poses a significant risk of making ill-considered AI deployment decisions that do not adhere to principles of administrative law.

To address this gap, I advocate for a Five Factor Framework (FFF) that focuses on various interconnected aspects of the decision-making context and the foundational principles guiding administrative processes. This framework encourages agencies to:

1. Identify the interests involved, including affected parties and the potential harms or rights at stake.
2. Examine the nature and ethical dimensions of the decision, encompassing its functional needs and procedural demands as aligned with the Administrative Procedure Act's (APA) reasoned decision-making standards.
3. Assess the characteristics of the proposed technology—its abilities, limitations, and error risks—through the lens of doctrines relevant to arbitrary and capricious review.
4. Evaluate the integration of human oversight in decision-making—how human judgement complements or replaces automated interventions, ensuring compliance with nondelegation principles and due process requirements.
5. Identify the legal parameters governing the decision, including protections related to due process and civil rights along with any sector-specific statutory obligations.

Collectively, these inquiries create a robust, legally grounded foundation for assessing whether technological interventions serve the public good. If an agency concludes that the advantages of incorporating AI for a critical decision outweigh the potential downsides, it should engage with the public for feedback on its evaluation. This could take the form of public notices, community meetings, or consultations with advocacy groups representing affected populations. Achieving democratic legitimacy necessitates not only technical conformity but also citizen participation in decisions that influence healthcare access.

Ultimately, the core issue is not whether AI can enhance the efficiency of government healthcare systems, but whether such efficiency justifies the risks associated with AI errors that could impact individuals’ health, rights, or lives. In the absence of a principled framework dictating when AI should be applied or avoided, agencies run the risk of systemically automating harm. The public deserves a better approach, and the legal system demands it.

Government Healthcare Agencies' Use of AI: Is It Beneficial for the Public?

A sunlit government healthcare agency boardroom where mature doctors and officials in suits and lab coats gather around a polished wooden table, each holding a tablet. On the wall behind them is a subtle official health agency seal. A large touchscreen display shows an AI-powered medical dashboard—patient charts, ECG graphs, and a stylized digital-brain icon—bathed in warm natural light, as the group leans in, engaged in thoughtful discussion about public interest.

The Use of AI by Government Healthcare Agencies: Is It in the Public Interest? - petrieflom.law.harvard.edu

- Federal healthcare agencies are increasingly adopting AI, with FDA usage surging by 148%.  
- AI in healthcare raises concerns about transparency, oversight, and potential harm to patients.  
- A proposed Five Factor Framework aims to guide justified AI deployment in decision-making.

Federal and state departments are rapidly expanding their use of artificial intelligence, and healthcare agencies are no exception. According to the U.S. Department of Health and Human Services (HHS) AI Use Case Inventory, between FY 2024 and 2025 the U.S. Food and Drug Administration (FDA) experienced a 148 percent surge in AI use, the U.S. Centers for Disease Control and Prevention (CDC) an 87 percent increase, the Centers for Medicare and Medicaid Services (CMS) a 78 percent rise, and the National Institutes of Health (NIH) a 51 percent jump. States are following suit.

These developments are being pursued under the banner of boosting efficiency, accuracy, and cost savings. But the accelerating adoption of AI in public healthcare agencies raises a more fundamental question: Are there some governmental decisions for which AI should not be used at all or at least not be used without human oversight? In high‑stakes domains like healthcare, the answer may be yes. As the technology becomes more complex, opaque, and autonomous, the potential for harm may increase. Greater complexity can make errors harder to detect and when an error occurs, tracing its source and correcting it can be difficult, increasing the risk that mistakes go unnoticed. In healthcare, even rare errors can have serious consequences for patients.

Types of AI used by health care and other agencies have progressed over the last several years from rule-based algorithms, to predictive machine-learning systems, and most recently, generative and agentive systems. Less than a decade ago, health agencies relied primarily on rule‑based algorithms. They were transparent, deterministic, and relatively easy to audit. For example, in 2020, as the nation confronted a deadly and uncontrolled COVID‑19 virus and no vaccine yet existed, states allocated scarce supplies of monoclonal antibodies available under an FDA emergency use authorization. To maximize lives saved, some states, such as Utah and New York, adopted algorithms that prioritized individuals by race and/or sex. These were simple algorithms; their logic was explicit, and their goals transparent. This transparency also enabled accountability via legal challenges under anti‑discrimination laws.

Fast forward to 2023 and the Medicaid “unwinding.” Again, states relied on rule‑based algorithms — this time to determine who remained eligible for Medicaid. But programmer errors, data flaws, and inadequate oversight led to hundreds of thousands of improper disenrollments. A successful class action suit against TennCare alleged violations of the Medicaid Act, the ADA, and due process, stemming in part from TennCare’s computerized Eligibility Determination System (TEDS). Around the same time, the National Health Law Program filed a complaint with the FTC alleging that Deloitte Consulting’s defective eligibility software caused widespread improper terminations in Texas, although the software was used by 20 other states. The FTC has not acted, but the allegations underscore the risks of automated eligibility systems even before the arrival of more advanced AI.

Since 2023, AI capabilities have progressed at rapid pace, with major leaps in model performance, computing capacity, and real‑world deployment. Agencies are increasingly adopting machine‑learning models, “black box” systems, and generative large language models (LLMs). A recent U.S. Government Accountability Office (GAO) report found that while overall AI use cases across 11 federal agencies doubled from 2023 to 2024, generative AI use cases increased nine‑fold, with health-based agencies leading the way. States are also experimenting with generative AI and producing guidance on using the technology.

While documented harms associated with predictive machine learning systems and generative AI remain a small fraction of their use in healthcare, a growing body of evidence highlights serious risks. Studies reveal troubling vulnerabilities including diagnostic errors, unsafe triage advice, privacy breaches, and questionable applications in public sector decision‑making. One striking example involves Medicare Advantage preauthorization. Beneficiaries filed a class action suit alleging that UnitedHealth Group used an AI program to deny medically necessary post‑acute care. In March 2026, a federal district court ordered the defendants to disclose the AI’s error rates, which plaintiff’s alleged were 90 percent, meaning “nine of 10 appealed denials were ultimately reversed.”

These examples illustrate why government healthcare agencies must conduct comprehensive evaluations before deploying AI systems that could cause serious harm. Existing federal frameworks require AI impact assessments for “high‑impact” uses affecting individual rights or safety. But these frameworks assume that AI will be deployed and focus on mitigating risks, not determining whether AI should be used in the first place. The assessments require agencies to explain the system’s purpose and benefits; evaluate data quality and model fitness; assess impacts on privacy, civil rights, and civil liberties; analyze costs; and obtain independent review. These steps, however, do not answer the threshold question: Should AI be used for this decision at all?

Agencies currently lack a method for determining when AI use is justified, what level of human judgment is required, and how the nature of the governmental function should shape the design and deployment of automated systems. Without a consistent framework, agencies risk making AI deployment decisions that fail to satisfy administrative law principles of arbitrariness review and procedural regularity.

To fill this gap, I propose a Five Factor Framework (FFF) requiring an inquiry into several interrelated dimensions of the decision environment and the doctrinal commitments that shape administrative decision‑making. The framework calls on agencies to:

identify the interests at stake — who may be affected and what harms or rights are implicated. examine the nature and normative structure of the decision itself: its functional demands, procedural requirements, and value‑laden qualities, situated within the APA’s requirements of reasoned decision‑making. assess the characteristics of the technology proposed to perform the task, including its capabilities, limitations, and error modes, considering doctrines governing arbitrary and capricious review. evaluate the role of humans in the decision‑making process — how human judgment, oversight, and discretion are integrated or displaced, and whether the allocation of authority is consistent with nondelegation principles, due process requirements, and statutory expectations. identify the legal constraints governing the decision, from due process and civil rights protections to sector‑specific statutory requirements.

Together, these five inquiries provide a comprehensive, doctrinally grounded foundation for determining whether a technological intervention aligns with the public interest. If an agency concludes that the benefits of using AI for a high-impact decision outweigh the risks, it should seek public input on its assessment — through notice and comment, town halls, and/or engagement with advocacy groups representing affected communities. Democratic legitimacy requires more than technical compliance; it requires public participation in decisions that shape access to healthcare.

In the end, the question is not whether AI can make government healthcare systems more efficient; it is whether efficiency is worth the price of AI errors that cost people their health, their rights, or their lives. Without a principled framework guiding when AI should or should not be used, agencies risk automating harm at scale. The public deserves better, and the law demands it.

The Use of AI by Government Healthcare Agencies: Is It in the Public Interest?

Tech executives have frequently cautioned that artificial intelligence has already reached the proficiency of entry-level staff, with predictions suggesting it could cut white-collar employment in half by 2030. Given this context, many are now focusing on the healthcare sector, which boasts low unemployment, the potential for salaries exceeding $200,000, and a profession that is less likely to be overtaken by automated systems in the near future.

However, individuals contemplating medical school should bear in mind an important fact: job security does not always equate to job satisfaction.

A 2025 study conducted by the shift work platform Deputy, which encompassed responses from 1.28 million users, indicates that roles such as doctors, paramedics, and chiropractors rank among the least satisfied in the workforce. Approximately 38% of workers in doctors' offices and medical clinics reported dissatisfaction with their jobs. Chiropractors and personnel in critical and emergency services also showed considerable unhappiness. Interestingly, when considering roles in animal health, four out of the five least satisfying jobs in the UK today are tied to the healthcare field.

Despite the perceived fulfillment associated with healthcare careers, employees in this sector often experience burnout, exacerbated by demanding hours and high-stakes situations.

The report identifies several factors contributing to low morale: “Staff shortages, emotional challenges, unpredictable schedules, and an aging population” are significant elements affecting employee happiness.

Below are the top 10 sectors with the highest levels of dissatisfaction, according to the study:

1. Doctors’ Office/Medical Clinic – 37.84%
2. Animal Health – 17.95%
3. Chiropractors – 12.93%
4. Critical & Emergency Services – 12.05%
5. Call Centres – 12.00%
6. Catering – 8.60%
7. Delivery and Postal Services – 6.97%
8. Care Facilities – 6.22%
9. Cleaning Services – 5.80%
10. Private Services (e.g., Chefs, Gardeners) – 5.62%

In contrast, it appears that Generation Z is finding greater happiness in hospitality roles.

Interestingly, positions that recent graduates may have traditionally overlooked, such as those in fast food or waitressing, are emerging as potentially more fulfilling career paths. Hospitality received notable praise in Deputy’s study, with half of the ten happiest job sectors represented within it, despite the common challenges of stress, irregular hours, and lower wages.

In terms of job satisfaction, hospitality roles led the findings. Staff in sit-down restaurants (89.7%), fast food and cashier positions (82.9%), food pop-ups (82.5%), and café employees (82%) expressed some of the highest satisfaction levels across various fields.

Florists, childcare workers, and cleaners also reported significant job contentment.

The report points out that the satisfaction derived from these roles stems less from financial compensation or status and more from their daily work experiences: “These positions may enjoy structured routines, manageable workloads, and strong team bonds, underscoring the emotional importance of good operational practices and a positive workplace atmosphere.”

Separately, data indicates that wages for bartenders and baristas are currently growing at a faster rate than those for traditional office positions.

According to the research, here are the top 10 happiest sectors:

1. Tobacco, E-cigarette, and Vape Stores – 93.4%
2. Sit-Down Restaurants – 89.7%
3. Fast Food/Cashier Restaurants – 82.9%
4. Florists – 82.9%
5. Food Pop-Ups – 82.5%
6. Cafes/Coffee Shops – 82%
7. Dentists – 81.8%
8. Childcare/Community Centres – 78.4%
9. Catering – 75.3%
10. Cleaning Services – 64.3%

This article first appeared on Fortune.com on August 31, 2025.

Gen Z seeks AI-resistant healthcare careers—chiropractors, doctors, and paramedics report the lowest job satisfaction.

A dimly lit hospital corridor at night, its walls streaked with shadow. Three exhausted medical professionals—a chiropractor leaning against a treatment table, a doctor slumped over a clipboard, and a paramedic resting on a stretcher—each with drawn, weary faces under a single flickering overhead lamp. In the foreground, a lone Gen Z adult with a backpack and smartphone stands in silhouette, gazing thoughtfully upward at a glowing red neon sign reading “AI-PROOF CAREERS.” Ominous, dark theme, moody shadows, natural harsh corridor lighting.

Gen Z eye up AI-proof healthcare—chiropractors, doctors and paramedics are the unhappiest workers - Fortune

- Tech leaders anticipate AI could replace half of white-collar jobs by 2030.  
- Healthcare jobs, despite low unemployment, report high dissatisfaction among workers.  
- Hospitality jobs rank highest in satisfaction due to manageable workloads and team camaraderie.  

Tech leaders have consistently warned AI is already as good as entry-level workers and it could halve white-collar jobs by 2030. So it’s no wonder they’re eyeing up jobs in health care, which offer low unemployment rates, the potential to earn over $200,000, and are unlikely to be replaced with robotic doctors and nurses anytime soon.

But there’s one thing they should know before filling out medical school applications: Pursuing job security doesn’t necessarily guarantee job satisfaction.

That’s because 2025 research from shift work platform Deputy, which surveyed 1.28 million users, ranks doctors, paramedics, and even chiropractors as the unhappiest workers.

In fact, doctors’ offices and medical clinics recorded the highest levels of dissatisfaction, with nearly 38% of respondents saying they’re unhappy in their jobs. Chiropractors and staff in critical and emergency services weren’t far behind. And if you include animal health roles, 4 out of the 5 worst jobs for happiness in the UK right now are in healthcare.

Despite health care’s reputation for meaningful work, these roles are often more likely than most to leave workers burned out and ground down by long hours and high stakes.

“Staffing shortages, emotional strain, unpredictable rosters, and an ageing population are cited as key contributors to declining morale,” the report highlighted.

Top 10 unhappiest industry sectors, per the research

Doctors Office/Medical Clinic – 37.84% Animal Health – 17.95% Chiropractors – 12.93% Critical & Emergency Services – 12.05% Call Centres – 12.00% Catering – 8.60% Delivery and Postal Services – 6.97% Care Facilities – 6.22% Cleaning Services – 5.80% Private Services (Chefs, Gardeners etc) – 5.62%

Gen Z may be happier in hospitality jobs

What’s perhaps most surprising is that jobs many recent grads might have once looked down on—like fast food or waitressing roles—are emerging as a safer bet for a more satisfying career.

Hospitality fared well in Deputy’s study, making up half of the 10 happiest job sectors, despite the sector’s reputation for high stress, unsociable hours, and low pay.

Hospitality jobs dominated the happiness rankings. Sit-down restaurant staff (89.7%), fast food and cashier restaurant workers (82.9%), food pop-up teams (82.5%), and café or coffee shop employees (82%) all reported some of the highest job satisfaction scores of any sector.

Florists, childcare workers and cleaners also reported notably high levels of job contentment.

What makes these roles so satisfying? The report suggests it’s less about pay or prestige, and more about the day-to-day experience: “These roles may benefit from clearer routines, manageable workloads, and stronger team camaraderie, highlighting the emotional value of operational structure and positive workplace culture.”

Although probably not at the top of most graduates’ dream career list, separate data also show wage growth for bartenders and baristas is outpacing that of desk workers.

Top 10 happiest industry sectors, per the research

Tobacco, E-cigarette and Vape Stores – 93.4% Sit Down Restaurants – 89.7% Fast Food/Cashier Restaurants – 82.9% Florists – 82.9% Food Pop-Ups – 82.5% Cafes/Coffee Shops – 82% Dentists – 81.8% Childcare/Community Centres – 78.4% Catering – 75.3% Cleaning Services – 64.3%



A version of this story originally published on Fortune.com on August 31, 2025

Read more on the future of work from Fortune’s Orianna Rosa Royle:

Gen Z eye up AI-proof healthcare—chiropractors, doctors and paramedics are the unhappiest workers

The integration of artificial intelligence in various sectors is becoming widespread, with health care being a critical area where the interplay between human intelligence and AI can have significant implications for patient outcomes.

Dr. Jose Francisco Abad, a family medicine doctor and a certified clinical informatics expert at UPMC, has addressed the nuanced and shifting dialogue regarding the effectiveness of AI in the health care environment.

When asked about the potential issues of self-diagnosing through the internet, whether aided by AI or not, Dr. Abad reflected on past warnings about online searches leading to alarming conclusions. He remarked, "Previously, I referred to Google as ‘three clicks to death.’ Inputting a symptom like a cough can quickly lead you down a path to a dire diagnosis, such as lung cancer." He emphasized that AI could exacerbate this issue by generating unfounded assumptions about our health, potentially causing psychological distress without the necessary context and support from a healthcare provider. This human touch is vital when relaying test results, as it helps prepare patients emotionally for various outcomes and emphasizes the importance of follow-up care.

To evaluate the credibility of online health information utilized by AI, Dr. Abad advised caution. If the AI references lesser-known websites, trust should be exercised cautiously. He underscored the reliability of established organizations like the CDC, World Health Organization, and other reputable bodies. Often, concerns he had regarding AI outputs stemmed from unreliable sources.

On the subject of AI's diagnostic capabilities, Dr. Abad pointed out that AIs can only provide recommendations based on the data they receive. He referred to the adage, "Garbage in, garbage out," highlighting that incorrect input will lead to incorrect conclusions.

Dr. Abad also elaborated on why human professionals excel in making direct diagnoses. Physicians possess contextual knowledge and extensive experience that help them interpret nonverbal cues and differentiate between patient conditions. This wealth of information is often lost in AI interactions.

When discussing the reliability of medical advice from AI, he expressed a necessary skepticism. It’s crucial to approach AI-generated recommendations with caution, particularly for complex health inquiries. While AI can provide reassurance for straightforward questions, it should be used as a supplementary tool rather than a definitive resource.

For instance, Dr. Abad discussed how AI might assist in addressing simple queries, like whether certain medications are safe during pregnancy. However, it should also inspire deeper conversations with medical professionals who can make comprehensive assessments.

In his practice, Dr. Abad mentioned a practical application of AI that helps him take notes during patient consultations while also prompting him to consider additional factors that may have been overlooked, enhancing his ability to make thorough evaluations.

Looking to the future, Dr. Abad is optimistic about the evolution of AI in health care. He believes that as artificial intelligence continues to advance, the ideal outcome will be a synergistic relationship between AI capabilities and human expertise, emphasizing the necessity of keeping healthcare providers actively involved in the diagnostic process.

In a Heartbeat: Exploring the crossroads of AI and individual health care

Sunlit modern living room scene: a middle-aged patient gently presses a sleek, skin-toned wearable patch over her heart, its tiny red heart icon softly pulsing. Warm natural light streams through a window onto a wooden table where a tablet shows live vital signs and golden waveform lines rising from the device toward the screen. A doctor’s face appears on the tablet in a caring video-call frame. Subtle potted plant and cozy décor underscore the human-centric fusion of AI and personal healthcare.

In a Heartbeat: Navigating the intersection of AI and personal healthcare - TribLIVE.com

- AI in healthcare can exacerbate self-diagnosis anxiety, leading to potential psychological harm.  
- Doctors' experience allows for nuanced understanding that AI lacks in diagnosis and patient care.  
- While AI can aid in organization, human interpretation remains crucial for accurate healthcare decisions.

The use of artificial intelligence is becoming ubiquitous, but when it comes to health care, determining when and where organic intelligence should drive action could be a lifesaving distinction.

Dr. Jose Francisco Abad, a family medicine physician and certified clinical informatics specialist guiding AI implementation at UPMC, recently parsed the complex and evolving question of AI’s viability in health care.

Q: How can self-diagnosis via the internet, with or without AI, be problematic?

A: I used to say this before AI, but Google was ‘three clicks to death.’ You put in a symptom like a cough and then you look at cough and it says bloody cough. And then, next thing you know, you have lung cancer.

AI is probably going to be an accelerated form of three clicks to death, where it’s going to help you make assumptions about something that you may or may not have without context — which I think has a tremendous amount of psychological harm.

When you order a test or when you give advice to a patient, you’re kind of assuming that that information is going to [provide] space to process the emotion or feeling. So if I order a test and I say, ‘Hey, I’m going to order a test that will let me know if you have X disease.’ Then you prep the patient in understanding what’s possible from the result of that test. You give them reassurance that you’re going to follow up with them depending on the outcome.

The information in and of itself, without that human intervention or without that compassion or empathy, can be very detrimental because you could spiral into obsessive thoughts about something that may or may not come to pass.

Q: What are some ways to determine if an online health source being used by AI is credible?

A: When the AI is saying ‘This is where I’m getting the information’ and it’s from a website that doesn’t have any kind of national recognition or notoriety, you should be really careful about believing what it says.

Whereas if it says something like the CDC, World Health Organization, American Academy of Family Physicians or American Academy of Pediatrics, these are trusted organizations that you know are invested in providing accurate and good care for patients.

A lot of the times when I’ve second-guessed an AI answer, it’s usually because the source is wrong.

Q: How can human input affect AI’s ability to make a diagnosis?

A: AIs can really only make recommendations or give advice based off the information that they’re provided. AI tools like Grok and ChatGPT and Gemini will just answer a question at face value.

There’s an adage that I was taught in high school: “Garbage in, garbage out.” So a calculator, if you say 5 times 5 is 25, but you only have five groups of four apples, the calculator isn’t going to know that the information that you put in was incorrect and is going to give you an answer based on the assumption that it was correct.

Q: How is a doctor better at direct diagnosis?

A: Physicians have context and breadth. Over years of experience and seeing patients and being in different care settings, I can tell the difference between somebody who says my pain is 2 out of 10 and they’re writhing in pain and they can’t stop shaking and they can’t complete a sentence versus somebody who’s calm, serene and looks at you dead in the eye and says I’m in 10 out of 10 pain.

So there’s a lot of nonverbal (information). There’s a lot of experience-based context that changes how information is processed. That context and breadth of experience let you determine other factors that would help you make a decision. It’s completely excluded in a one-person-to-AI-based inquiry.

Q: Why should a health care professional’s words always take prededence over AI?

A: A doctor has spent countless hours and years to be able to give that advice. They’ve passed through college, they’ve passed through medical school, they completed a residency, they got board certified and so they got their medical license after residency. And they have to do continuing medical education, which means their knowledge is tested and their assumptions are kept up to date.

Whereas in AI, there’s nothing. Nobody is testing an AI on its medical knowledge.

Q: How can AI’s functionality affect my results?

A: AI is kind of lazy. It’s going to spend the least amount of energy and the least amount of work to try to get you an adequate answer. And so if you have limited information and you don’t give it much direction and you’re using a fast answer, you’re going to get the quickest thing that takes the least amount of time and expends the least amount of energy.

Q: Is it ever OK to take medical advice from an AI answer? Why or why not?

A: My initial reaction is you can never trust it, because that’s on the assumption that what you’re telling the AI is accurate. So I think you [should] always have a little bit of mistrust. But at the same time, sometimes AIs can be really helpful for people who are worrying about something and don’t need to.

If your question is simple, then I think the AI can be a little bit more trusted. But if your question is complex or you’re not giving enough information, I think you have to take it with a grain of salt and use it to help you make your next decision.

For example, I think it could be used for reassurance if you have a straightforward question like ‘Can I take Tylenol during pregnancy?’ That should be something that the AI would pretty much give you a straight answer that’s commonly regarded to be safe in this scenario.

It can also prompt you to think of things that you weren’t thinking of. If [someone is] asking for help about a medical issue, then it can give a list of things to consider that helps to build a conversation with a doctor who does evaluate them. But it shouldn’t act as a sole source of truth. It needs to act as a tool to guide how to think, and then you need experience and a human to interpret all of that information.

The thing that I would recommend is just to push assumptions with the AI, or write a prompt that is as much detail as possible.

Q: How are some doctors using AI?

A: I use an AI to listen to the conversation [with a patient]. It writes my notes.

But an AI can also think about, ‘What could I be missing? And so it will prompt me and it’ll say, ‘Hey, your patient mentioned that they’re fatigued and they snore loudly when they were talking to you about their cough. Did you want to consider doing a sleep study because these are signs of sleep apnea?’

It’s really helpful because sometimes I get lost in the breadth of all the symptoms that somebody’s giving me. It helps to organize these thoughts and to consider something.

But I don’t go ‘Oh, my patient has sleep apnea’ and just diagnose it. I still have to do a test and ask other questions. It furthers the discussion and the evaluation and the analysis. It cannot be the final evaluation, analysis or definitive source.

Q: Is AI’s application in health care improving?

A: I think as AI continues to learn, it’s going to continue to get smarter and better. The one truth that we always talk about is the computer is not greater than brain and brain is not greater than computer, but brain and computer together are better. Keeping a human in the loop is always going to be important.

In a Heartbeat: Navigating the intersection of AI and personal healthcare

Maine's healthcare system is facing challenges as its aging population encounters intricate health issues and rural hospitals experience workforce shortages. To address these hurdles, state officials are seeking innovative methods to enhance healthcare delivery in rural communities. As articulated in a request for funding from the federal Rural Health Transformation Program, Maine desires "a new approach to delivering rural health care." 

Part of this new strategy may involve incorporating artificial intelligence, although specific implementations are yet to be clarified.

The Rural Health Transformation Program, which amounts to $50 billion over five years, was a last-minute addition from congressional Republicans to President Donald Trump’s Big Beautiful Bill Act last year. This initiative was crafted to mitigate concerns related to proposed cuts of more than $900 billion to Medicaid over the next decade.

In its initial year, Maine is set to receive $190 million from this program. The state intends to allocate these funds to recruit healthcare professionals in rural regions, enhance community-based healthcare, eliminate transportation obstacles, foster regional cooperation, and explore other initiatives. Originally, Maine hoped to use some of the funding to assist hospitals and clinics in covering treatment for low-income and uninsured patients; however, the Centers for Medicare and Medicaid (CMS) has since declined this aspect, as reported by KFF Health News.

Dr. Mehmet Oz, leading CMS, has encouraged states to craft "bold, creative plans," emphasizing the integration of innovative technologies. "President Trump is determined to end the crisis facing rural hospitals, and he’s tasked me with achieving this via AI and telemedicine," Health and Human Services Secretary Robert F. Kennedy Jr. stated before a Senate committee, alluding to concepts like an “AI nurse indistinguishable from a human being.”

Maine's proposal includes a commitment to invest $200 million over five years into technological improvements for rural health services. This investment aims to upgrade electronic medical record systems and expand telehealth offerings. Additionally, there are intentions to develop a Maine Rural AI Hub to aid providers in integrating modern AI technologies, alongside establishing a Rural Health AI Innovation Institute to foster new technology tailored for rural populations. Nonetheless, the proposal lacks extensive details on the specific AI technologies to be pursued.

In an email, a spokesperson for the Maine Department of Health and Human Services, which is managing the program, noted that plans for the AI Hub are still in the early stages and offered no further comments.

AI represents a diverse array of technologies utilizing algorithms and substantial datasets to discern patterns and generate outputs that typically require human cognition. In healthcare, AI applications streamline administrative tasks, like scheduling and note-taking, and enhance diagnostic processes, such as analyzing medical imaging.

However, the adoption of AI in healthcare is not without its challenges. A December policy brief by the National Rural Health Association highlighted concerns regarding data quality, user privacy, and community engagement as significant barriers to successful implementation.

The state's proposal mentions collaborating with Duke University to create the Maine Rural AI Hub, which will provide the necessary expertise and guidance to help rural organizations establish effective governance frameworks for AI integration into their clinical practices and business operations. It also outlines a budget of $500,250 for establishing the Rural Health AI Innovation Institute in its first year.

The Maine Artificial Intelligence Task Force, in its recent report, recommended exploring AI applications in rural healthcare and positioning Maine as an "innovation hub." Their findings indicate that local health systems are already utilizing AI for tasks like ambient documentation and remote monitoring, leading to improved provider retention and more accurate diagnoses.

While Duke University did not respond to requests for further commentary, its Health AI Partnership aims to assist healthcare organizations in effectively and ethically employing AI technologies.

Maine has appointed MCD Global Health, based in Hallowell, as its "technology and innovation partner," as announced in a press release. The organization is currently seeking proposals for approximately $30 million in funding, aiming to help providers upgrade their electronic medical record systems, with additional requests for proposals from digital health and AI companies forthcoming.

Noah Nesin, a family medicine physician and board member of the Maine Public Health Association, expressed mixed feelings about the plans for the Rural AI Hub. "While the promise of AI is substantial, there are significant privacy considerations," he noted. Concerns mainly pertain to the potential use of patients' medical records for training AI systems.

For smaller rural practices lacking extensive resources, AI could be transformative, Nesin suggested, pointing out tools like OpenEvidence, designed to help providers make informed decisions without combing through vast medical literature. AI can also help identify high-risk population groups that need proactive healthcare interventions. Additionally, it can reduce the administrative burden on providers.

Rebecca Boulos, the executive director of the MPHA, echoed the concerns regarding privacy and emphasized the importance of securing patient consent and clarity on data usage and oversight. She also raised points about infrastructure requirements for widespread AI adoption, such as data centers and broadband access. "Discussions about AI in healthcare must address the necessary infrastructure to facilitate its implementation," Boulos stressed.

This article was initially published by The Maine Monitor, an independent news organization. To stay updated with their coverage, consider subscribing to their free newsletter.

Maine to investigate the impact of AI on healthcare in rural areas.

Warm morning sun streams through pine-framed windows of a small Maine clinic, bathing the warm wooden interior in golden light. A nurse holds a tablet showing a friendly AI-powered telehealth dashboard with vital signs and predictive graphs. Across the polished wooden desk, a weathered farmer leans forward with hopeful eyes. Through the window, a red barn and rolling fields under a clear sky complete the rural backdrop.

Maine to explore the role of AI in rural healthcare - Bangor Daily News

- Maine is seeking $190 million to modernize rural healthcare and address workforce shortages.  
- The state plans to establish AI-focused initiatives to improve rural healthcare delivery systems.  
- Privacy and infrastructure concerns are raised as Maine explores AI's role in healthcare.

As Maine’s aging population grapples with complex health needs and rural hospitals struggle with workforce shortages, the state is looking for ways to modernize how healthcare services are offered in rural areas. What Maine needs, state officials wrote in a request for funding from the federal Rural Health Transformation Program, is “a new approach to delivering rural health care.”

This will likely include the use of artificial intelligence, but exactly what that looks like remains to be seen.

The five-year, $50 billion Rural Health Transformation Program was congressional Republicans’ last-minute addition to President Donald Trump’s Big Beautiful Bill Act last summer — an attempt to address the fallout over the bill’s call to cut Medicaid spending by more than $900 billion over 10 years.

Maine is set to receive $190 million from the program in year one and has said it plans to use the money to recruit more healthcare workers in rural areas, strengthen community-based care, reduce transportation-related barriers, support regional collaboration and more. It was also originally hoping to use some of the funds to help hospitals and clinics pay for low-income, uninsured patients’ treatment. But the Centers for Medicare and Medicaid, which oversees the Rural Health Transformation Program, said no, KFF Health News reported.

CMS, with Dr. Mehmet Oz at the helm, has pushed states to use the money for “bold, creative plans,” including the use of new technologies.

“President Trump is determined to end the hemorrhaging of rural hospitals, and he’s asked me to do that through the use of AI, through telemedicine,” Health and Human Services Secretary Robert F. Kennedy Jr. told a Senate committee last year, describing an “AI nurse that you cannot distinguish from a human being.”

Maine’s proposal includes spending $200 million over the five years to improve technology used by rural healthcare providers, such as upgrading electronic medical record systems and expanding telehealth services.

It also mentions plans to “create a Maine Rural AI Hub to support rural providers in adopting today’s AI technologies and establish a Rural Health AI Innovation Institute to promote development of new AI technologies purpose-built for rural populations.”

But it offers few details about what this will entail or what sorts of AI technologies will be pursued.

A spokesperson for the Maine Department of Health and Human Services, which is overseeing the state’s program, declined to comment on the plans for an AI Hub, writing in an email that it is in the “very early phases.”

AI encompasses a broad swath of technologies that use algorithms and large data sets to identify patterns and generate information that would otherwise require human intelligence, from text summaries and audio transcriptions to more complex outputs. It is used in many healthcare settings to streamline administrative work, such as through programs that record appointments and generate notes, and as a diagnostic tool, such as by analyzing X-rays.

“While these advances offer considerable promise, the adoption of AI in healthcare also raises critical concerns,” the authors of a policy brief for The National Rural Health Association wrote in December. “Issues of data quality, limited training data, user privacy, accessibility, ethical and legal questions, and community engagement all present challenges to widespread implementation.”

The state’s proposal calls for the creation of a Maine Rural AI Hub through a partnership with Duke University, saying it will “provide expertise, oversight, and guidance to assist rural healthcare organizations to establish effective local governance models needed to accelerate and manage AI adoption in their clinical and business functions.”

Separately, the proposal calls for establishing a Rural Health AI Innovation Institute, budgeting $500,250 in its first-year plan.

Exploring the use of AI in rural healthcare delivery and establishing Maine as an “innovation hub” were among recommendations the Maine Artificial Intelligence Task Force made in a report published last fall.

“Across the state, health systems are already deploying AI tools such as ambient documentation, remote patient monitoring, and AI-assisted diagnostics, with early results showing gains in provider retention, reduced employee burnout, and more accurate and timely diagnoses,” the report said.

Spokespeople for Duke University did not respond to requests for additional information. On its website, Duke’s Health AI Partnership describes itself as a collaborative that seeks to help healthcare organizations use AI “safely, effectively, and ethically.”

The state has selected Hallowell-based MCD Global Health as its “technology and innovation partner” for the program, according to a July 29 press release. It is currently accepting applications for about $30 million in funding that will go to helping healthcare providers upgrade their electronic medical record systems; its website lists requests for proposals from digital health and AI providers as “coming soon.” A spokesperson for MCD Global Health did not respond to a request for an interview.

Noah Nesin, a family medicine physician who serves on the board of the Maine Public Health Association, learned of the state’s plans for a Rural AI Hub through his work with an accountable care organization that reviewed and gave feedback on Maine’s application. He said there are both pros and cons to consider.

“There are very significant privacy concerns around the use of AI and there’s very dramatic potential around the use of AI,” he said.

Some of these concerns revolve around whether patients’ medical records will be used to train the AI systems.

But for small, rural practices that do not have the benefit of a deep roster of experts or administrative help that a large, urban practice might have, AI could be a boon, Nesin said.

He pointed to an application called OpenEvidence, an AI-powered tool trained on medical literature intended to help providers make decisions without having to search through medical journals.

AI can also identify particular population groups that need more attention from providers — people who are at higher risk of hospitalization, for example, that would benefit from more proactive outreach and intervention. And it can assist providers by cutting down the time they have to spend on administrative tasks, he said.

Rebecca Boulos, MPHA’s executive director, echoed Nesin’s concerns around privacy.

Providers need to get patients’ consent to use AI and be clear with patients about how their data will be used, who will have access to it and what oversight exists, she said, noting that many people are skeptical of AI and skeptical of the healthcare system.

She also raised questions about the potential infrastructure needs that the increased use of AI could require, such as large data centers.

“If you’re going to have a conversation about AI in health care or in public health, it has to be within the context of ‘What do we need to facilitate the use of AI?’” Boulos said. “Whether that infrastructure is broadband or a data center — looking at resource demands, looking at land use, looking at community input and buy-in.”

This story was originally published by The Maine Monitor, a nonprofit and nonpartisan news organization. To get regular coverage from The Monitor, sign up for a free Monitor newsletter here.

Maine to explore the role of AI in rural healthcare

Artificial Intelligence (AI) presents a transformative opportunity for the biopharmaceutical sector, particularly in enhancing efficiency, driving down costs, and improving success rates in drug development. As the global population ages, there is a growing focus on developing treatments for age-related illnesses like Alzheimer’s, which significantly impact healthcare budgets. Traditionally, bringing a new drug to market is a lengthy and expensive process, often requiring 10 to 15 years and an investment ranging from $1 billion to $2.8 billion. Even more daunting is the fact that only around 10% of clinical candidates receive approval.

Sean Laaman, Head of U.S. SMID Cap Biotech Research, highlighted how AI can address these challenges by refining target selection, optimizing molecule design, and enhancing the execution of clinical trials. This shift has the potential to transform drug development from a conventional trial-and-error model into a more data-driven approach that prioritizes candidates with a higher likelihood of success. Anticipated benefits include shortening drug discovery timelines by approximately three years, reducing early-stage R&D costs by 25% to 60%, and cutting preclinical timelines by 30% to 50%. There is also early evidence indicating improved success rates in Phase 1 and 2 trials. However, whether these advantages will extend to later clinical phases remains to be observed in the coming years, which holds significant implications for biotech firms and their investors.

Analyst Terence Flynn noted that AI’s ability to streamline timelines and enhance success rates could lead to a 10% to 40% increase in drug approvals. The industry is also navigating significant changes, with an estimated $160 billion in revenue from major U.S. biopharma companies expected to lose patent protection before 2030. This forecast necessitates new strategic considerations for research and development, investment, and growth trajectories.

Moreover, investment in artificial intelligence continues to significantly influence merger and acquisition (M&A) activities across various sectors, including healthcare. John Collins and Tom Miles, Global Heads of M&A at Morgan Stanley, emphasized this trend in a report on M&A activities for the first half of 2026, stating that acquirers are increasingly pursuing AI-driven capabilities to boost productivity, speed up innovation, and secure a competitive edge. Within healthcare, M&A activity has seen pharmaceutical companies acquire biotech firms to replenish their drug portfolios, representing 11% of the overall M&A volume in the first half of 2026.

Ways AI Cuts Costs in Healthcare and Elder Care

A cozy sunlit clinic room: an elderly patient and a doctor lean over a tablet displaying a green heart icon and a downward‐trending cost graph; nearby, a small stack of medical bills sits partially in shadow, illustrating AI’s role in cutting aging healthcare expenses.

Can AI Reduce the Costs of Healthcare and Aging? - Morgan Stanley

- AI can enhance drug development efficiency, potentially reducing timelines and costs significantly.  
- Improved drug approval rates of 10 to 40% may result from AI advancements.  
- Biopharma companies face revenue challenges as $160 billion in patents expire before 2030.  

3. AI may expedite drug research and development to help support the needs of the aging population.

For the biopharma industry, AI is particularly promising in its opportunities for increasing efficiency, cost savings and success rates. One example is for exploring treatments for diseases occurring later in life, such as Alzheimer’s, that also contribute to healthcare spending. Developing a new drug is time- and capital-intensive, typically taking 10-15 years and more than $1 billion-$2.8 billion in capitalized cost. Furthermore, success rates are low: only about 10% of clinical candidates end up being approved.

“AI directly targets these inefficiencies through better target selection, more efficient molecule design, and smarter clinical trial execution, transforming development from a trial-and-error process into a more data-driven system that prioritizes higher-probability candidates,” Sean Laaman, Head of U.S. SMID Cap Biotech Research, recently noted. Estimated improvements include reductions in discovery timelines by ~3 years, early-stage R&D costs by ~25-60% and preclinical timelines by ~30-50%, as well as “early evidence of improved Phase 1-2 success rates.” While AI offers the potential for faster and cheaper drug development, it remains to be seen over the next several years whether it can also translate into success in Phase 2 and beyond, with implications for both biotech companies and investors.

If AI is able to improve timelines and the likelihood of success for a drug, it “has the potential to increase drug approvals by 10 to 40%,” U.S. Biopharma Analyst Terence Flynn said on Thoughts on the Market.

The industry is also facing changes as some $160 billion in revenue across U.S. large-cap biopharma companies is slated to go off patent prior to 2030, according to Morgan Stanley Research estimates, presenting new considerations for companies’ R&D, investor and growth strategies.

Across multiple sectors, including healthcare, “artificial intelligence investment continues to be a central theme shaping M&A activity,” John Collins and Tom Miles, Global Heads of M&A at Morgan Stanley, recently noted in a 1H2026 M&A report. “Acquirers are actively targeting AI-enabled capabilities to enhance productivity, accelerate innovation and build competitive advantage.” Healthcare M&A activity has featured biotech acquisitions by pharmaceutical companies as they aim to refill their drug pipelines. Healthcare transactions made up 11% of M&A volume in 1H2026, according to the report.

How Does AI Reduce Costs in Healthcare and Aging

As Maine faces the challenges of an aging populace with complex health needs and rural hospitals grappling with workforce shortages, state officials are actively seeking innovative solutions to modernize healthcare delivery in the region. In a bid for funding from the federal Rural Health Transformation Program, they emphasized the need for “a new approach to delivering rural health care,” which may incorporate advancements in artificial intelligence, though specific applications are still under development.

This five-year, $50 billion initiative was included by congressional Republicans as a late addition to President Donald Trump's Big Beautiful Bill Act last summer, aimed at addressing concerns over proposed cuts to Medicaid spending, which could exceed $900 billion over a decade.

Maine is set to receive $190 million during the first year, with plans to allocate these funds towards recruiting healthcare professionals in rural areas, enhancing community-based care, mitigating transportation obstacles, fostering regional collaboration, and more. Originally, some of the funds were intended to assist hospitals and clinics in covering treatment costs for uninsured, low-income patients, but the Centers for Medicare and Medicaid Services (CMS) rejected this proposal, as reported by KFF Health News.

CMS, led by Dr. Mehmet Oz, is encouraging states to utilize these resources for innovative strategies, emphasizing the integration of new technologies. Health and Human Services Secretary Robert F. Kennedy Jr. articulated last year to a Senate committee the intent to curb the crisis facing rural hospitals through AI and telemedicine, also referencing the futuristic concept of an “AI nurse” indistinguishable from a human.

Maine's strategy involves a $200 million investment over five years aimed at enhancing technology for rural healthcare providers, which includes upgrading electronic medical record systems and expanding telehealth capabilities. The vision also proposes the creation of a Maine Rural AI Hub to facilitate the adoption of contemporary AI technologies and the establishment of a Rural Health AI Innovation Institute to encourage the development of AI tailored for rural communities. However, the specifics remain vague regarding what technologies will be prioritized.

A representative from the Maine Department of Health and Human Services, which is spearheading the initiative, declined to discuss the AI Hub’s detailed plans, indicating that they are still in the “very early phases” of development.

AI encompasses a wide array of technologies that harness algorithms and extensive data sets to identify trends and produce insights that typically require human intelligence, ranging from simple text summaries and audio transcriptions to complex data analyses. In healthcare, AI is already streamlining administrative tasks, including appointment scheduling and note-taking, and serves as a diagnostic tool in areas like X-ray interpretation.

Despite the potential benefits, the implementation of AI in healthcare brings forth significant challenges, as highlighted by a policy brief from The National Rural Health Association. Issues concerning data quality, limited training data, user privacy, accessibility, and ethical as well as legal ramifications must be addressed to achieve broad adoption.

Maine's proposal aims to develop the Maine Rural AI Hub in collaboration with Duke University, which would provide expertise and oversight to assist rural healthcare organizations in establishing efficient governance models for AI integration in both clinical and administrative functions. Additionally, the plan includes the establishment of the Rural Health AI Innovation Institute, with an initial budget of $500,250.

The exploration of AI in rural healthcare and the goal of positioning Maine as an “innovation hub” were among the key recommendations from the Maine Artificial Intelligence Task Force's report released last fall. The report noted that various health systems in the state have already begun to employ AI tools for tasks like ambient documentation and remote patient monitoring, with promising early results in enhancing clinician retention and improving diagnostic accuracy.

Maine has chosen MCD Global Health, based in Hallowell, as its technology and innovation partner for this initiative. The organization is currently seeking applications for around $30 million in funding aimed at helping healthcare providers upgrade their electronic medical record systems, with further details on proposals for digital health and AI solutions forthcoming.

Noah Nesin, a family medicine physician involved with the Maine Public Health Association, is aware of the plans for the Rural AI Hub through his association with an accountable care organization that provided input on the application. He recognized both the potential advantages and significant privacy concerns associated with the deployment of AI in healthcare, particularly regarding how patient data might be utilized to train these systems.

Nonetheless, for smaller rural practices that lack the resources of larger urban counterparts, AI could offer significant advantages. He cited AI applications like OpenEvidence, which pull from medical literature to assist providers in making informed decisions without excessive time spent on research. AI could also help identify at-risk populations requiring more proactive care, thus improving patient outreach and intervention.

Rebecca Boulos, the executive director of the Maine Public Health Association, echoed concerns about privacy, emphasizing the necessity for providers to secure patient consent before employing AI tools and to maintain transparency regarding data usage, access, and oversight. She also raised critical considerations about the infrastructure demands associated with increased AI utilization, such as the need for robust data centers and broadband connectivity.

“If discussions about AI in healthcare or public health are to be productive, they must address the infrastructure required to support such technologies,” Boulos asserted, highlighting the importance of evaluating resource needs, land use, and community engagement in the process.

Maine to investigate the impact of AI on health care in rural areas

A sunlit rural Maine clinic with warm wooden walls: a female doctor at a simple desk reviews AI-powered diagnostic charts glowing gently on her tablet as an adult farmer in a plaid shirt leans forward attentively. Through a large window, evergreen pines and rolling hills bathed in morning light underscore the promise of AI-enhanced health care in the countryside.

Maine to explore the role of AI in rural health care - The Maine Monitor

- Maine plans to modernize rural health care using $190 million from a federal program.  
- Proposal includes a Rural AI Hub and upgraded technology for rural health providers.  
- Adoption of AI raises privacy concerns, but offers potential benefits for rural practices.  

As Maine’s aging population grapples with complex health needs and rural hospitals struggle with workforce shortages, the state is looking for ways to modernize how health care services are offered in rural areas. What Maine needs, state officials wrote in a request for funding from the federal Rural Health Transformation Program, is “a new approach to delivering rural health care.”

This will likely include the use of artificial intelligence, but exactly what that looks like remains to be seen.

The five-year, $50 billion Rural Health Transformation Program was congressional Republicans’ last-minute addition to President Donald Trump’s Big Beautiful Bill Act last summer –– an attempt to address the fallout over the bill’s call to cut Medicaid spending by more than $900 billion over 10 years.

Maine is set to receive $190 million from the program in year one and has said it plans to use the money to recruit more health care workers in rural areas, strengthen community-based care, reduce transportation-related barriers, support regional collaboration and more. It was also originally hoping to use some of the funds to help hospitals and clinics pay for low-income, uninsured patients’ treatment. But the Centers for Medicare and Medicaid, which oversees the Rural Health Transformation Program, said no, KFF Health News reported.

CMS, with Dr. Mehmet Oz at the helm, has pushed states to use the money for “bold, creative plans,” including the use of new technologies.

“President Trump is determined to end the hemorrhaging of rural hospitals, and he’s asked me to do that through the use of AI, through telemedicine,” Health and Human Services Secretary Robert F. Kennedy Jr. told a Senate committee last year, describing an “AI nurse that you cannot distinguish from a human being.”

Maine’s proposal includes spending $200 million over the five years to improve technology used by rural health care providers, such as upgrading electronic medical record systems and expanding telehealth services.

It also mentions plans to “create a Maine Rural AI Hub to support rural providers in adopting today’s AI technologies and establish a Rural Health AI Innovation Institute to promote development of new AI technologies purpose-built for rural populations.”

But it offers few details about what this will entail or what sorts of AI technologies will be pursued.

A spokesperson for the Maine Department of Health and Human Services, which is overseeing the state’s program, declined to comment on the plans for an AI Hub, writing in an email that it is in the “very early phases.”

AI encompasses a broad swath of technologies that use algorithms and large data sets to identify patterns and generate information that would otherwise require human intelligence, from text summaries and audio transcriptions to more complex outputs. It is used in many health care settings to streamline administrative work, such as through programs that record appointments and generate notes, and as a diagnostic tool, such as by analyzing X-rays.

“While these advances offer considerable promise, the adoption of AI in health care also raises critical concerns,” the authors of a policy brief for The National Rural Health Association wrote in December. “Issues of data quality, limited training data, user privacy, accessibility, ethical and legal questions, and community engagement all present challenges to widespread implementation.”

The state’s proposal calls for the creation of a Maine Rural AI Hub through a partnership with Duke University, saying it will “provide expertise, oversight, and guidance to assist rural healthcare organizations to establish effective local governance models needed to accelerate and manage AI adoption in their clinical and business functions.”

Separately, the proposal calls for establishing a Rural Health AI Innovation Institute, budgeting $500,250 in its first-year plan.

Exploring the use of AI in rural health care delivery and establishing Maine as an “innovation hub” were among recommendations the Maine Artificial Intelligence Task Force made in a report published last fall.

“Across the state, health systems are already deploying AI tools such as ambient documentation, remote patient monitoring, and AI-assisted diagnostics, with early results showing gains in provider retention, reduced employee burnout, and more accurate and timely diagnoses,” the report said.

Spokespeople for Duke University did not respond to requests for additional information. On its website, Duke’s Health AI Partnership describes itself as a collaborative that seeks to help health care organizations use AI “safely, effectively, and ethically.”

The state has selected Hallowell-based MCD Global Health as its “technology and innovation partner” for the program, according to a July 29 press release. It is currently accepting applications for about $30 million in funding that will go to helping health care providers upgrade their electronic medical record systems; its website lists requests for proposals from digital health and AI providers as “coming soon.” A spokesperson for MCD Global Health did not respond to a request for an interview.

A screenshot from MCD Global Health’s website.

Noah Nesin, a family medicine physician who serves on the board of the Maine Public Health Association, learned of the state’s plans for a Rural AI Hub through his work with an accountable care organization that reviewed and gave feedback on Maine’s application. He said there are both pros and cons to consider.

“There are very significant privacy concerns around the use of AI and there’s very dramatic potential around the use of AI,” he said.

Some of these concerns revolve around whether patients’ medical records will be used to train the AI systems.

But for small, rural practices that do not have the benefit of a deep roster of experts or administrative help that a large, urban practice might have, AI could be a boon, Nesin said.

He pointed to an application called OpenEvidence, an AI-powered tool trained on medical literature intended to help providers make decisions without having to search through medical journals.

AI can also identify particular population groups that need more attention from providers –– people who are at higher risk of hospitalization, for example, that would benefit from more proactive outreach and intervention. And it can assist providers by cutting down the time they have to spend on administrative tasks, he said.

Rebecca Boulos, MPHA’s executive director, echoed Nesin’s concerns around privacy.

Providers need to get patients’ consent to use AI and be clear with patients about how their data will be used, who will have access to it and what oversight exists, she said, noting that many people are skeptical of AI and skeptical of the health care system.

She also raised questions about the potential infrastructure needs that the increased use of AI could require, such as large data centers.

“If you’re going to have a conversation about AI in health care or in public health, it has to be within the context of ‘What do we need to facilitate the use of AI?’” Boulos said. “Whether that infrastructure is broadband or a data center –– looking at resource demands, looking at land use, looking at community input and buy-in.”

Maine to explore the role of AI in rural health care

Lawmakers in Wisconsin are collaborating with specialists in artificial intelligence (AI) and healthcare to explore the technology's integration into the medical sector.

Recently, a legislative study committee held its first informational gathering at the State Capitol, focused on understanding how AI is utilized in relationships between patients and providers, as well as in interactions between patients and insurers.

The committee aims to gather insights that will inform future legislation aimed at setting standards for AI applications in healthcare and insurance services. 

Samantha Scotti, representing the National Conference of State Legislatures, was the first expert to present her insights to the committee. She highlighted that her organization has been monitoring the increasing utilization of AI across various facets of healthcare, ranging from administrative roles such as appointment scheduling and claims processing to clinical functions like drafting post-visit notes and aiding in diagnoses.

With no federal regulations currently in place, Scotti emphasized the proactive role that state governments are taking to implement guidelines for this evolving technology. “Policymakers throughout the U.S. are focusing on safeguarding patient safety and privacy, ensuring transparency in AI applications, and determining the necessary level of oversight,” she noted. Impressively, this year alone has seen over 285 bills related to AI introduced across 44 states.

Wisconsin has yet to establish any legislation regarding AI in healthcare, prompting the formation of the study committee. State Representative Adam Neylon, a Republican from Pewaukee, expressed concerns about overly restrictive policies, stating, “I’ve seen a plethora of subpar proposals driven by fear. We need to consider how much bureaucracy we impose on healthcare providers and businesses.”

On the other hand, State Senator Sarah Keyeski, a Democrat from Lodi, pointed out the necessity for lawmakers to agree on a clear definition of AI before crafting any bills. “We need to address what AI really means, especially since the technology is continuously evolving,” she remarked.

Some citizen members of the committee echoed the sentiment that establishing a clear definition is crucial for formulating effective legislative measures. Phillip Blair, Vice President of Human Resources at Gamber-Johnson, noted the importance of a precise definition, stating, “AI is intricate and has existed longer than many recognize, making a solid definition vital for our initiatives."

David Hoffert, Director of AI Policy at Epic Systems Corporation, added, “It's essential to frame our discussions around the specific concerns that arise from AI so that our definition aligns with those worries.”

Chairing the committee, State Senator Rachael Cabral-Guevara from Fox Crossing announced that they plan to hold five additional meetings before introducing any legislative recommendations.

Legislative committee examining the application of AI in healthcare

A bright, wood-paneled legislative hearing room flooded with natural sunlight; a semicircular table of diverse lawmakers in suits lean in as a medical expert stands beside a large screen showing a stylized human silhouette overlaid with AI circuit nodes and a heartbeat icon; tablets, stethoscopes, and open policy papers scattered across the polished wood surface; tall windows reveal a city skyline beyond—capturing the moment of focused debate on AI in healthcare.

Legislative committee studying use of AI in healthcare - Spectrum News

- Wisconsin lawmakers are studying AI's impact on healthcare to inform future legislation.  
- The committee will examine AI's role in patient-provider and insurer-patient interactions.  
- Key discussions include defining AI and ensuring patient safety and privacy standards.  

MADISON, Wis. — State lawmakers are joining experts in both AI and healthcare to learn more about how the technology is being used in the medical field.

What You Need To Know State lawmakers are joining experts in both AI and healthcare to learn more about how the technology is being used in the medical field



A legislative study committee on the use of AI in healthcare met for its initial informational hearing at the State Capitol Thursday



The committee is tasked with learning about the use of AI in the patient-provider relationship, as well as in interactions between insurers and patients



At the end of the study, legislation will be proposed to address standards for the use of AI in healthcare services and insurance

A legislative study committee on the use of AI in healthcare met for its initial informational hearing at the State Capitol Thursday.

The committee is tasked with learning about the use of AI in the patient-provider relationship, as well as in interactions between insurers and patients.

At the end of the study, legislation will be proposed to address standards for the use of AI in healthcare services and insurance.

The first presentation to the committee came from Samantha Scotti, who represents the National Conference of State Legislatures.

Scotti said her organization has been tracking the growing use of AI in several different aspects of the healthcare industry.

“From administrative tasks like scheduling appointments to processing claims, to functions in clinical settings like drafting after visit notes or helping with diagnoses," Scotti said.

With the absence of any federal regulations, Scotti said state governments are stepping in to put guardrails around the technology.

“Policymakers across the U.S. are considering how to protect patient safety and patient privacy, ensure transparency in the use of AI, and just generally determine the appropriate level of oversight," Scotti said.

Scotti said this year alone, more than 285 bills on the topic were introduced across 44 different states.

Wisconsin has yet to pass any policy on AI in health care, which is what led to the creation of the study committee.

State Rep. Adam Neylon, R-Pewaukee, believes Wisconsin shouldn't be as heavy-handed as some of the proposals put forward by other states.

“I saw a lot of bad policy up there and a lot of bad ideas, and a lot of fear," Neylon said. "How much red tape do we need to give these providers and these businesses.”

State Sen. Sarah Keyeski, D-Lodi, said before any bills are drafted, lawmakers need to first come to an agreement on how best to define AI.

“That might be something that we need to grapple with up front before we get into more of the nuances, and given the fact that it’s always changing, what does that mean as we try to define it," Keyeski said.

Some of the citizen members of the committee agree that the definition needs to be the starting point for building effective legislation.

“AI is so complex, and it’s been around for a lot longer than many people realize, so a concrete definition is going to be critical to anything we do," said Phillip Blair, VP of Human Resources with Gamber-Johnson.

“What is it about AI that is different and therefore scary, I think that’s an important framing for us to consider is what is the concern, and therefore what definition matches that concern," said David Hoffert, Director of AI Policy with Epic Systems Corporation.

State Sen. Rachael Cabral-Guevara, R-Fox Crossing, is leading the committee, and said there will be five more meetings before any legislative proposals are made.

Legislative committee studying use of AI in healthcare

Raman Kaur, AGNP, MS, serves as the Vice President of Technology at Elsevier and recently appeared on Cancer Nursing Today to share insights on the integration of artificial intelligence (AI) within oncology nursing. She also discussed findings from Elsevier's fifth annual Clinician of the Future report, which reveals how nurses feel about AI aiding their decision-making processes in clinical scenarios.

During the third part of her interview, Kaur emphasized the important role oncology nurses have in guiding AI's development and application in the healthcare sector, particularly as AI emerges as a pivotal tool in point-of-care settings. The insights nurses provide are invaluable for healthcare systems in adopting AI in thorough, sensible, and inventive ways.

For a deeper understanding, you can watch the video above or examine the transcript provided below, which has been edited for clarity.

TRANSCRIPT:

Hello, I’m Raman Kaur, Vice President of Technology at Elsevier in the Health Education Department, a division of our Health Markets Business. My focus is on steering technology strategies and innovations across platforms related to nursing, health, and medical education. I bring a strong clinical background as an adult gerontology nurse practitioner, having worked on the frontlines in acute care and outpatient environments.

As AI becomes an essential ally in clinical settings, what should be the involvement of nurses in directing how this technology is applied in the future?

In reviewing the entire report, there's a significant consensus that AI is not intended to replace clinicians. Approximately 80% of nurses believe AI will evolve into a crucial assistant within the next 5 to 10 years—possibly even sooner. Personally, I have witnessed AI effectively assist in various lower-stakes tasks in my life, and I envision a future where AI becomes a reliable partner in clinical workflows. This highlights the importance of nurses actively participating in the design and rollout of AI technologies in healthcare workflows. Nurses have an in-depth understanding of the daily realities of patient care, arguably more so than any other team members. Their extensive documentation spans numerous aspects of a patient's healthcare journey, alongside their educational and monitoring responsibilities. This comprehensive role positions nurses to identify where AI can be genuinely beneficial and where it might introduce complications. 

Moreover, the report points out a disconnect, as many nurses feel their voices are absent from crucial decision-making discussions. It’s essential to recognize those perspectives that will significantly influence healthcare system decisions regarding the right tools and clinical insights required. Their input is vital for the successful implementation of AI tools, but it doesn't end there—nursing perspectives are also crucial in ongoing evaluations and adjustments of AI technologies. Having worked within these workflows and now leading a healthcare technology initiative, I believe nurses are uniquely qualified to assess AI pragmatically. They don't simply consider time savings; they also question whether AI reduces clicks, enhances quality, and improves patient explanations. Trust in AI builds on its utility and the relevant support it can offer to nurses. Engaging nursing voices early in the design process and identifying advocates within the nursing team can significantly enhance the overall AI offerings, leading to higher adoption rates and growing trust.

What is the importance of nurses participating in the evaluation and trust-building processes for AI tools in clinical settings?

There are two key points. First, to cultivate trust in AI systems, it’s insufficient to merely provide them for clinician use and expect them to self-train. Effective training for clinicians is necessary, accompanied by robust governance that includes continual adjustments and evaluations of the AI technologies, along with safeguards that prioritize clinical judgment over replacement—this is vital. Second, for AI systems to yield substantial benefits, nurse clinicians need to be integral to shaping the evaluation process, determining which tools are adopted, and how those tools evolve. Their insights are crucial for ensuring that the tools are practical and comprehensive, addressing real-world bedside situations and the daily support patients require, rather than focusing solely on overarching care objectives.

Raman Kaur, AGNP, MS, Emphasizes the Role of Nurses as Essential Designers in the Future of AI in Healthcare

A bright, sunlit hospital innovation lab: a poised nurse wearing a lab coat and scrubs, her ID badge reading “Raman Kaur, AGNP, MS,” stands at a large glass board. Soft morning light streams through windows, illuminating her hand-drawn neural network and medical icon diagrams sketched in warm marker tones. Two fellow nurses stand nearby with tablets and notebooks, watching thoughtfully. In the background, beds and medical monitors hint at a real clinical setting, conveying a collaborative spirit of nurses as the architects of AI-driven healthcare.

Shaping the Future of AI in Healthcare: Raman Kaur, AGNP, MS, Highlights Nurses As Key Architects - Cancer Nursing Today

- Nurses overwhelmingly believe AI will become a crucial clinical assistant in healthcare.  
- Their insights can improve AI implementation and address potential friction points effectively.  
- Involvement of nurses is essential for trust and evaluation of AI tools in practice.

Raman Kaur, AGNP, MS, vice president of technology at Elsevier, joined Cancer Nursing Today to discuss oncology nursing insights on AI adoption in the clinical setting, as well as what Elsevier’s fifth annual Clinician of the Future report reveals about nurses’ sentiments towards AI decision-making in their day-to-day practice.

In the third segment of this conversation, Kaur highlighted how, as AI becomes a key assistant for point of care, oncology nurses play a valuable role in shaping how the technology is implemented across the healthcare landscape, and how nursing perspectives can help healthcare systems develop and use AI in comprehensive, pragmatic, and innovative ways.

Watch the video above and read the transcript below to learn more.

This transcript has been edited for clarity.

TRANSCRIPT:

My name is Raman Kaur, and I am Vice President of Technology at Elsevier in the Health Education Department. It is part of the Health Markets Business at Elsevier. I lead technology strategy and innovation across nursing, health, and medical education platforms. Per my background, I am an adult gerontology nurse practitioner, and I’ve had frontline clinical experience in acute care and outpatient settings as well.

As AI becomes a critical assistant for point of care, what role do you think nurses should play in shaping how AI is implemented across the healthcare landscape in the future?

If you look at the report in its entirety, there’s a strong agreement that AI isn’t about replacing clinicians. About 80% of nurses say that AI will become a critical assistant in the next 5 to 10 years. It may not even take the entire decade. I can certainly say that AI is proving to be a great assistant in many other lower-stake tasks in my life, and I would love to see a future, which I think we’re not that far from, having AI be a trustworthy assistant in clinical workflows. To your point, that’s exactly why nurses should have that meaningful role in shaping AI and how AI is introduced across different healthcare workflows. Nurses single-handedly understand the day-to-day reality of patient care, in my humble opinion, the most out of the interdisciplinary team. They have a lot of documentation that spans across different types and parts of the healthcare journey that the patient has, in addition to education and monitoring. The role is quite comprehensive and overarching. Nurses can speak to where AI will truly be helpful and where it may create friction. Furthermore, the report does a great job of showing that many nurses are not feeling that they’re fully represented in that decision-making process. There’s a real opportunity in identifying those voices that will be very critical as the decision-makers in healthcare systems identify the right tool and the right type of clinical acumen the tool needs to have. Those voices will help the implementation of these AI tools become more successful. That doesn’t stop there. Having the voice of the nurse clinician in further evaluations of accuracy and in tuning of the AI tools is also equally important. To give a little bit of a personal touch, in my humble opinion as a clinician, and formerly being part of these workflows and now leading a healthcare technology team, nurses are single-handedly very equipped to evaluate AI practically with the right level of pragmatism. They don’t just look at the time saved. They ask very important questions like, “Is it reducing the number of clicks? Is it saving time with the quality that I’m asking for? Does it help me explain something better to a patient than what I could do otherwise?” Trust also comes with the usefulness and the appropriate support they can provide to a nurse clinician. So, bringing their perspective early in the process and identifying those AI-forward voices, those stewards in your team, will enhance the comprehensive nature of the AI offering to a degree where you don’t have to wonder if it’s going to be successful. You will probably be looking at the adoption, and the trustworthiness will grow automatically.

Why is it critical for nurses to get involved in shaping the evaluation of AI tools, as well as building trust around its use in the clinical setting?

Two points. If we’re looking to build trust in the AI systems, it’s going to be more than just giving it to the clinicians to use and allowing them to train it. It requires practical training that clinicians need to have in order to be effective. Clear governance, including continuous tuning and training of the actual AI tool itself, ongoing evaluation, and then safeguards that support clinical judgment rather than trying to replace it, that’s number 1. Number 2, if the healthcare systems want AI to deliver meaningful value, if they want to see the benefits that the AI is promising, then nurse clinicians absolutely need to be a part of shaping how it’s evaluated, which tool is introduced, and how we continue to improve the tool. Their perspective is essential to making the offering pragmatic and comprehensive and making sure that the tool is reflecting the reality of the bedside and the support that patients need on a day-to-day basis, not just an overarching goal of care.

AI in healthcare: Is a chatbot capable of responding to your medical inquiries?

More from AI in healthcare

Heidi developed scalable, production-ready AI for the healthcare industry.

Healthcare Is Introducing AI Solutions — It’s Not Prepared for AI Partners

Government Healthcare Agencies' Use of AI: Is It Beneficial for the Public?

Gen Z seeks AI-resistant healthcare careers—chiropractors, doctors, and paramedics report the lowest job satisfaction.