A newly proposed bill mandates that healthcare decision-making be firmly under the authority of doctors or nurses, even in situations where artificial intelligence (AI) plays a role in guiding care. Additionally, the legislation prohibits healthcare providers from utilizing AI for tasks that are legally required to be performed by licensed professionals.

This bill was backed by the California Nurses Association, the same organization whose members recently protested at Kaiser hospitals due to concerns regarding the implementation of AI technology.

Opposition to the bill came from various hospital organizations, health plans, and technology firms. They argued that the bill's broad definition of AI could encompass tools that are already regulated and commonly used in the healthcare sector.

Last week, the Legislature approved the bill, which will become law in January, pending the signature of Governor Newsom.

This legislation may impose restrictions on AI in healthcare.

A dimly lit legislative chamber with warm tungsten light casting long shadows over a heavy wooden desk. On the desk, a polished gavel lies next to a rolled-up bill titled “AI in Healthcare Act.” A sleek metallic robotic arm clutching a stethoscope is ensnared in bright red tape. Behind the desk, a single U.S. flag stands solemnly, its fabric catching the faint glow.

This bill could put limits on AI in healthcare - KALW

- The bill mandates doctors or nurses to make final decisions on patient care.  
- California Nurses Association sponsored the bill amid protests against AI use in hospitals.  
- Hospital groups oppose the bill, arguing its AI definition could hinder existing technologies.  

The bill would require a doctor or nurse to have the final say in a patient's care, even when an AI system helps guide it. It would also bar health providers from using AI to do work the law says must be done by a licensed professional.

The bill was sponsored by the California Nurses Association, the same union whose members picketed Kaiser hospitals this week over the use of AI.

Hospital groups, health plans, and tech companies opposed the bill. They say its definition of AI is too broad, and could sweep in tools already regulated and widely used.

The Legislature passed the bill last week. The law takes effect in January if Governor Newsom signs it.

This bill could put limits on AI in healthcare

Facebook has introduced a new feature designed to enhance user experience and engagement on its platform. This update aims to make interactions more meaningful by allowing users to connect more deeply with their friends and interests. With a focus on fostering community, the platform is rolling out tools that encourage users to share their thoughts and experiences more freely. 

Additionally, the changes include an improved interface that makes it easier to navigate and discover content that aligns with users' passions. By leveraging advanced algorithms, Facebook will curate personalized feeds, ensuring that relevant posts and updates are front and center. 

This initiative also underscores Facebook's commitment to user privacy and data protection. The new features are developed with transparency in mind, empowering users to manage their settings confidently and decide what to share, with whom, and when.

Overall, these enhancements reflect Facebook's ongoing effort to adapt to the evolving needs of its community, promoting richer interactions while maintaining a safe and user-friendly environment.

Worries over AI in healthcare, Addressing conspiracies: PD Letters

A shadowy clinic office at night: a physician’s hands hold an open envelope stamped “PD Letters.” The desk is strewn with typed pages scrawled with red “Conspiracy” marks and doodled warning symbols. In the corner, a silent AI diagnostic unit looms like a dark machine, its cables snaking like vines. A single warm desk lamp illuminates the letters, while the rest of the room fades into deep shadows.

Concern about AI in healthcare, Calling out conspiracies: PD Letters - The Press Democrat

- Facebook introduces new privacy controls to enhance user data protection and transparency.
- Users can now manage ad preferences directly from their profiles for better customization.
- The platform aims to improve user trust and safety amid increasing regulatory scrutiny.

Share on Facebook (Opens in new window)

Facebook

Concern about AI in healthcare, Calling out conspiracies: PD Letters

David Shaywitz, an Adjunct Fellow, shared insights on the role of artificial intelligence in healthcare during his appearance on NewsNation’s podcast, ‘Hot Take with Jesse Weber.’ He explored the reliability of AI technology when it comes to making crucial medical decisions, weighing the potentials and limitations it presents within the healthcare sector. Shaywitz’s discussion sheds light on whether AI can be trusted to play a significant role in patient care and clinical outcomes.

Shaywitz Talks about AI in Healthcare on NewsNation's 'Hot Take with Jesse Weber' Podcast

A bright podcast studio bathed in warm natural light, with Dr. Shaywitz and host Jesse Weber seated at a wooden table, speaking into vintage-style microphones. On Dr. Shaywitz’s laptop screen, a softly glowing neural-network graphic in amber tones hints at AI in medicine. Behind them, a large NewsNation logo is prominently displayed on a textured wall, and subtle medical symbols—a stethoscope and caduceus—are integrated into the décor.

Discussing AI in Healthcare: Shaywitz on NewsNation's 'Hot Take with Jesse Weber' Podcast - American Enterprise Institute - AEI

- David Shaywitz explores the reliability of AI technology in making medical decisions.  
- He discusses the potential benefits AI brings to healthcare decision-making processes.  
- The podcast delves into concerns surrounding AI's effectiveness in critical situations.  

Adjunct Fellow David Shaywitz discusses AI in healthcare and if the technology is reliable enough to make critical medical decisions on NewsNation’s ‘Hot Take with Jesse Weber’ Podcast.

Discussing AI in Healthcare: Shaywitz on NewsNation’s ‘Hot Take with Jesse Weber’ Podcast

Sending information to us via email does not create an attorney-client relationship and does not place any confidentiality obligations on our law firm or the lawyer receiving the information. By clicking "OK," you confirm your understanding that we are not required to keep any information you provide confidential unless we already represent you or have mutually agreed to accept limited confidential information as part of a prospective client relationship. Therefore, if you are not currently a client or someone we are considering as a potential client, any information sent to us via email could be disclosed to third parties or potentially used against you.

If you're interested in discussing how to become a client, we encourage you to reach out to one of our attorneys to set up a meeting or phone call. Should you need to share confidential information with one of our lawyers before establishing an attorney-client relationship, we recommend discussing the confidentiality protections the firm can offer for that information prior to its submission. Thank you for considering Foley Hoag.

NEHEN: Looking Past the Hype: Integrating AI in Healthcare

A bright hospital consultation room flooded with warm morning sunlight. A doctor in scrubs holds a sleek tablet displaying patient vital signs overlaid with subtle digital neural‐network lines; the NEHEN logo appears prominently on the screen. In the softly lit background, a colleague reviews charts by a window and modern medical equipment lines the walls, conveying a seamless integration of AI into healthcare.

NEHEN: Beyond the Hype: Implementing AI Across Healthcare - Foley Hoag

- Emailing information does not create an attorney-client relationship or confidentiality obligation.  
- Information submitted may be disclosed if not a current or prospective client.  
- Discuss confidentiality protections before sharing sensitive information with attorneys.

Transmitting information to us by e-mail unilaterally does not establish an attorney-client relationship or impose an obligation on either the law firm or even the receiving lawyer to keep the transmitted information confidential. By clicking "OK," you acknowledge that we have no obligation to maintain the confidentiality of any information you submit to us unless we already represent you or unless we have agreed to receive limited confidential material/information from you as a prospective client. Thus, if you are not a client or someone we have agreed to consider as a prospective client, information you submit to us by e-mail may be disclosed to others or used against you.

If you would like to discuss becoming a client, please contact one of our attorneys to arrange for a meeting or telephone conference. If you wish to disclose confidential information to a lawyer in the firm before an attorney-client relationship is established, the protections that the law firm will provide to such information from a prospective client should be discussed before such information is submitted. Thank you for your interest in Foley Hoag.

OK

Cancel

NEHEN: Beyond the Hype: Implementing AI Across Healthcare

In a recent analysis, JPMorgan's Rajiv Batra has expressed a positive outlook on the performance of emerging-market benchmarks, suggesting that they demonstrate reduced sensitivity to interest rate fluctuations. He emphasizes that these markets are increasingly aligned with sectors such as artificial intelligence (AI) and healthcare. Batra underlines his continued optimism regarding AI investments, even in the face of challenges related to safety, regulatory scrutiny, and financing issues. His insights suggest a strategic shift in how investors might approach these sectors, advocating for the potential benefits of integrating AI and healthcare into their portfolios despite the prevailing economic uncertainties.

JPMorgan: Emerging market benchmarks are less sensitive to interest rates, prefer AI and healthcare sectors

A bright, modern trading floor bathed in warm morning sunlight through floor-to-ceiling windows. In the center, a large JPMorgan logo is etched on a glass pillar. Below it, a sleek digital ticker shows emerging-market benchmarks flattening as two rising charts glow: one stylized with a neural-network motif, the other bearing a subtle medical caduceus. In the corner, an interest-rate gauge sits nearly idle, its needle barely moved—visually narrating the move toward AI and healthcare.

JPMorgan: EM benchmarks less rate-sensitive, favor AI and healthcare - CNBC

- JPMorgan's Rajiv Batra highlights emerging-market benchmarks' resilience to rate changes.  
- He believes equities and higher bond yields can thrive simultaneously.  
- Batra remains optimistic about AI and healthcare despite associated risks.

Share Share Article via Facebook Share Article via Twitter Share Article via LinkedIn Share Article via Email

JPMorgan: EM benchmarks less rate-sensitive, favor AI and healthcare

JPMorgan’s Rajiv Batra says higher bond yields and equities can coexist, arguing that emerging-market benchmarks are less rate-sensitive and more geared toward AI services and healthcare. He explains why remains bullish on AI despite safety, regulatory and financing concerns.

JPMorgan: EM benchmarks less rate-sensitive, favor AI and healthcare

Suki, a leading platform in Ambient Clinical Intelligence (ACI), has joined forces with MedStar Health’s National Center for Human Factors in Healthcare to embark on a groundbreaking research initiative focused on the integration of ambient AI in healthcare delivery. This partnership aims to delve into the effects of this technology, marking a significant advancement in understanding how ambient AI can reshape clinical environments.

Historically, the healthcare sector has concentrated on accelerating the adoption of new technologies. However, many organizations have overlooked the critical science of human factors essential for assessing the success of technology within intricate clinical settings. This collaboration will leverage MedStar Health’s extensive research on the application of AI in patient care while applying comprehensive implementation science principles to introduce Suki's technology across various MedStar Health facilities. The initiative seeks to fill this knowledge gap and enhance the practical effectiveness of AI tools in healthcare.

The research agenda will focus on several vital areas crucial for achieving clinical and operational success:

1. **Facilitating Adoption**: Recognizing that the uptake of ambient AI varies widely among healthcare professionals and organizations, experts in human factors from MedStar Health will design and evaluate interventions aimed at creating a scalable, replicable model for increasing adoption across the sector.

2. **Enhancing the Human Experience**: The study will explore how ambient AI impacts not just clinicians but also trainees and patients. Researchers will assess its influence on administrative workloads, clinical decision-making, and the overall experience of patients and providers in diverse healthcare settings.

3. **Ensuring Patient Safety and Quality of Care**: Part of the investigation will focus on how ambient AI integrates with other clinical workflows, with an emphasis on maintaining safety and care quality while enhancing the experiences of both clinicians and patients.

As health systems transition from small-scale pilot programs to broader implementations, the demand for robust, evidence-based insights is more critical than ever. Through its collaboration with MedStar’s National Center for Human Factors, Suki is making a significant investment in comprehensive research to ascertain how ambient AI can be integrated effectively and safely within complex clinical environments.

Sudha Jayaraman, MD, MSc, FACS, who leads this initiative at Suki, emphasizes the importance of collaboration among clinicians, informaticists, and implementation specialists to ensure that the development and application of ambient AI is undertaken thoughtfully. This sentiment is echoed by Joshua Biro, PhD, the principal investigator for the partnership, who highlights the essential role of rigorous evaluation in actualizing the transformative potential of ambient AI. Biro states that the collaboration seeks to ensure these tools support healthcare professionals in their daily routines while enhancing patient care.

According to Raj Ratwani, PhD, director of the National Center for Human Factors, the successful design and deployment of ambient AI hinges on a human-centered approach. The partnership with Suki is dedicated to creating tools that alleviate burdens on healthcare providers, improve patient engagement, and deliver significant advantages across diverse patient demographics.

The overarching aim of this collaboration is to elevate industry standards for the evaluation and integration of ambient AI into clinical practice. By uniting Suki's expertise in ACI with MedStar Health’s deep-rooted knowledge in safety and human factors, they aspire to set new benchmarks for the way ambient AI technologies are assessed and embraced within the healthcare landscape.

Employing a mixed-methods research approach, the partnership will blend quantitative and qualitative methods to investigate the adoption and performance of ambient AI in real-world clinical scenarios. The insights derived from this research are expected to be published in scientific journals and disseminated throughout the healthcare community, contributing to the shared knowledge base in this evolving field.

Suki and MedStar Health's National Center for Human Factors in Healthcare Collaborate to Advance Ambient AI Adoption Science

A sunlit, state-of-the-art hospital control room where two adult healthcare professionals in crisp white lab coats examine an ambient AI interface seamlessly built into the wall—displaying floating patient vitals, voice waveform cues, and human-centered workflow icons. Warm natural light pours through floor-to-ceiling windows, casting optimistic shadows. On a frosted glass partition behind them, the Suki logo and MedStar Health emblem stand side by side, symbolizing their pioneering partnership.

Suki and MedStar Health’s National Center for Human Factors in Healthcare Partner to Pioneer the Science of Ambient AI Adoption | Newswise - Newswise

- Suki and MedStar Health are collaborating to research ambient AI's impact on healthcare.  
- The partnership aims to improve adoption and integration of AI in clinical environments.  
- Research will evaluate AI's effects on patient care, clinician experience, and care quality.  

Suki and MedStar Health’s National Center for Human Factors in Healthcare Partner to Pioneer the Science of Ambient AI Adoption Source Newsroom: MedStar Health

Newswise — WASHINGTON, D.C. & REDWOOD CITY, CA — Suki, the leading Ambient Clinical Intelligence (ACI) platform, and MedStar Health’s National Center for Human Factors in Healthcare are announcing a strategic research collaboration. This partnership aims to study the impact of ambient AI on healthcare delivery.

The "Science of Use": Moving Beyond Surface Metrics

While the industry has focused on rapid adoption, few institutions use the science of human factors to understand technology success in complex clinical environments. By working with the National Center for Human Factors’s industry-leading research on the use of AI in patient care, this collaboration applies rigorous implementation science to the rollout of Suki across MedStar Health clinical sites to bridge this knowledge gap.

Strategic Research Pillars

The collaboration will execute a high-priority research portfolio focused on the fundamental drivers of clinical and operational success:

Designing for Adoption: With adoption of ambient AI varying significantly across clinicians and health systems, MedStar Health’s human factors experts will design and test interventions to create a replicable blueprint to scale adoption.

With adoption of ambient AI varying significantly across clinicians and health systems, MedStar Health’s human factors experts will design and test interventions to create a replicable blueprint to scale adoption. The Human Experience: Researchers will examine how ambient AI affects clinicians, trainees and patients, including its impact on administrative burden, clinical reasoning and the patient-provider experience across diverse care settings and populations.

Researchers will examine how ambient AI affects clinicians, trainees and patients, including its impact on administrative burden, clinical reasoning and the patient-provider experience across diverse care settings and populations. Patient Safety & Care Quality: Researchers will evaluate how ambient AI performs within clinical workflows, with a focus on safety, care quality and the experience of clinicians and patients.

Why This Matters Now

As health systems move from pilot programs to scaling implementation, the need for credible, rigorous real-world evidence is increasingly important. By partnering with MedStar Health’s National Center for Human Factors, Suki is investing in rigorous research to better understand how ambient AI can be safely and effectively implemented in complex clinical environments.

As Sudha Jayaraman, MD, MSc, FACS, who leads the effort at Suki, explains, "Through the Suki Research Collaborative, we're bringing together clinicians, informaticists, and implementation scientists to ensure that ambient AI is developed, tested, implemented, and evaluated thoughtfully and responsibly."

“Realizing the transformational potential of ambient AI requires thoughtful, rigorous evaluation to ensure it’s implemented safely and effectively at all health systems,” said Joshua Biro, PhD, principal investigator for this partnership and research scientist for the MedStar Health National Center for Human Factors in Healthcare. “By combining human factors research with Suki’s innovation, we can make sure these tools genuinely support clinicians in their daily work and improve the care patients receive.”

“Ambient AI has the potential to fundamentally reshape the patient-provider experience, but only if it is designed and deployed with people at the center,” said Raj Ratwani, PhD, director of the National Center for Human Factors in Healthcare and vice president of Scientific Affairs for MedStar Health Research Institute. “Our work with Suki focuses on ensuring these tools reduce burden without compromising clinical thinking, enhance patient engagement, and deliver benefits across all patient populations.”

A Vision for Standardized Excellence

The goal of this partnership is to move the industry higher up the evidence pyramid. By combining Suki’s expertise in Ambient Clinical Intelligence with MedStar Health’s legacy in safety and human factors, the collaboration will help advance the standards for how ambient AI is evaluated and integrated into clinical care.

The Methods & Milestones

The collaboration will use mixed-methods research, combining quantitative and qualitative approaches to study how ambient AI is adopted and performs in real-world clinical environments. Findings from the research are expected to be submitted for scientific publication and shared with the broader healthcare community.

Suki and MedStar Health’s National Center for Human Factors in Healthcare Partner to Pioneer the Science of Ambient AI Adoption

Dive Brief:

Healthcare organizations are rapidly adopting agentic artificial intelligence, but governance frameworks for managing the technology are not evolving at the same pace, according to a new survey from digital identity security firm Imprivata.

More than 85% of leaders responsible for AI strategy at healthcare organizations say they’re confident they have visibility into AI agent activity and are able to fully control and govern an autonomous agent’s actions.

However, 72% of respondents admit AI tools are deployed without IT approval at least occasionally within their organizations, highlighting a disconnect between confidence and reality.

Dive Insight:

Healthcare organizations are embracing agentic AI while simultaneously struggling with fundamental governance and oversight challenges, according to the report.

Agentic AI are AI systems that can act largely independently, making complex multi-step decisions and solving problems with minimal human intervention. The healthcare sector has invested heavily in agentic AI, especially for back office use cases such as prior authorization and revenue cycle management automation.

More than a quarter of leaders have already implemented agentic AI at their organizations, according to market research firm Vanson Bourne, which conducted the survey on Imprivata’s behalf. An additional 44% are piloting or conducting proof-of-concept projects and another 21% plan to implement it in the next year.

The vast majority of respondents expect agentic AI to have a “transformative” impact on clinical and operational workflows.

However, agentic AI presents newfound security risks — especially as use cases expand, giving agents access to a wider variety of internal systems, experts say. That’s because, unlike traditional software that relies on user input to act, agents can act autonomously, traversing different information systems and executing actions on behalf of users.

“If an agent has excessive permissions, operates outside its intended scope, or takes a high-risk action without appropriate oversight, the consequences can directly impact care delivery,” said Dr. Sean Kelly, chief medical and growth officer and senior vice president of customer strategy at Imprivata.

“An agent could access or expose sensitive patient information, enter incorrect information into a medical record, alter a medication or dosage or act under a clinician’s authority in a way that the clinician never intended. Because agents can operate autonomously and at machine speed, a single error could also propagate before someone recognizes what is happening,” Kelly said.

More than half of respondents ranked security among their top concerns when adopting agentic AI. Although leaders expressed confidence in their ability to govern AI agents, instances of shadow AI, when employees use AI tools that haven’t been authorized by their companies, are still rampant.

According to a separate survey by Wolters Kluwer, 40% of medical workers and administrators said they were aware of colleagues using unauthorized AI tools, while nearly 20% reported they have used an unsanctioned tool themselves.

Imprivata’s survey shows that organizations are taking fragmented approaches to managing AI agents. Sometimes information technology departments centrally manage the tools, while security teams manage others. Some organizations report ad hoc or unapproved deployments.

And healthcare leaders are still ironing out the specifics of AI governance and how much humans need to be in the loop when an agent is making decisions, according to the research.

That process varies widely based on where the agent is deployed — whether in an administrative, operational or clinical setting.

“Oversight must match the level of clinical risk. Agents need clearly defined identities, permissions, and boundaries, with human review for higher-risk activities and an audit trail for accountability,” Kelly said. “The more autonomy we give these systems, the more important those safeguards become to detect and contain problems before they create broader operational or patient safety risks.”

The healthcare industry has long sounded the alarm about potential patient safety risks posed by improperly implemented AI. ECRI, a nonprofit focused on healthcare safety and quality, named insufficient governance of AI in healthcare as one of its top 10 patient safety concerns last year.

Various industry consortia have issued guidance to alleviate these concerns. The Coalition for Health AI, a network of thousands of healthcare systems and industry groups, issued AI governance playbooks last year to help health systems responsibly and safely roll out AI tools.

Healthcare’s agentic AI boom is outpacing governance: report

Generate a cinematic news thumbnail for title:Healthcare’s agentic AI boom is outpacing governance: report. No text in image.

Healthcare’s agentic AI boom is outpacing governance: report - Healthcare Dive

Illinois State University faculty and student researchers are helping drive the future of healthcare innovation through a newly recognized partnership with Healthy Tray, an Illinois-based digital health startup that is transforming how hospitals measure and manage patient nutrition.

Healthy Tray recently received an Illinois Innovation Voucher from the Illinois Science & Technology Coalition (ISTC), a competitive award designed to connect innovative Illinois companies with the state’s leading research institutions. The award will support the continued development and validation of Healthy Tray’s artificial intelligence-powered food imaging platform through a collaborative effort involving Illinois State University and OSF HealthCare St. Francis Medical Center.

At Illinois State, the project is led by Dr. Julie Schumacher from the Department of Family and Consumer Sciences and Dr. Abdelmounaam Rezgui from the School of Information Technology. The University is collaborating with Healthy Tray on a project titled “An AI-Powered Technology for Accurate Nutrition Intake.”

The technology aims to improve the accuracy and efficiency of nutrition monitoring in healthcare settings. By leveraging artificial intelligence and food imaging technology, the platform is designed to provide real-time insights into patient food consumption, helping healthcare providers better understand nutritional intake and make more informed decisions about patient care.

The Illinois Innovation Voucher award underscores the important role Illinois State University plays in connecting research expertise with industry partners to solve real-world challenges. Through the project, Illinois State faculty and student researchers will contribute their knowledge and technical skills while gaining valuable experience working on an emerging healthcare technology with the potential for widespread clinical impact.

“Healthy Tray is honored and grateful to the Illinois Science & Technology Coalition, Illinois State University, and OSF HealthCare St. Francis Medical Center for their academic and clinical partnership and shared commitment to advancing healthcare innovation,” said Michele Peplinski, founder and CEO of Healthy Tray. “This award is an exciting milestone that demonstrates the power of bringing together entrepreneurial vision, academic research, and clinical expertise to address real-world healthcare challenges. Together, we are working to redefine how nutrition is measured and managed in hospitals, positioning real-time nutrition data as essential to patient care and advancing the future of Food Is Medicine in Illinois and beyond.”

Partnerships such as this demonstrate the University’s commitment to innovation, entrepreneurship, and applied research. By working alongside industry leaders and healthcare professionals, Illinois State researchers are helping accelerate the translation of new ideas into practical solutions that benefit communities throughout Illinois and beyond.

The collaboration also highlights the power of bringing together diverse expertise. Healthy Tray provides entrepreneurial vision and technological innovation, Illinois State University contributes research and academic excellence, and OSF HealthCare St. Francis Medical Center offers clinical expertise and real-world healthcare insight. Together, these partners are working to advance a technology that could redefine how nutrition is measured and managed in hospitals.

As the project moves forward, the ISTC award serves as both recognition of the team’s accomplishments and an investment in the future of healthcare innovation. It reflects the growing impact of Illinois State University faculty and student researchers who continue to play a vital role in developing solutions that address complex challenges and improve lives through research, collaboration, and discovery.

Illinois State University partnership advances AI innovation in healthcare

Generate a cinematic news thumbnail for title:Illinois State University partnership advances AI innovation in healthcare. No text in image.

Illinois State University partnership advances AI innovation in healthcare - Illinois State University News

Global Health Forum puts AI and healthcare governance in focus

Generate a cinematic news thumbnail for title:Global Health Forum puts AI and healthcare governance in focus. No text in image.

Global Health Forum puts AI and healthcare governance in focus - news.cgtn.com

Artificial Intelligence & Machine Learning , Fraud Management & Cybercrime , Fraud Risk Management

How the DOJ Uses AI to Detect Healthcare Fraud

Attorney Adrienne Frazior of Polsinelli on Data Analytics, Fraud Detection

Adrienne Frazior, healthcare litigation practice vice chair, Polsinelli

The U.S. Department of Justice is using artificial intelligence and advanced data analytics to identify healthcare fraud as it happens, rather than months or years later. That advancement can help prosecutors detect anomalies sooner, said Adrienne Frazior, government investigations attorney and vice chair of law firm Polsinelli's healthcare litigation practice.

See Also: A Darkening Landscape: AI, Friend and Foe of Cyber Resilience

The DOJ's data fusion center brings analytics experts together with multiple federal units, including the Department of Health and Human Services' Office of Inspector General, the FBI, and other agencies, said Frazior, who had previously worked in the DOJ's healthcare fraud unit.

AI can flag extreme outliers, such as healthcare providers who submit bills claiming a highly unrealistic number of service hours a day, helping investigators focus on suspicious claims faster. Quicker detection can also support faster prosecutions and recovery of taxpayer funds.

But Frazior cautioned that analytics tell only part of the story. Cyberattacks or flawed data could mimic fraud, so prosecutors still need interviews, verification and conventional investigative techniques before drawing conclusions.

"I just can't stress enough how much providers need to be looking at their own data. They need to be on the forefront of that. That's their obligation - before it is the DOJ's obligation to do it on the back end," she said.

In this video interview with ISMG, Frazior also discussed:

How AI helps DOJ identify unusual billing patterns and emerging healthcare fraud schemes;

Why providers should analyze their own claims data to identify compliance risks before regulators do;

Why human review remains essential when AI or analytics flag suspicious activity.

With 10 years of experience as a former federal prosecutor, Frazior has an extensive understanding of the complex regulatory and compliance issues that affect the healthcare industry. At Polsinelli, she defends corporate executives and companies in federal criminal investigations.

How the DOJ Uses AI to Detect Healthcare Fraud

Generate a cinematic news thumbnail for title:How the DOJ Uses AI to Detect Healthcare Fraud. No text in image.

How the DOJ Uses AI to Detect Healthcare Fraud - BankInfoSecurity

As AI enters health care, are we ready?

Credit: Unsplash/CC0 Public Domain

Artificial intelligence is rapidly becoming part of how people seek, interpret and act on health information. Yet the technology is advancing faster than the evidence needed to determine whether these systems actually help people understand what matters, make informed health decisions and know what to do next.

A new perspective published in the journal Nature Human Behaviour and led by University of Alabama professor Rebecca K. Ivic, with CUNY SPH distinguished lecturer Scott C. Ratzan and Emory University School of Medicine professor emerita Ruth M. Parker, introduces the concept of health-literate artificial intelligence and proposes four principles for designing and governing AI-mediated health communication: comprehension, agency, accountability and proportionality.

"We are deploying AI into health contexts faster than we are building the evidence needed to know whether it actually helps people understand, decide and act," Ivic says.

As AI systems are used to answer questions about symptoms, summarize health guidance, interpret risk and support decisions, they may provide technically plausible responses while failing to communicate how urgent a situation is, how certain the information is, what alternatives exist, when professional care is needed and when to seek human judgment and dialogue for appropriate decision-making.

The authors define health-literate AI as AI designed to align information, guidance and responsibility with users' abilities, contexts and needs. The framework shifts attention beyond whether AI produces information that is technically accurate or explainable to whether people can understand what that information means, make informed decisions and determine what to do next.

The four components are:

Comprehension: Can people interpret AI output in relation to their language, prior knowledge, and circumstances?

Agency: Does the information support meaningful and informed action, including clear choices and next steps?

Accountability: Are evidence, uncertainty, limitations, and responsibility visible?

Proportionality: Does the amount and urgency of guidance match the stakes of the situation?

Together, these components provide a basis for considering whether AI-mediated communication is fit for purpose in health-relevant settings.

"Given all the discussion and warnings about the potential harms of AI, we need expert thinking, research and analysis that protects people from potential harms while realizing the promise of this new technology for health," Ratzan says.

"Is AI in health care the holy grail or the tower of Babel?" Parker asks. "In medicine, there are immediate questions: Can all people understand and safely use what these systems tell them in order to improve health outcomes? Is its use cost-effective and also aligned with our human values?"

The framework builds on decades of health literacy research that has increasingly examined not only individuals' skills but also how governments, organizations and systems make health information easier or harder to understand and use. For more than 30 years, Parker and Ratzan have contributed substantially to this body of work, including research on health literacy as a systems and policy concern. Ivic proposes a novel model for today's digital information environment, examining how platforms, institutions and emerging technologies shape the production, interpretation and use of health information.

Rather than placing the burden entirely on individuals to interpret increasingly complex automated information, health-literate AI asks how the design and governance of AI systems themselves can support understanding and informed action. The authors also challenge designers to envision how AI could help operationalize affordability of health care, a long-standing, well-acknowledged goal that we have not achieved.

"This model gives researchers, developers and regulators a shared way to address a more consequential question," Ivic says. "The question is not simply whether AI can generate an answer, but whether people can understand it, use it and act on it appropriately. Health-literate AI should do more of the work of making quality health information understandable, actionable and accountable, with design and governance working together to advance responsible AI in health."

The authors argue that this is particularly important in high-stakes or ambiguous situations. AI systems should distinguish among informing, advising and deciding and make those roles clear to users. When automated output is insufficient, systems should also identify when human judgment, clinical evaluation or ethical deliberation is needed.

"The goal is not to make people better at navigating increasingly complex AI," Ivic says. "It is to give the field a way to begin measuring whether that is actually happening."

The perspective argues that health literacy should not simply require people to adapt to increasingly complex technologies. As AI becomes an increasingly important intermediary between people and health information, the systems themselves should be designed around the people who must understand and use them.

The authors call for adoption of the principles of health-literate AI to inform design, evaluation, research and governance across health-relevant AI ecosystems. They also point to an important evidence gap: More research is needed to determine whether AI actually improves comprehension, informed decision-making and health-related action.

"AI in health care no doubt offers opportunities for innovation, but more research, guardrails and regulation are needed to ensure its benefits clearly outweigh its risks," Parker says. "Its design and use need to enhance users' abilities to understand and make decisions that improve health, are affordable and are consistently aligned with our human values."

Who's behind this story? Robert Egan Bachelor's in mathematical biology, Master's in creative writing. Well-traveled with unique perspectives on science and language. Full profile →

As AI enters health care, are we ready?

Generate a cinematic news thumbnail for title:As AI enters health care, are we ready?. No text in image.

This legislation may impose restrictions on AI in healthcare.

More from AI in healthcare

Worries over AI in healthcare, Addressing conspiracies: PD Letters

Shaywitz Talks about AI in Healthcare on NewsNation's 'Hot Take with Jesse Weber' Podcast

NEHEN: Looking Past the Hype: Integrating AI in Healthcare

JPMorgan: Emerging market benchmarks are less sensitive to interest rates, prefer AI and healthcare sectors