Increasingly, patients are arriving at their medical appointments after discussing their symptoms with AI chatbots, often placing as much trust in these digital assistants as they do in their healthcare providers.
However, the issue arises when a chatbot offers incorrect guidance and a patient suffers as a result. The question of accountability remains murky. Meghan O’Connor, a health law partner at Quarles & Brady and co-chair of the firm’s AI team, highlights that this critical issue in healthcare law is still evolving.
"This represents one of the most pressing unresolved questions in the realm of healthcare law today. The distribution of liability will depend heavily on the specific circumstances and is likely to be clarified through years of legal proceedings," she explained.
O’Connor identifies three key stakeholders in the liability discussion: the AI developer, the patient, and the healthcare provider.
AI companies typically include disclaimers stating that their outputs shouldn't be considered medical advice, but these disclaimers may not provide much protection if the chatbot is promoted in a way that leads patients to perceive its responses as diagnostic recommendations.
When it comes to patients, O’Connor believes that courts are unlikely to assign significant blame to individuals who trust a resource that appears to offer authoritative information.
Providers, however, face a more precarious situation. Once a patient mentions they've acted on misleading AI advice, a provider's failure to rectify that misinformation could lead to malpractice claims. O’Connor emphasizes that determining the standard of care involves assessing what a reasonable provider would have done when faced with identical information.
"My practical advice for healthcare providers is clear: correct any misinformation, document the conversation thoroughly, and handle AI-derived data the same way you would any other patient-reported information that contradicts your clinical judgment—by addressing it directly and noting your actions," she advised.
When a patient refers to AI-generated information during their visit, it becomes part of their medical history, just like if they had shared insights from an online search or advice from a friend, O’Connor points out.
She warns against inaction.
"Consider this analogy: if a patient mentions seeing on an online forum that drinking pickle juice can cure glaucoma, no responsible ophthalmologist would remain silent. Misinformation from AI should be treated with the same level of seriousness. The origin of the erroneous information does not diminish a provider’s obligation to their patient once it's been introduced in a healthcare context," O’Connor asserted.
Though legal frameworks have yet to catch up with the advent of AI chatbots, O’Connor’s concluding advice for providers is to engage proactively rather than ignore these discussions. Investing a little time now could significantly impact patient outcomes in the future.
