Utah is forging ahead with its innovative experiments in the realm of artificial intelligence in healthcare, recently granting approval for new pilot programs that will facilitate prescription issuance and refills without direct clinician oversight.
On Friday, the state's Office of Artificial Intelligence Policy greenlit several new initiatives, including a collaboration with Nolla Health. This developer has created a dermatology application designed to evaluate acne through images submitted by patients. The pilot program will explore the application’s capability to prescribe initial medications and refills for a limited range of topical treatments to adults in Utah who have mild to moderate acne. Initially, each prescription will receive a review from two physicians, but as the program advances, this oversight will diminish, leading to retrospective checks instead.
These recent projects represent the latest in Utah’s healthcare AI initiatives, despite facing criticism from healthcare professionals and stakeholders. Earlier in the year, regulators initiated a pilot that allows an AI system from Doctronic to manage medication renewals without any clinician intervention.
Additional pilots sanctioned on Friday include programs from August AI, which will leverage AI technology for renewing prescriptions for chronic condition management, and Expect Fitness, which aims to provide virtual pelvic floor therapy for women dealing with issues such as urinary incontinence, pelvic pain, and postpartum complications.
In all these pilots, companies are required to secure written approval from regulators before advancing to subsequent phases that may involve reduced human oversight. As of now, none of these pilot programs have commenced.
Utah’s Artificial Intelligence Learning Laboratory, often referred to as the AI sandbox, was established by legislation in 2024. Lawmakers designed the lab to provide consumer protections while fostering innovation by creating a controlled setting where AI tools and methods can be tested and evaluated prior to broader implementation. Companies interested in piloting their products must enter an agreement with the state, which outlines operational boundaries, safeguards, and required reporting.
However, these initiatives have not been without opposition from the medical community. For instance, following the launch of Doctronic’s pilot last year, the state’s medical licensing board requested its suspension, claiming they were not adequately informed about its initiation. Doctors voiced concerns regarding the absence of essential clinical judgment within the AI framework, warning that patients might be prescribed outdated medications.
In response, regulators have stood firm, maintaining that human oversight will be in place during the initial phases of these AI systems.
Experts continue to voice apprehensions about the risks associated with these AI pilots, even for seemingly straightforward tasks like prescription renewals. In a commentary for the JAMA Health Forum, Michelle Mello, a law and health policy professor at Stanford, pointed out that the agreement with Doctronic lacks clarity regarding accountability for potential prescribing errors by the AI that could harm patients.
Despite these concerns, Utah regulators remain committed to exploring further pilot opportunities. Last month, the Office of Artificial Intelligence Policy entered into master agreements with University of Utah Health and Intermountain Health. While these agreements do not constitute pilots themselves, they set the groundwork for potential future initiatives.
