Unlocking the potential of artificial intelligence (AI) in healthcare necessitates enhanced coherence between payment structures, regulatory standards, and the targeted outcomes that these advanced technologies aim to achieve.
Health systems are ramping up their investment in AI, yet they are under pressure to demonstrate tangible results. Alarmingly, a large majority of initiatives—approximately 80%—fail to progress beyond the pilot phase, and around 95% do not yield a satisfactory return on investment. Hospital administrators frequently cite ongoing challenges with unsuccessful pilot programs and ongoing assessments of vendor performance.
In response to these difficulties, the Digital Medicine Society (DiMe) and Qualified Health convened a closed meeting with leaders from 12 health systems and six AI tech developers. The aim was to share insights on the practicalities of making, implementing, and evaluating AI-related decisions in healthcare.
This collaborative effort is guided by a mutual goal: enabling health systems to extract immediate value from AI investments while making informed choices that will position them favorably as technology improves and becomes more widely adopted.
Here are five key insights that emerged from this conversation, shedding light on how health systems are currently navigating their AI strategies and projects:
1. Align AI portfolios with organizational priorities. Many health systems often make AI-related choices detached from their core strategic aims. Instead of aligning investments with system-level objectives, decisions are frequently influenced by vendor actions, peer trends, or internal requests. This disconnect is most evident in academic medical centers, where the need to balance care delivery, research, and education often results in competing interests.
Health systems achieving measurable outcomes have addressed this misalignment by defining a select set of priorities that guide the choice and execution of AI solutions. These guiding objectives typically focus on enhancing access, stabilizing finances, and alleviating workforce pressures, which help shape what use cases are pursued, how success is defined, and when a project should be scaled or ceased.
2. Emphasize prioritization as an operating discipline. AI solution requests are coming in from multiple fronts within health systems. Executives expect demonstrable results related to access, expenses, and workforce performance, while frontline practitioners identify specific instances where AI could alleviate burdens, enhance workflows, or clear bottlenecks. However, the cumulative demand often outstrips the operational capacity of these systems.
Effective prioritization entails making tough trade-offs, where endorsing one project typically means sidelining others. This issue often leads to an expanding portfolio of pilots without a clear trajectory for scaling. Successful systems maintain discipline in prioritizing use cases that have readily identifiable implementation pathways linked to strategic objectives.
The source of funding also significantly impacts these decisions; centrally funded initiatives tend to align more closely with enterprise goals, while department-driven efforts may lack this cohesion. Furthermore, systems showing progress have established foundational capabilities in workforce training, operational procedures, and data integration, which are critical for long-term success.
3. Transform governance from conceptual to operational. Governance is often cited as a hindrance to AI adoption, primarily because it is rarely defined in actionable terms. Many health systems approach governance in theoretical ways, discussing it with respect to principles and oversight but failing to outline specific decision-making processes necessary for advancing from assessment to implementation.
Effective governance comprises three essential functions: selecting suitable solutions, executing them correctly, and ensuring sustained performance without introducing undue risk. Systems that are advancing have adopted more practical governance models, establishing systematic pathways for solution evaluations and pilot implementations, along with clearly defined accountability.
Central to these governance models is outlining the division of responsibilities between the health system and the vendor, particularly as AI features become more integrated into products. While health systems are accountable for performance, that accountability often lacks clarity.
4. Adjust decision cycles to keep pace with AI innovations. The fast-evolving nature of AI innovations exceeds the capacity of many health systems to assess, adopt, and effectively manage new technologies. This challenge is particularly noticeable in decision-making processes, many of which remain tethered to monthly committee meetings that can cause significant delays.
Simultaneously, the pressure for high performance intensifies as AI is introduced into areas where reliability is crucial across teams and workflows. Health systems making headway are altering their decision-making approaches, utilizing smaller, empowered teams to streamline evaluations and approvals, thereby reducing testing and iteration cycles.
Furthermore, as vendor strategies evolve, health systems are exploring various alternatives if a required capability isn’t available soon enough, fostering a competitive dynamic among vendors while also shifting contract models for greater flexibility and performance accountability.
5. Measure value as a comprehensive portfolio. All participating health systems acknowledged that quantifying AI-derived value is more complex than previously anticipated. Although several implementations offer clear advantages, translating those benefits into effective decision-making metrics remains a challenge.
While financial performance is a primary concern for executives, many tangible benefits—including clinician efficiency, diminished administrative burdens, and enhanced patient access—require different evaluation metrics. This creates a tug-of-war because while CFOs seek measurable returns, clinical leaders prioritize workflow integration, and operational heads focus on throughput and capacity.
Health systems on the move are directly addressing this conundrum by defining value across a few specific domains and tracking performance in these areas. Current value realized tends to concentrate on financial health, provider experience, and access to care, necessitating a shift from one-off evaluations to ongoing monitoring that holds AI systems accountable in real-world circumstances.
The dialogue highlighted the need for health systems to better align their actions with payment models and regulatory standards, which ultimately establish the framework for what can be prioritized and scaled. The interplay between financial incentives and regulatory requirements profoundly influences which AI applications progress, emphasizing the importance of assessing both immediate and long-term impacts.
Promisingly, forward-thinking health systems are already extracting substantial value from AI in areas such as financial outcomes, provider engagement, and patient accessibility. What sets these systems apart is their strategic clarity in prioritizing, implementing, and sustaining effective AI initiatives.
Nevertheless, inconsistency remains prevalent across healthcare organizations, where disparities in strategy and measurement hinder the scalability of successful approaches. Current literature on health AI often lacks practical insights that reflect the actual deployment realities within health systems.
To truly harness the value of AI, more operation-focused work is required—this includes developing capabilities that prioritize effectively amid competing demands, implementing robust systems adaptable across varying environments, and maintaining accountability over time.
Both DiMe and Qualified Health are committed to further engagement on this front, concentrating on actionable execution to facilitate value capture for both healthcare systems and patients alike, including providing open-access resources and ongoing shared learning opportunities based on real-world applications.
