Presented by MongoDB
Creating AI solutions that are precise, secure, and dependable is a significant technical challenge for organizations operating under strict compliance mandates, particularly in the healthcare, financial services, and transportation sectors. These industries often lag behind in technological advancements due to regulations that necessitate a cautious approach to innovation. Additionally, many organizations are now faced with the task of modernizing their data infrastructures in order to meet the increasing demand for AI capabilities.
A notable example of successful modernization is Australian-founded AI Care Partner, Heidi. Its primary offering, Heidi Scribe, automates a substantial amount of administrative tasks that can consume clinicians' time, operating in over 190 countries and facilitating approximately 2.7 million patient interactions weekly. This growth is based on foundational infrastructure choices made long before the company expanded globally, according to Yu Liu, co-founder and CTO of Heidi.
“In most sectors, an AI feature with a two percent error rate may be merely an inconvenience, but in healthcare, this same error rate poses a serious risk to clinical safety,” Liu explains. “Therefore, our architecture is designed with the understanding that every output may be subject to scrutiny and auditing in patient care.”
Exploring the unique architectural challenges of deploying AI in healthcare
For Heidi, data residency is a fundamental requirement rather than an optional feature. Clinicians in different locations—whether in Sydney, London, Tokyo, or Denver—are governed by various regulatory frameworks such as the Australian Privacy Principles, GDPR, APPI, and HIPAA, necessitating that patient data remains within the respective regions.
Heidi employs fully logically isolated production environments worldwide, ensuring that data residency is an inherent characteristic of its architecture. Additionally, the system must be designed from the outset for auditability because organizations need to track the model's inputs, outputs, and clinician modifications for any session, even months later.
“The impact of change has to be minimized,” Liu emphasizes. “While less regulated industries allow for rapid deployment and subsequent fixes, in healthcare, we prioritize creating safe changes by default, implementing continuous integration gates on high-risk changes, utilizing canary releases, and treating database schema modifications with the same diligence as code reviews. Our speed is a result of this commitment to safety.”
Selecting a database for AI workflows
Heidi manages a wide range of medical data from diverse sources, including forms, referrals, and clinician notes, all of which must be unified into a consistent format and stored in a single location to align seamlessly with AI workflows. Traditional database structures were not suited for this requirement.
Consequently, a document database was the ideal solution for Heidi. By leveraging MongoDB, the team acquired the flexibility necessary to handle rapidly evolving AI data without the need for continual adjustments to the database architecture.
“The model constitutes about 20% of the overall system, and the data architecture dictates whether the remaining 80% performs effectively under actual clinical loads,” Liu states.
An AI Scribe session encompasses a variety of data types: transcripts, structured notes, templates, documents, patient context, and EHR integration states, all of which can change frequently. MongoDB enables this data to exist in forms that align with how clinicians function, allowing Heidi to adapt and evolve without being hampered by migration freezes as the product develops.
“MongoDB Atlas stood out for its combination of the document model's advantages—like seamless scalability, flexibility, and performance—with built-in AI-optimized features, including MongoDB Vector Search,” Liu notes. “Heidi does not need to integrate an additional vector database to enhance its platform.”
With more than 130 cloud regions worldwide, in addition to on-premises and hybrid options, MongoDB Atlas is the most accessible and globally distributed database solution available. Its unified query API allows developers to create full-text search capabilities, real-time analytics, and event-driven applications without complicating their architecture.
“Heidi Scribe translates large quantities of medical documents into vector embeddings using LangChain in Atlas, which facilitates semantic searches that connect transcribed medical terms directly to external knowledge bases,” Liu adds. “Switching to Atlas has cut latency on key APIs by almost 33%.”
Establishing a trustworthy clinical retrieval-augmented generation (RAG) system
“Retrieval is fundamentally a data architecture issue, rather than solely an AI problem,” Liu asserts. "In consumer applications, you collect data from the open web and hope for the best, whereas in healthcare, the focus is on ensuring compliance in what you retrieve."
Heidi Evidence draws from authorized clinical knowledge repositories, including collaborations with BMJ Best Practice, NICE CKS, and MIMS, and is designed to be jurisdiction-aware, ensuring that a clinician in the U.K. receives guidance relevant to the U.K. context, while an Australian clinician receives the Australian formulary—since what is accurate in one region may not be applicable in another.
Heidi’s embeddings and vector indexes reside within MongoDB Vector Search, situated in the same regionally isolated systems as the rest of its data. This ensures that retrieval processes cannot violate residency requirements, and obviates the need for a separate vector database with its own compliance and security protocols. Citations are treated as definitive contracts rather than suggestions, as the model only interacts with retrieved segments that are already linked to source records.
Region-specific isolation supports global compliance and scalability
“Each region operates a fully isolated production setup with its own MongoDB Atlas clusters, computing resources, and security keys,” Liu explains.
“This is what enables us to present a clear residency answer to U.S. health systems, NHS trusts, or Australian hospital groups, as this is enforced by our infrastructure rather than it being solely a contractual promise," he continues. "Managing multiple isolated regions efficiently relies on a consistent and well-maintained database infrastructure. Our multi-cloud framework means that launching a new region involves establishing another deployment using existing operational frameworks.”
This architectural approach has proved particularly effective in the U.S., where Beth Israel Lahey Health, among New England’s largest health systems, implemented Heidi’s AI Scribe after a pilot revealed that 74% of clinicians experienced reduced documentation time after hours, commonly referred to as “pajama time.” MaineGeneral Health, a non-profit health system, also selected Heidi as a strategic partner for its initiatives in rural healthcare.
"Establishing a presence in the U.S. meant we could create another regional setup using the infrastructure we had previously designed, rather than having to retroactively adjust for HIPAA compliance," Liu indicates.
Key takeaways and future plans
“Re-partitioning a large, continuously active data collection involves significant engineering efforts, whereas selecting a shard key upfront is part of initial design discussions,” Liu reflects. "We are currently engaging in this critical work alongside MongoDB partners, and the takeaway for those developing data-intensive AI products is that establishing horizontal scalability for fast-growing data is a foundational choice, much like ensuring data residency.”
Heidi plans to extend its offerings beyond consult notes to enhance the entire clinical workflow, encompassing pre-visit context, post-visit documentation, referrals, and workflow automation. The company is also investigating how MongoDB, large language models, and its own tools can create an integrated ecosystem for clinical processes.
“In the realm of healthcare AI, reliability engineering equates to trust engineering,” Liu states. “A clinician can lose trust just as quickly due to downtime, latency, or data inconsistencies as they would from an incorrect note. Much of our crucial work—such as canary releases with automatic rollbacks, continuous integration gates on changes to the database, and consistency checks across regions—is performed discreetly. Building clinician trust is the product we offer, and trust itself is deeply embedded in our architectural design.”
