Discordance between a deep learning model and clinical-grade variant pathogenicity classification in a rare disease cohort.

Genetic testing has become an essential component in the diagnosis and management of a wide range of clinical conditions, from cancer to developmental disorders, especially in rare Mendelian diseases. Efforts to identify rare phenotype-associated variants have predominantly focused on protein-truncating variants, while the interpretation of missense variants presents a considerable challenge. Deep learning algorithms excel in various applications across biomedical tasks1,2, yet accurately distinguishing between pathogenic and benign genetic variants remains an elusive goal3-5. Specifically, even the most sophisticated models encounter difficulties in accurately assessing the pathogenicity of missense variants of uncertain significance (VUS). Our investigation of AlphaMissense (AM)5, the latest iteration of deep learning methods for predicting the potential functional impact of missense variants and assessing gene essentiality, reveals important limitations in its ability to identify pathogenic missense variants within a rare disease cohort. Indeed, AM struggles to accurately assess the pathogenicity of variants in intrinsically disordered regions (IDRs), leading to unreliable gene-level essentiality scores for certain genes containing IDRs. This limitation highlights the challenges in applying AM faces in the context of clinical genetics6.

Cumulus: a federated electronic health record-based learning system powered by Fast Healthcare Interoperability Resources and artificial intelligence.

OBJECTIVE: To address challenges in large-scale electronic health record (EHR) data exchange, we sought to develop, deploy, and test an open source, cloud-hosted app "listener" that accesses standardized data across the SMART/HL7 Bulk FHIR Access application programming interface (API). METHODS: We advance a model for scalable, federated, data sharing and learning. Cumulus software is designed to address key technology and policy desiderata including local utility, control, and administrative simplicity as well as privacy preservation during robust data sharing, and artificial intelligence (AI) for processing unstructured text. RESULTS: Cumulus relies on containerized, cloud-hosted software, installed within a healthcare organization's security envelope. Cumulus accesses EHR data via the Bulk FHIR interface and streamlines automated processing and sharing. The modular design enables use of the latest AI and natural language processing tools and supports provider autonomy and administrative simplicity. In an initial test, Cumulus was deployed across 5 healthcare systems each partnered with public health. Cumulus output is patient counts which were aggregated into a table stratifying variables of interest to enable population health studies. All code is available open source. A policy stipulating that only aggregate data leave the institution greatly facilitated data sharing agreements. DISCUSSION AND CONCLUSION: Cumulus addresses barriers to data sharing based on (1) federally required support for standard APIs, (2) increasing use of cloud computing, and (3) advances in AI. There is potential for scalability to support learning across myriad network configurations and use cases.

Moving Biosurveillance Beyond Coded Data Using AI for Symptom Detection From Physician Notes: Retrospective Cohort Study.

BACKGROUND: Real-time surveillance of emerging infectious diseases necessitates a dynamically evolving, computable case definition, which frequently incorporates symptom-related criteria. For symptom detection, both population health monitoring platforms and research initiatives primarily depend on structured data extracted from electronic health records. OBJECTIVE: This study sought to validate and test an artificial intelligence (AI)-based natural language processing (NLP) pipeline for detecting COVID-19 symptoms from physician notes in pediatric patients. We specifically study patients presenting to the emergency department (ED) who can be sentinel cases in an outbreak. METHODS: Subjects in this retrospective cohort study are patients who are 21 years of age and younger, who presented to a pediatric ED at a large academic children's hospital between March 1, 2020, and May 31, 2022. The ED notes for all patients were processed with an NLP pipeline tuned to detect the mention of 11 COVID-19 symptoms based on Centers for Disease Control and Prevention (CDC) criteria. For a gold standard, 3 subject matter experts labeled 226 ED notes and had strong agreement (F1-score=0.986; positive predictive value [PPV]=0.972; and sensitivity=1.0). F1-score, PPV, and sensitivity were used to compare the performance of both NLP and the International Classification of Diseases, 10th Revision (ICD-10) coding to the gold standard chart review. As a formative use case, variations in symptom patterns were measured across SARS-CoV-2 variant eras. RESULTS: There were 85,678 ED encounters during the study period, including 4% (n=3420) with patients with COVID-19. NLP was more accurate at identifying encounters with patients that had any of the COVID-19 symptoms (F1-score=0.796) than ICD-10 codes (F1-score =0.451). NLP accuracy was higher for positive symptoms (sensitivity=0.930) than ICD-10 (sensitivity=0.300). However, ICD-10 accuracy was higher for negative symptoms (specificity=0.994) than NLP (specificity=0.917). Congestion or runny nose showed the highest accuracy difference (NLP: F1-score=0.828 and ICD-10: F1-score=0.042). For encounters with patients with COVID-19, prevalence estimates of each NLP symptom differed across variant eras. Patients with COVID-19 were more likely to have each NLP symptom detected than patients without this disease. Effect sizes (odds ratios) varied across pandemic eras. CONCLUSIONS: This study establishes the value of AI-based NLP as a highly effective tool for real-time COVID-19 symptom detection in pediatric patients, outperforming traditional ICD-10 methods. It also reveals the evolving nature of symptom prevalence across different virus variants, underscoring the need for dynamic, technology-driven approaches in infectious disease surveillance.

Explainable machine learning for predicting conversion to neurological disease: Results from 52,939 medical records.

Objective: This study assesses the application of interpretable machine learning modeling using electronic medical record data for the prediction of conversion to neurological disease. Methods: A retrospective dataset of Cleveland Clinic patients diagnosed with Alzheimer's disease, amyotrophic lateral sclerosis, multiple sclerosis, or Parkinson's disease, and matched controls based on age, sex, race, and ethnicity was compiled. Individualized risk prediction models were created using eXtreme Gradient Boosting for each neurological disease at four timepoints in patient history. The prediction models were assessed for transparency and fairness. Results: At timepoints 0-months, 12-months, 24-months, and 60-months prior to diagnosis, Alzheimer's disease models achieved the area under the receiver operating characteristic curve on a holdout test dataset of 0.794, 0.742, 0.709, and 0.645; amyotrophic lateral sclerosis of 0.883, 0.710, 0.658, and 0.620; multiple sclerosis of 0.922, 0.877, 0.849, and 0.781; and Parkinson's disease of 0.809, 0.738, 0.700, and 0.651, respectively. Conclusions: The results demonstrate that electronic medical records contain latent information that can be used for risk stratification for neurological disorders. In particular, patient-reported outcomes, sleep assessments, falls data, additional disease diagnoses, and longitudinal changes in patient health, such as weight change, are important predictors.

GSDMB/ORMDL3 Rare/Common Variants Are Associated with Inhaled Corticosteroid Response among Children with Asthma.

Inhaled corticosteroids (ICS) are efficacious in the treatment of asthma, which affects more than 300 million people in the world. While genome-wide association studies have identified genes involved in differential treatment responses to ICS in asthma, few studies have evaluated the effects of combined rare and common variants on ICS response among children with asthma. Among children with asthma treated with ICS with whole exome sequencing (WES) data in the PrecisionLink Biobank (91 White and 20 Black children), we examined the effect and contribution of rare and common variants with hospitalizations or emergency department visits. For 12 regions previously associated with asthma and ICS response (DPP10, FBXL7, NDFIP1, TBXT, GLCCI1, HDAC9, TBXAS1, STAT6, GSDMB/ORMDL3, CRHR1, GNGT2, FCER2), we used the combined sum test for the sequence kernel association test (SKAT) adjusting for age, sex, and BMI and stratified by race. Validation was conducted in the Biorepository and Integrative Genomics (BIG) Initiative (83 White and 134 Black children). Using a Bonferroni threshold for the 12 regions tested (i.e., 0.05/12 = 0.004), GSDMB/ORMDL3 was significantly associated with ICS response for the combined effect of rare and common variants (p-value = 0.003) among White children in the PrecisionLink Biobank and replicated in the BIG Initiative (p-value = 0.02). Using WES data, the combined effect of rare and common variants for GSDMB/ORMDL3 was associated with ICS response among asthmatic children in the PrecisionLink Biobank and replicated in the BIG Initiative. This proof-of-concept study demonstrates the power of biobanks of pediatric real-life populations in asthma genomic investigations.

Real world performance of the 21st Century Cures Act population-level application programming interface.

OBJECTIVE: To evaluate the real-world performance of the SMART/HL7 Bulk Fast Health Interoperability Resources (FHIR) Access Application Programming Interface (API), developed to enable push button access to electronic health record data on large populations, and required under the 21st Century Cures Act Rule. MATERIALS AND METHODS: We used an open-source Bulk FHIR Testing Suite at 5 healthcare sites from April to September 2023, including 4 hospitals using electronic health records (EHRs) certified for interoperability, and 1 Health Information Exchange (HIE) using a custom, standards-compliant API build. We measured export speeds, data sizes, and completeness across 6 types of FHIR. RESULTS: Among the certified platforms, Oracle Cerner led in speed, managing 5-16 million resources at over 8000 resources/min. Three Epic sites exported a FHIR data subset, achieving 1-12 million resources at 1555-2500 resources/min. Notably, the HIE's custom API outperformed, generating over 141 million resources at 12 000 resources/min. DISCUSSION: The HIE's custom API showcased superior performance, endorsing the effectiveness of SMART/HL7 Bulk FHIR in enabling large-scale data exchange while underlining the need for optimization in existing EHR platforms. Agility and scalability are essential for diverse health, research, and public health use cases. CONCLUSION: To fully realize the interoperability goals of the 21st Century Cures Act, addressing the performance limitations of Bulk FHIR API is critical. It would be beneficial to include performance metrics in both certification and reporting processes.

Cumulus: A federated EHR-based learning system powered by FHIR and AI.

Objective: To address challenges in large-scale electronic health record (EHR) data exchange, we sought to develop, deploy, and test an open source, cloud-hosted app 'listener' that accesses standardized data across the SMART/HL7 Bulk FHIR Access application programming interface (API). Methods: We advance a model for scalable, federated, data sharing and learning. Cumulus software is designed to address key technology and policy desiderata including local utility, control, and administrative simplicity as well as privacy preservation during robust data sharing, and AI for processing unstructured text. Results: Cumulus relies on containerized, cloud-hosted software, installed within a healthcare organization's security envelope. Cumulus accesses EHR data via the Bulk FHIR interface and streamlines automated processing and sharing. The modular design enables use of the latest AI and natural language processing tools and supports provider autonomy and administrative simplicity. In an initial test, Cumulus was deployed across five healthcare systems each partnered with public health. Cumulus output is patient counts which were aggregated into a table stratifying variables of interest to enable population health studies. All code is available open source. A policy stipulating that only aggregate data leave the institution greatly facilitated data sharing agreements. Discussion and Conclusion: Cumulus addresses barriers to data sharing based on (1) federally required support for standard APIs (2), increasing use of cloud computing, and (3) advances in AI. There is potential for scalability to support learning across myriad network configurations and use cases.

Beyond compliance with the 21st Century Cures Act Rule: a patient controlled electronic health information export application programming interface.

OBJECTIVE: The 21st Century Cures Act Final Rule requires that certified electronic health records (EHRs) be able to export a patient's full set of electronic health information (EHI). This requirement becomes more powerful if EHI exports use interoperable application programming interfaces (APIs). We sought to advance the ecosystem, instantiating policy desiderata in a working reference implementation based on a consensus design. MATERIALS AND METHODS: We formulate a model for interoperable, patient-controlled, app-driven access to EHI exports in an open source reference implementation following the Argonaut FHIR Accelerator consensus implementation guide for EHI Export. RESULTS: The reference implementation, which asynchronously provides EHI across an API, has three central components: a web application for patients to request EHI exports, an EHI server to respond to requests, and an administrative export management web application to manage requests. It leverages mandated SMART on FHIR/Bulk FHIR APIs. DISCUSSION: A patient-controlled app enabling full EHI export from any EHR across an API could facilitate national-scale patient-directed information exchange. We hope releasing these tools sparks engagement from the health IT community to evolve the design, implement and test in real-world settings, and develop patient-facing apps. CONCLUSION: To advance regulatory innovation, we formulate a model that builds on existing requirements under the Cures Act Rule and takes a step toward an interoperable, scalable approach, simplifying patient access to their own health data; supporting the sharing of clinical data for both improved patient care and medical research; and encouraging the growth of an ecosystem of third-party applications.

Generative Language Models and Open Notes: Exploring the Promise and Limitations.

Patients' online record access (ORA) is growing worldwide. In some countries, including the United States and Sweden, access is advanced with patients obtaining rapid access to their full records on the web including laboratory and test results, lists of prescribed medications, vaccinations, and even the very narrative reports written by clinicians (the latter, commonly referred to as "open notes"). In the United States, patient's ORA is also available in a downloadable form for use with other apps. While survey studies have shown that some patients report many benefits from ORA, there remain challenges with implementation around writing clinical documentation that patients may now read. With ORA, the functionality of the record is evolving; it is no longer only an aide memoire for doctors but also a communication tool for patients. Studies suggest that clinicians are changing how they write documentation, inviting worries about accuracy and completeness. Other concerns include work burdens; while few objective studies have examined the impact of ORA on workload, some research suggests that clinicians are spending more time writing notes and answering queries related to patients' records. Aimed at addressing some of these concerns, clinician and patient education strategies have been proposed. In this viewpoint paper, we explore these approaches and suggest another longer-term strategy: the use of generative artificial intelligence (AI) to support clinicians in documenting narrative summaries that patients will find easier to understand. Applied to narrative clinical documentation, we suggest that such approaches may significantly help preserve the accuracy of notes, strengthen writing clarity and signals of empathy and patient-centered care, and serve as a buffer against documentation work burdens. However, we also consider the current risks associated with existing generative AI. We emphasize that for this innovation to play a key role in ORA, the cocreation of clinical notes will be imperative. We also caution that clinicians will need to be supported in how to work alongside generative AI to optimize its considerable potential.

The SMART Text2FHIR Pipeline.

Objective: To implement an open source, free, and easily deployable high throughput natural language processing module to extract concepts from clinician notes and map them to Fast Healthcare Interoperability Resources (FHIR). Materials and Methods: Using a popular open-source NLP tool (Apache cTAKES), we create FHIR resources that use modifier extensions to represent negation and NLP sourcing, and another extension to represent provenance of extracted concepts. Results: The SMART Text2FHIR Pipeline is an open-source tool, released through standard package managers, and publicly available container images that implement the mappings, enabling ready conversion of clinical text to FHIR. Discussion: With the increased data liquidity because of new interoperability regulations, NLP processes that can output FHIR can enable a common language for transporting structured and unstructured data. This framework can be valuable for critical public health or clinical research use cases. Conclusion: Future work should include mapping more categories of NLP-extracted information into FHIR resources and mappings from additional open-source NLP tools.