Randomized Controlled Comparative Effectiveness Trial of Risk Model-Guided Clinical Decision Support for Suicide Screening.

Suicide prevention requires risk identification, appropriate intervention, and follow-up. Traditional risk identification relies on patient self-reporting, support network reporting, or face-to-face screening with validated instruments or history and physical exam. In the last decade, statistical risk models have been studied and more recently deployed to augment clinical judgment. Models have generally been found to be low precision or problematic at scale due to low incidence. Few have been tested in clinical practice, and none have been tested in clinical trials to our knowledge. Methods: We report the results of a pragmatic randomized controlled trial (RCT) in three outpatient adult Neurology clinic settings. This two-arm trial compared the effectiveness of Interruptive and Non-Interruptive Clinical Decision Support (CDS) to prompt further screening of suicidal ideation for those predicted to be high risk using a real-time, validated statistical risk model of suicide attempt risk, with the decision to screen as the primary end point. Secondary outcomes included rates of suicidal ideation and attempts in both arms. Manual chart review of every trial encounter was used to determine if suicide risk assessment was subsequently documented. Results: From August 16, 2022, through February 16, 2023, our study randomized 596 patient encounters across 561 patients for providers to receive either Interruptive or Non-Interruptive CDS in a 1:1 ratio. Adjusting for provider cluster effects, Interruptive CDS led to significantly higher numbers of decisions to screen (42%=121/289 encounters) compared to Non-Interruptive CDS (4%=12/307) (odds ratio=17.7, p-value <0.001). Secondarily, no documented episodes of suicidal ideation or attempts occurred in either arm. While the proportion of documented assessments among those noting the decision to screen was higher for providers in the Non-Interruptive arm (92%=11/12) than in the Interruptive arm (52%=63/121), the interruptive CDS was associated with more frequent documentation of suicide risk assessment (63/289 encounters compared to 11/307, p-value<0.001). Conclusions: In this pragmatic RCT of real-time predictive CDS to guide suicide risk assessment, Interruptive CDS led to higher numbers of decisions to screen and documented suicide risk assessments. Well-powered large-scale trials randomizing this type of CDS compared to standard of care are indicated to measure effectiveness in reducing suicidal self-harm. ClinicalTrials.gov Identifier: NCT05312437.

Why do probabilistic clinical models fail to transport between sites.

The rising popularity of artificial intelligence in healthcare is highlighting the problem that a computational model achieving super-human clinical performance at its training sites may perform substantially worse at new sites. In this perspective, we argue that we should typically expect this failure to transport, and we present common sources for it, divided into those under the control of the experimenter and those inherent to the clinical data-generating process. Of the inherent sources we look a little deeper into site-specific clinical practices that can affect the data distribution, and propose a potential solution intended to isolate the imprint of those practices on the data from the patterns of disease cause and effect that are the usual target of probabilistic clinical models.

Soft phenotyping for sepsis via EHR time-aware soft clustering.

OBJECTIVE: Sepsis is one of the most serious hospital conditions associated with high mortality. Sepsis is the result of a dysregulated immune response to infection that can lead to multiple organ dysfunction and death. Due to the wide variability in the causes of sepsis, clinical presentation, and the recovery trajectories, identifying sepsis sub-phenotypes is crucial to advance our understanding of sepsis characterization, to choose targeted treatments and optimal timing of interventions, and to improve prognostication. Prior studies have described different sub-phenotypes of sepsis using organ-specific characteristics. These studies applied clustering algorithms to electronic health records (EHRs) to identify disease sub-phenotypes. However, prior approaches did not capture temporal information and made uncertain assumptions about the relationships among the sub-phenotypes for clustering procedures. METHODS: We developed a time-aware soft clustering algorithm guided by clinical variables to identify sepsis sub-phenotypes using data available in the EHR. RESULTS: We identified six novel sepsis hybrid sub-phenotypes and evaluated them for medical plausibility. In addition, we built an early-warning sepsis prediction model using logistic regression. CONCLUSION: Our results suggest that these novel sepsis hybrid sub-phenotypes are promising to provide more accurate information on sepsis-related organ dysfunction and sepsis recovery trajectories which can be important to inform management decisions and sepsis prognosis.

Next-generation phenotyping: introducing phecodeX for enhanced discovery research in medical phenomics.

MOTIVATION: Phecodes are widely used and easily adapted phenotypes based on International Classification of Diseases codes. The current version of phecodes (v1.2) was designed primarily to study common/complex diseases diagnosed in adults; however, there are numerous limitations in the codes and their structure. RESULTS: Here, we present phecodeX, an expanded version of phecodes with a revised structure and 1,761 new codes. PhecodeX adds granularity to phenotypes in key disease domains that are under-represented in the current phecode structure-including infectious disease, pregnancy, congenital anomalies, and neonatology-and is a more robust representation of the medical phenome for global use in discovery research. AVAILABILITY AND IMPLEMENTATION: phecodeX is available at https://github.com/PheWAS/phecodeX.

Timeline Registration for Electronic Health Records.

Electronic Health Record (EHR) data are captured over time as patients receive care. Accordingly, variations among patients, such as when a patient presents for care during the course of a disease, introduce bias into standard longitudinal EHR data analysis methods. We, therefore, aim to provide an alignment method that reduces this bias. We structure this task as a registration problem. While limited prior research on longitudinal EHR data considered registration, we propose a robust registration method to provide better data alignment by estimating the optimum time shift at each time point. We validate the proposed method for mortality prediction. We utilize a Recurrent Neural Network (RNN), time-varying Cox regression model, and Logistic Regression (LR) for mortality prediction. Results suggest our proposed registration method enhances mortality prediction with at least a 1-2% increase in major evaluation metrics utilized.

Knowledgebase strategies to aid interpretation of clinical correlation research.

OBJECTIVE: Knowledgebases are needed to clarify correlations observed in real-world electronic health record (EHR) data. We posit design principles, present a unifying framework, and report a test of concept. MATERIALS AND METHODS: We structured a knowledge framework along 3 axes: condition of interest, knowledge source, and taxonomy. In our test of concept, we used hypertension as our condition of interest, literature and VanderbiltDDx knowledgebase as sources, and phecodes as our taxonomy. In a cohort of 832 566 deidentified EHRs, we modeled blood pressure and heart rate by sex and age, classified individuals by hypertensive status, and ran a Phenome-wide Association Study (PheWAS) for hypertension. We compared the correlations from PheWAS to the associations in our knowledgebase. RESULTS: We produced PhecodeKbHtn: a knowledgebase comprising 167 hypertension-associated diseases, 15 of which were also negatively associated with blood pressure (pos+neg). Our hypertension PheWAS included 1914 phecodes, 129 of which were in the PhecodeKbHtn. Among the PheWAS association results, phecodes that were in PhecodeKbHtn had larger effect sizes compared with those phecodes not in the knowledgebase. DISCUSSION: Each source contributed unique and additive associations. Models of blood pressure and heart rate by age and sex were consistent with prior cohort studies. All but 4 PheWAS positive and negative correlations for phecodes in PhecodeKbHtn may be explained by knowledgebase associations, hypertensive cardiac complications, or causes of hypertension independently associated with hypotension. CONCLUSION: It is feasible to assemble a knowledgebase that is compatible with EHR data to aid interpretation of clinical correlation research.

NLM and the IAIMS initiative: Cross-institutional academic/advanced systems contributing to the evolution of networked information and resources.

The Integrated Academic/Advanced Information Systems (IAIMS) program began in 1983 and was based on a study by the Association of American Medical Colleges (AAMC). Donald A.B. Lindberg M.D. was a member of the AAMC Advisory Committee. The U.S. National Library of Medicine (NLM) grants for IAIMS were initiated in 1984 the same year Dr. Lindberg became Director of the NLM. This chapter presents an overview of IAIMS and its progression through three stages with Dr. Lindberg's leadership.

NLM and the IAIMS Initiative: Cross-Institutional Academic/Advanced Systems Contributing to the Evolution of Networked Information and Resources.

The Integrated Academic/Advanced Information Systems (IAIMS) program began in 1983 and was based on a study by the Association of American Medical Colleges (AAMC). Donald A.B. Lindberg M.D. was a member of the AAMC Advisory Committee. The U.S. National Library of Medicine (NLM) grants for IAIMS were initiated in 1984 the same year Dr. Lindberg became Director of the NLM. This chapter presents an overview of IAIMS and its progression through three stages with Dr. Lindberg's leadership.

Prospective Validation of an Electronic Health Record-Based, Real-Time Suicide Risk Model.

Importance: Numerous prognostic models of suicide risk have been published, but few have been implemented outside of integrated managed care systems. Objective: To evaluate performance of a suicide attempt risk prediction model implemented in a vendor-supplied electronic health record to predict subsequent (1) suicidal ideation and (2) suicide attempt. Design, Setting, and Participants: This observational cohort study evaluated implementation of a suicide attempt prediction model in live clinical systems without alerting. The cohort comprised patients seen for any reason in adult inpatient, emergency department, and ambulatory surgery settings at an academic medical center in the mid-South from June 2019 to April 2020. Main Outcomes and Measures: Primary measures assessed external, prospective, and concurrent validity. Manual medical record validation of coded suicide attempts confirmed incident behaviors with intent to die. Subgroup analyses were performed based on demographic characteristics, relevant clinical context/setting, and presence or absence of universal screening. Performance was evaluated using discrimination (number needed to screen, C statistics, positive/negative predictive values) and calibration (Spiegelhalter z statistic). Recalibration was performed with logistic calibration. Results: The system generated 115 905 predictions for 77 973 patients (42 490 [54%] men, 35 404 [45%] women, 60 586 [78%] White, 12 620 [16%] Black). Numbers needed to screen in highest risk quantiles were 23 and 271 for suicidal ideation and attempt, respectively. Performance was maintained across demographic subgroups. Numbers needed to screen for suicide attempt by sex were 256 for men and 323 for women; and by race: 373, 176, and 407 for White, Black, and non-White/non-Black patients, respectively. Model C statistics were, across the health system: 0.836 (95% CI, 0.836-0.837); adult hospital: 0.77 (95% CI, 0.77-0.772); emergency department: 0.778 (95% CI, 0.777-0.778); psychiatry inpatient settings: 0.634 (95% CI, 0.633-0.636). Predictions were initially miscalibrated (Spiegelhalter z = -3.1; P = .001) with improvement after recalibration (Spiegelhalter z = 1.1; P = .26). Conclusions and Relevance: In this study, this real-time predictive model of suicide attempt risk showed reasonable numbers needed to screen in nonpsychiatric specialty settings in a large clinical system. Assuming that research-valid models will translate without performing this type of analysis risks inaccuracy in clinical practice, misclassification of risk, wasted effort, and missed opportunity to correct and prevent such problems. The next step is careful pairing with low-cost, low-harm preventive strategies in a pragmatic trial of effectiveness in preventing future suicidality.

Developing an Implementation Strategy for Systematic Measurement of Patient-Reported Outcomes at an Academic Health Center.

EXECUTIVE SUMMARY: Patient-reported outcome measures (PROMs) are used in research and have the potential to improve clinical care. We sought to develop a strategy for integrating PROMs into routine clinical care at an academic health center. The implementation strategy consisted of three phases. The first, exploratory phase, focused on engaging leadership and conducting an inventory of current efforts to collect PROMs. The inventory revealed 87 patient-reported outcome efforts, 47 of which used validated PROMs (62% for research, 21% for clinical care, 17% for quality). In the second, preparatory phase, we identified three pilot implementation sites chosen with facilitators determined in the exploratory phase. Using data from local needs assessments at the pilot sites, we constructed a timeline for inclusion of PROM efforts across the clinical enterprise. In the third phase, we adapted a technology platform for capturing PROMs using the electronic health record and began implementing this platform at the pilot sites. We found that integrating PROMs into routine clinical practice is highly complex. This complexity necessitates change management at the enterprise level.

Coordinated Management of Academic Health Centers.

Academic health centers (AHCs) are the nation's primary resource for healthcare discovery, innovation, and training. US healthcare revenue growth has declined sharply since 2009, and is forecast to remain well below historic levels for the foreseeable future. As the cost of education and research at nearly all AHCs is heavily subsidized through large transfers from clinical care margins, our institutions face a mounting crisis. Choices centering on how to increase the cost-effectiveness of the AHC enterprise require unprecedented levels of alignment to preserve an environment that nurtures creativity. Management processes require governance models that clarify decision rights while harnessing the talents and the intellectual capital of a large, diverse enterprise to nimbly address unfamiliar organizational challenges. This paper describes key leadership tactics aimed at propelling AHCs along this journey - one that requires from all leaders a commitment to resilience, optimism, and willingness to embrace change.

Getting started: Reply to Condon et al. (2017) and Rossiter (2017).

Condon, Weston, and Hill (2017) and Rossiter (2017) expressed concerns about the recommended panel of psychosocial vital signs that should be included in electronic health records (EHRs). Condon et al. (2017) would prefer a broader array of measures and Rossiter (2017) a set of optimal measures based on consensus. Our task was to identify a core set of measures to be used now in all EHRs based on usefulness and low burden. We anticipate that with new evidence, changing needs, and greater input from stakeholders, the recommended measures will be elaborated or modified. (PsycINFO Database Record

National Academy of Medicine Social and Behavioral Measures: Associations With Self-Reported Health.

INTRODUCTION: Social and behavioral factors play important roles in physical and mental health; however, they are not routinely assessed in the healthcare system. A brief panel of measures of social and behavioral determinants of health (SBDs) were recommended in a National Academy of Medicine report for use in electronic health records. Initial testing of the panel established feasibility of use and robustness of the measures. This study evaluates their convergent and divergent validity in relation to self-reported physical and mental health and social desirability bias. METHODS: Adults, aged ≥18 years, were recruited through Qualtrics online panel survey platform in 2015 (data analyzed in 2015-2016). Participants completed the (1) panel of SBD measures; (2) 12-Item Short Form Health Survey to assess associations with global physical and mental health; and (3) Marlowe-Crowne Social Desirability Scale short form to assess whether social desirability influenced associations between SBD measures and self-reported health. RESULTS: The sample included 513 participants (mean age, 47.9 [SD=14.2] years; 65.5% female). Several SBD domain measures were associated with physical and mental health. Adjusting for age, poorer physical and mental health were observed among participants reporting higher levels of financial resource strain, stress, depression, physical inactivity, current tobacco use, and a positive score for intimate partner violence. These associations remained significant after adjustment for social desirability bias. CONCLUSIONS: SBD domains were associated with global measures of physical and mental health and were not impacted by social desirability bias. The panel of SBD measures should now be tested in clinical settings.