To weight or not to weight? The effect of selection bias in 3 large electronic health record-linked biobanks and recommendations for practice.

OBJECTIVES: To develop recommendations regarding the use of weights to reduce selection bias for commonly performed analyses using electronic health record (EHR)-linked biobank data. MATERIALS AND METHODS: We mapped diagnosis (ICD code) data to standardized phecodes from 3 EHR-linked biobanks with varying recruitment strategies: All of Us (AOU; n = 244 071), Michigan Genomics Initiative (MGI; n = 81 243), and UK Biobank (UKB; n = 401 167). Using 2019 National Health Interview Survey data, we constructed selection weights for AOU and MGI to represent the US adult population more. We used weights previously developed for UKB to represent the UKB-eligible population. We conducted 4 common analyses comparing unweighted and weighted results. RESULTS: For AOU and MGI, estimated phecode prevalences decreased after weighting (weighted-unweighted median phecode prevalence ratio [MPR]: 0.82 and 0.61), while UKB estimates increased (MPR: 1.06). Weighting minimally impacted latent phenome dimensionality estimation. Comparing weighted versus unweighted phenome-wide association study for colorectal cancer, the strongest associations remained unaltered, with considerable overlap in significant hits. Weighting affected the estimated log-odds ratio for sex and colorectal cancer to align more closely with national registry-based estimates. DISCUSSION: Weighting had a limited impact on dimensionality estimation and large-scale hypothesis testing but impacted prevalence and association estimation. When interested in estimating effect size, specific signals from untargeted association analyses should be followed up by weighted analysis. CONCLUSION: EHR-linked biobanks should report recruitment and selection mechanisms and provide selection weights with defined target populations. Researchers should consider their intended estimands, specify source and target populations, and weight EHR-linked biobank analyses accordingly.

Neurological diagnoses in hospitalized COVID-19 patients associated with adverse outcomes: A multinational cohort study.

Few studies examining the patient outcomes of concurrent neurological manifestations during acute COVID-19 leveraged multinational cohorts of adults and children or distinguished between central and peripheral nervous system (CNS vs. PNS) involvement. Using a federated multinational network in which local clinicians and informatics experts curated the electronic health records data, we evaluated the risk of prolonged hospitalization and mortality in hospitalized COVID-19 patients from 21 healthcare systems across 7 countries. For adults, we used a federated learning approach whereby we ran Cox proportional hazard models locally at each healthcare system and performed a meta-analysis on the aggregated results to estimate the overall risk of adverse outcomes across our geographically diverse populations. For children, we reported descriptive statistics separately due to their low frequency of neurological involvement and poor outcomes. Among the 106,229 hospitalized COVID-19 patients (104,031 patients ≥18 years; 2,198 patients <18 years, January 2020-October 2021), 15,101 (14%) had at least one CNS diagnosis, while 2,788 (3%) had at least one PNS diagnosis. After controlling for demographics and pre-existing conditions, adults with CNS involvement had longer hospital stay (11 versus 6 days) and greater risk of (Hazard Ratio = 1.78) and faster time to death (12 versus 24 days) than patients with no neurological condition (NNC) during acute COVID-19 hospitalization. Adults with PNS involvement also had longer hospital stay but lower risk of mortality than the NNC group. Although children had a low frequency of neurological involvement during COVID-19 hospitalization, a substantially higher proportion of children with CNS involvement died compared to those with NNC (6% vs 1%). Overall, patients with concurrent CNS manifestation during acute COVID-19 hospitalization faced greater risks for adverse clinical outcomes than patients without any neurological diagnosis. Our global informatics framework using a federated approach (versus a centralized data collection approach) has utility for clinical discovery beyond COVID-19.

To weight or not to weight? Studying the effect of selection bias in three large EHR-linked biobanks.

Objective: To explore the role of selection bias adjustment by weighting electronic health record (EHR)-linked biobank data for commonly performed analyses. Materials and methods: We mapped diagnosis (ICD code) data to standardized phecodes from three EHR-linked biobanks with varying recruitment strategies: All of Us (AOU; n=244,071), Michigan Genomics Initiative (MGI; n=81,243), and UK Biobank (UKB; n=401,167). Using 2019 National Health Interview Survey data, we constructed selection weights for AOU and MGI to be more representative of the US adult population. We used weights previously developed for UKB to represent the UKB-eligible population. We conducted four common descriptive and analytic tasks comparing unweighted and weighted results. Results: For AOU and MGI, estimated phecode prevalences decreased after weighting (weighted-unweighted median phecode prevalence ratio [MPR]: 0.82 and 0.61), while UKB's estimates increased (MPR: 1.06). Weighting minimally impacted latent phenome dimensionality estimation. Comparing weighted versus unweighted PheWAS for colorectal cancer, the strongest associations remained unaltered and there was large overlap in significant hits. Weighting affected the estimated log-odds ratio for sex and colorectal cancer to align more closely with national registry-based estimates. Discussion: Weighting had limited impact on dimensionality estimation and large-scale hypothesis testing but impacted prevalence and association estimation more. Results from untargeted association analyses should be followed by weighted analysis when effect size estimation is of interest for specific signals. Conclusion: EHR-linked biobanks should report recruitment and selection mechanisms and provide selection weights with defined target populations. Researchers should consider their intended estimands, specify source and target populations, and weight EHR-linked biobank analyses accordingly.

Vaccine Effectiveness Against Long COVID in Children.

OBJECTIVES: Vaccination reduces the risk of acute coronavirus disease 2019 (COVID-19) in children, but it is less clear whether it protects against long COVID. We estimated vaccine effectiveness (VE) against long COVID in children aged 5 to 17 years. METHODS: This retrospective cohort study used data from 17 health systems in the RECOVER PCORnet electronic health record program for visits after vaccine availability. We examined both probable (symptom-based) and diagnosed long COVID after vaccination. RESULTS: The vaccination rate was 67% in the cohort of 1 037 936 children. The incidence of probable long COVID was 4.5% among patients with COVID-19, whereas diagnosed long COVID was 0.8%. Adjusted vaccine effectiveness within 12 months was 35.4% (95 CI 24.5-44.7) against probable long COVID and 41.7% (15.0-60.0) against diagnosed long COVID. VE was higher for adolescents (50.3% [36.6-61.0]) than children aged 5 to 11 (23.8% [4.9-39.0]). VE was higher at 6 months (61.4% [51.0-69.6]) but decreased to 10.6% (-26.8% to 37.0%) at 18-months. CONCLUSIONS: This large retrospective study shows moderate protective effect of severe acute respiratory coronavirus 2 vaccination against long COVID. The effect is stronger in adolescents, who have higher risk of long COVID, and wanes over time. Understanding VE mechanism against long COVID requires more study, including electronic health record sources and prospective data.

Vaccine Effectiveness Against Long COVID in Children: A Report from the RECOVER EHR Cohort.

Objective: Vaccination reduces the risk of acute COVID-19 in children, but it is less clear whether it protects against long COVID. We estimated vaccine effectiveness (VE) against long COVID in children aged 5-17 years. Methods: This retrospective cohort study used data from 17 health systems in the RECOVER PCORnet electronic health record (EHR) Program for visits between vaccine availability, and October 29, 2022. Conditional logistic regression was used to estimate VE against long COVID with matching on age group (5-11, 12-17) and time period and adjustment for sex, ethnicity, health system, comorbidity burden, and pre-exposure health care utilization. We examined both probable (symptom-based) and diagnosed long COVID in the year following vaccination. Results: The vaccination rate was 56% in the cohort of 1,037,936 children. The incidence of probable long COVID was 4.5% among patients with COVID-19, while diagnosed long COVID was 0.7%. Adjusted vaccine effectiveness within 12 months was 35.4% (95 CI 24.5 - 44.5) against probable long COVID and 41.7% (15.0 - 60.0) against diagnosed long COVID. VE was higher for adolescents 50.3% [36.3 - 61.0]) than children aged 5-11 (23.8% [4.9 - 39.0]). VE was higher at 6 months (61.4% [51.0 - 69.6]) but decreased to 10.6% (-26.8 - 37.0%) at 18-months. Discussion: This large retrospective study shows a moderate protective effect of SARS-CoV-2 vaccination against long COVID. The effect is stronger in adolescents, who have higher risk of long COVID, and wanes over time. Understanding VE mechanism against long COVID requires more study, including EHR sources and prospective data. Article Summary: Vaccination against COVID-19 has a protective effect against long COVID in children and adolescents. The effect wanes over time but remains significant at 12 months. What's Known on This Subject: Vaccines reduce the risk and severity of COVID-19 in children. There is evidence for reduced long COVID risk in adults who are vaccinated, but little information about similar effects for children and adolescents, who have distinct forms of long COVID. What This Study Adds: Using electronic health records from US health systems, we examined large cohorts of vaccinated and unvaccinated patients <18 years old and show that vaccination against COVID-19 is associated with reduced risk of long COVID for at least 12 months. Contributors' Statement: Drs. Hanieh Razzaghi and Charles Bailey conceptualized and designed the study, supervised analyses, drafted the initial manuscript, and critically reviewed and revised the manuscript.Drs. Christopher Forrest and Yong Chen designed the study and critically reviewed and revised the manuscript.Ms. Kathryn Hirabayashi, Ms. Andrea Allen, and Dr. Qiong Wu conducted analyses, and critically reviewed and revised the manuscript.Drs. Suchitra Rao, H Timothy Bunnell, Elizabeth A. Chrischilles, Lindsay G. Cowell, Mollie R. Cummins, David A. Hanauer, Benjamin D. Horne, Carol R. Horowitz, Ravi Jhaveri, Susan Kim, Aaron Mishkin, Jennifer A. Muszynski, Susanna Nagie, Nathan M. Pajor, Anuradha Paranjape, Hayden T. Schwenk, Marion R. Sills, Yacob G. Tedla, David A. Williams, and Ms. Miranda Higginbotham critically reviewed and revised the manuscript.All authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work. Authorship statement: Authorship has been determined according to ICMJE recommendations.

Clinical phenotypes and outcomes in children with multisystem inflammatory syndrome across SARS-CoV-2 variant eras: a multinational study from the 4CE consortium.

Background: Multisystem inflammatory syndrome in children (MIS-C) is a severe complication of SARS-CoV-2 infection. It remains unclear how MIS-C phenotypes vary across SARS-CoV-2 variants. We aimed to investigate clinical characteristics and outcomes of MIS-C across SARS-CoV-2 eras. Methods: We performed a multicentre observational retrospective study including seven paediatric hospitals in four countries (France, Spain, U.K., and U.S.). All consecutive confirmed patients with MIS-C hospitalised between February 1st, 2020, and May 31st, 2022, were included. Electronic Health Records (EHR) data were used to calculate pooled risk differences (RD) and effect sizes (ES) at site level, using Alpha as reference. Meta-analysis was used to pool data across sites. Findings: Of 598 patients with MIS-C (61% male, 39% female; mean age 9.7 years [SD 4.5]), 383 (64%) were admitted in the Alpha era, 111 (19%) in the Delta era, and 104 (17%) in the Omicron era. Compared with patients admitted in the Alpha era, those admitted in the Delta era were younger (ES -1.18 years [95% CI -2.05, -0.32]), had fewer respiratory symptoms (RD -0.15 [95% CI -0.33, -0.04]), less frequent non-cardiogenic shock or systemic inflammatory response syndrome (SIRS) (RD -0.35 [95% CI -0.64, -0.07]), lower lymphocyte count (ES -0.16 × 109/uL [95% CI -0.30, -0.01]), lower C-reactive protein (ES -28.5 mg/L [95% CI -46.3, -10.7]), and lower troponin (ES -0.14 ng/mL [95% CI -0.26, -0.03]). Patients admitted in the Omicron versus Alpha eras were younger (ES -1.6 years [95% CI -2.5, -0.8]), had less frequent SIRS (RD -0.18 [95% CI -0.30, -0.05]), lower lymphocyte count (ES -0.39 × 109/uL [95% CI -0.52, -0.25]), lower troponin (ES -0.16 ng/mL [95% CI -0.30, -0.01]) and less frequently received anticoagulation therapy (RD -0.19 [95% CI -0.37, -0.04]). Length of hospitalization was shorter in the Delta versus Alpha eras (-1.3 days [95% CI -2.3, -0.4]). Interpretation: Our study suggested that MIS-C clinical phenotypes varied across SARS-CoV-2 eras, with patients in Delta and Omicron eras being younger and less sick. EHR data can be effectively leveraged to identify rare complications of pandemic diseases and their variation over time. Funding: None.

Developing a Semantically Based Query Recommendation for an Electronic Medical Record Search Engine: Query Log Analysis and Design Implications.

BACKGROUND: An effective and scalable information retrieval (IR) system plays a crucial role in enabling clinicians and researchers to harness the valuable information present in electronic health records. In a previous study, we developed a prototype medical IR system, which incorporated a semantically based query recommendation (SBQR) feature. The system was evaluated empirically and demonstrated high perceived performance by end users. To delve deeper into the factors contributing to this perceived performance, we conducted a follow-up study using query log analysis. OBJECTIVE: One of the primary challenges faced in IR is that users often have limited knowledge regarding their specific information needs. Consequently, an IR system, particularly its user interface, needs to be thoughtfully designed to assist users through the iterative process of refining their queries as they encounter relevant documents during their search. To address these challenges, we incorporated "query recommendation" into our Electronic Medical Record Search Engine (EMERSE), drawing inspiration from the success of similar features in modern IR systems for general purposes. METHODS: The query log data analyzed in this study were collected during our previous experimental study, where we developed EMERSE with the SBQR feature. We implemented a logging mechanism to capture user query behaviors and the output of the IR system (retrieved documents). In this analysis, we compared the initial query entered by users with the query formulated with the assistance of the SBQR. By examining the results of this comparison, we could examine whether the use of SBQR helped in constructing improved queries that differed from the original ones. RESULTS: Our findings revealed that the first query entered without SBQR and the final query with SBQR assistance were highly similar (Jaccard similarity coefficient=0.77). This suggests that the perceived positive performance of the system was primarily attributed to the automatic query expansion facilitated by the SBQR rather than users manually manipulating their queries. In addition, through entropy analysis, we observed that search results converged in scenarios of moderate difficulty, and the degree of convergence correlated strongly with the perceived system performance. CONCLUSIONS: The study demonstrated the potential contribution of the SBQR in shaping participants' positive perceptions of system performance, contingent upon the difficulty of the search scenario. Medical IR systems should therefore consider incorporating an SBQR as a user-controlled option or a semiautomated feature. Future work entails redesigning the experiment in a more controlled manner and conducting multisite studies to demonstrate the effectiveness of EMERSE with SBQR for patient cohort identification. By further exploring and validating these findings, we can enhance the usability and functionality of medical IR systems in real-world settings.

Evaluation of software impact designed for biomedical research: Are we measuring what's meaningful?

Software is vital for the advancement of biology and medicine. Through analysis of usage and impact metrics of software, developers can help determine user and community engagement. These metrics can be used to justify additional funding, encourage additional use, and identify unanticipated use cases. Such analyses can help define improvement areas and assist with managing project resources. However, there are challenges associated with assessing usage and impact, many of which vary widely depending on the type of software being evaluated. These challenges involve issues of distorted, exaggerated, understated, or misleading metrics, as well as ethical and security concerns. More attention to the nuances, challenges, and considerations involved in capturing impact across the diverse spectrum of biological software is needed. Furthermore, some tools may be especially beneficial to a small audience, yet may not have comparatively compelling metrics of high usage. Although some principles are generally applicable, there is not a single perfect metric or approach to effectively evaluate a software tool's impact, as this depends on aspects unique to each tool, how it is used, and how one wishes to evaluate engagement. We propose more broadly applicable guidelines (such as infrastructure that supports the usage of software and the collection of metrics about usage), as well as strategies for various types of software and resources. We also highlight outstanding issues in the field regarding how communities measure or evaluate software impact. To gain a deeper understanding of the issues hindering software evaluations, as well as to determine what appears to be helpful, we performed a survey of participants involved with scientific software projects for the Informatics Technology for Cancer Research (ITCR) program funded by the National Cancer Institute (NCI). We also investigated software among this scientific community and others to assess how often infrastructure that supports such evaluations is implemented and how this impacts rates of papers describing usage of the software. We find that although developers recognize the utility of analyzing data related to the impact or usage of their software, they struggle to find the time or funding to support such analyses. We also find that infrastructure such as social media presence, more in-depth documentation, the presence of software health metrics, and clear information on how to contact developers seem to be associated with increased usage rates. Our findings can help scientific software developers make the most out of the evaluations of their software so that they can more fully benefit from such assessments.

Informative missingness: What can we learn from patterns in missing laboratory data in the electronic health record?

BACKGROUND: In electronic health records, patterns of missing laboratory test results could capture patients' course of disease as well as ​​reflect clinician's concerns or worries for possible conditions. These patterns are often understudied and overlooked. This study aims to identify informative patterns of missingness among laboratory data collected across 15 healthcare system sites in three countries for COVID-19 inpatients. METHODS: We collected and analyzed demographic, diagnosis, and laboratory data for 69,939 patients with positive COVID-19 PCR tests across three countries from 1 January 2020 through 30 September 2021. We analyzed missing laboratory measurements across sites, missingness stratification by demographic variables, temporal trends of missingness, correlations between labs based on missingness indicators over time, and clustering of groups of labs based on their missingness/ordering pattern. RESULTS: With these analyses, we identified mapping issues faced in seven out of 15 sites. We also identified nuances in data collection and variable definition for the various sites. Temporal trend analyses may support the use of laboratory test result missingness patterns in identifying severe COVID-19 patients. Lastly, using missingness patterns, we determined relationships between various labs that reflect clinical behaviors. CONCLUSION: In this work, we use computational approaches to relate missingness patterns to hospital treatment capacity and highlight the heterogeneity of looking at COVID-19 over time and at multiple sites, where there might be different phases, policies, etc. Changes in missingness could suggest a change in a patient's condition, and patterns of missingness among laboratory measurements could potentially identify clinical outcomes. This allows sites to consider missing data as informative to analyses and help researchers identify which sites are better poised to study particular questions.

A pilot intervention of using a mobile health app (ONC Roadmap) to enhance health-related quality of life in family caregivers of pediatric patients with cancer.

Background: The Roadmap mobile health (mHealth) app was developed to provide health-related quality of life (HRQOL) support for family caregivers of patients with cancer. Methods: Eligibility included: family caregivers (age ≥18 years) who self-reported as the primary caregiver of their pediatric patient with cancer; patients (age ≥5 years) who were receiving cancer care at the University of Michigan. Feasibility was calculated as the percentage of caregivers who logged into ONC Roadmap and engaged with it at least twice weekly for at least 50% of the 120-day study duration. Feasibility and acceptability was also assessed through a Feasibility and Acceptability questionnaire and the Mobile App Rating Scale to specifically assess app-quality. Exploratory analyses were also conducted to assess HRQOL self- or parent proxy assessments and physiological data capture. Results: Between September 2020-September 2021, 100 participants (or 50 caregiver-patient dyads) consented and enrolled in the ONC Roadmap study for 120-days. Feasibility of the study was met, wherein the majority of caregivers (N=32; 65%) logged into ONC Roadmap and engaged with it at least twice weekly for at least 50% of the study duration (defined a priori in the Protocol). The Feasibility and Acceptability questionnaire responses indicated that the study was feasible and acceptable with the majority (>50%) reporting Agree or Strongly Agree with positive Net Favorability [(Agree + Strongly Agree) - (Disagree + Totally Disagree)] in each of the domains (e.g., Fitbit use, ONC Roadmap use, completing longitudinal assessments, engaging in similar future study, study expectations). Improvements were seen across the majority of the mental HRQOL domains across all groups; even though underpowered, there were significant improvements in caregiver-specific aspects of HRQOL and anxiety and in depression and fatigue for children (ages 8-17 years), and a trend toward improvement in depression for children ages 8-17 years and in fatigue for adult patients. Conclusions: This study supports that mHealth technology may be a promising platform to provide HRQOL support for caregivers of pediatric patients with cancer. Importantly, the findings suggest that the study protocol was feasible, and participants were favorable to participate in future studies of this intervention alongside routine cancer care delivery.

Acute respiratory distress syndrome after SARS-CoV-2 infection on young adult population: International observational federated study based on electronic health records through the 4CE consortium.

PURPOSE: In young adults (18 to 49 years old), investigation of the acute respiratory distress syndrome (ARDS) after severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection has been limited. We evaluated the risk factors and outcomes of ARDS following infection with SARS-CoV-2 in a young adult population. METHODS: A retrospective cohort study was conducted between January 1st, 2020 and February 28th, 2021 using patient-level electronic health records (EHR), across 241 United States hospitals and 43 European hospitals participating in the Consortium for Clinical Characterization of COVID-19 by EHR (4CE). To identify the risk factors associated with ARDS, we compared young patients with and without ARDS through a federated analysis. We further compared the outcomes between young and old patients with ARDS. RESULTS: Among the 75,377 hospitalized patients with positive SARS-CoV-2 PCR, 1001 young adults presented with ARDS (7.8% of young hospitalized adults). Their mortality rate at 90 days was 16.2% and they presented with a similar complication rate for infection than older adults with ARDS. Peptic ulcer disease, paralysis, obesity, congestive heart failure, valvular disease, diabetes, chronic pulmonary disease and liver disease were associated with a higher risk of ARDS. We described a high prevalence of obesity (53%), hypertension (38%- although not significantly associated with ARDS), and diabetes (32%). CONCLUSION: Trough an innovative method, a large international cohort study of young adults developing ARDS after SARS-CoV-2 infection has been gather. It demonstrated the poor outcomes of this population and associated risk factor.

Long-term kidney function recovery and mortality after COVID-19-associated acute kidney injury: An international multi-centre observational cohort study.

Background: While acute kidney injury (AKI) is a common complication in COVID-19, data on post-AKI kidney function recovery and the clinical factors associated with poor kidney function recovery is lacking. Methods: A retrospective multi-centre observational cohort study comprising 12,891 hospitalized patients aged 18 years or older with a diagnosis of SARS-CoV-2 infection confirmed by polymerase chain reaction from 1 January 2020 to 10 September 2020, and with at least one serum creatinine value 1-365 days prior to admission. Mortality and serum creatinine values were obtained up to 10 September 2021. Findings: Advanced age (HR 2.77, 95%CI 2.53-3.04, p < 0.0001), severe COVID-19 (HR 2.91, 95%CI 2.03-4.17, p < 0.0001), severe AKI (KDIGO stage 3: HR 4.22, 95%CI 3.55-5.00, p < 0.0001), and ischemic heart disease (HR 1.26, 95%CI 1.14-1.39, p < 0.0001) were associated with worse mortality outcomes. AKI severity (KDIGO stage 3: HR 0.41, 95%CI 0.37-0.46, p < 0.0001) was associated with worse kidney function recovery, whereas remdesivir use (HR 1.34, 95%CI 1.17-1.54, p < 0.0001) was associated with better kidney function recovery. In a subset of patients without chronic kidney disease, advanced age (HR 1.38, 95%CI 1.20-1.58, p < 0.0001), male sex (HR 1.67, 95%CI 1.45-1.93, p < 0.0001), severe AKI (KDIGO stage 3: HR 11.68, 95%CI 9.80-13.91, p < 0.0001), and hypertension (HR 1.22, 95%CI 1.10-1.36, p = 0.0002) were associated with post-AKI kidney function impairment. Furthermore, patients with COVID-19-associated AKI had significant and persistent elevations of baseline serum creatinine 125% or more at 180 days (RR 1.49, 95%CI 1.32-1.67) and 365 days (RR 1.54, 95%CI 1.21-1.96) compared to COVID-19 patients with no AKI. Interpretation: COVID-19-associated AKI was associated with higher mortality, and severe COVID-19-associated AKI was associated with worse long-term post-AKI kidney function recovery. Funding: Authors are supported by various funders, with full details stated in the acknowledgement section.

Hospitalizations Associated With Mental Health Conditions Among Adolescents in the US and France During the COVID-19 Pandemic.

Importance: The COVID-19 pandemic has been associated with an increase in mental health diagnoses among adolescents, though the extent of the increase, particularly for severe cases requiring hospitalization, has not been well characterized. Large-scale federated informatics approaches provide the ability to efficiently and securely query health care data sets to assess and monitor hospitalization patterns for mental health conditions among adolescents. Objective: To estimate changes in the proportion of hospitalizations associated with mental health conditions among adolescents following onset of the COVID-19 pandemic. Design, Setting, and Participants: This retrospective, multisite cohort study of adolescents 11 to 17 years of age who were hospitalized with at least 1 mental health condition diagnosis between February 1, 2019, and April 30, 2021, used patient-level data from electronic health records of 8 children's hospitals in the US and France. Main Outcomes and Measures: Change in the monthly proportion of mental health condition-associated hospitalizations between the prepandemic (February 1, 2019, to March 31, 2020) and pandemic (April 1, 2020, to April 30, 2021) periods using interrupted time series analysis. Results: There were 9696 adolescents hospitalized with a mental health condition during the prepandemic period (5966 [61.5%] female) and 11 101 during the pandemic period (7603 [68.5%] female). The mean (SD) age in the prepandemic cohort was 14.6 (1.9) years and in the pandemic cohort, 14.7 (1.8) years. The most prevalent diagnoses during the pandemic were anxiety (6066 [57.4%]), depression (5065 [48.0%]), and suicidality or self-injury (4673 [44.2%]). There was an increase in the proportions of monthly hospitalizations during the pandemic for anxiety (0.55%; 95% CI, 0.26%-0.84%), depression (0.50%; 95% CI, 0.19%-0.79%), and suicidality or self-injury (0.38%; 95% CI, 0.08%-0.68%). There was an estimated 0.60% increase (95% CI, 0.31%-0.89%) overall in the monthly proportion of mental health-associated hospitalizations following onset of the pandemic compared with the prepandemic period. Conclusions and Relevance: In this cohort study, onset of the COVID-19 pandemic was associated with increased hospitalizations with mental health diagnoses among adolescents. These findings support the need for greater resources within children's hospitals to care for adolescents with mental health conditions during the pandemic and beyond.

SurvMaximin: Robust federated approach to transporting survival risk prediction models.

OBJECTIVE: For multi-center heterogeneous Real-World Data (RWD) with time-to-event outcomes and high-dimensional features, we propose the SurvMaximin algorithm to estimate Cox model feature coefficients for a target population by borrowing summary information from a set of health care centers without sharing patient-level information. MATERIALS AND METHODS: For each of the centers from which we want to borrow information to improve the prediction performance for the target population, a penalized Cox model is fitted to estimate feature coefficients for the center. Using estimated feature coefficients and the covariance matrix of the target population, we then obtain a SurvMaximin estimated set of feature coefficients for the target population. The target population can be an entire cohort comprised of all centers, corresponding to federated learning, or a single center, corresponding to transfer learning. RESULTS: Simulation studies and a real-world international electronic health records application study, with 15 participating health care centers across three countries (France, Germany, and the U.S.), show that the proposed SurvMaximin algorithm achieves comparable or higher accuracy compared with the estimator using only the information of the target site and other existing methods. The SurvMaximin estimator is robust to variations in sample sizes and estimated feature coefficients between centers, which amounts to significantly improved estimates for target sites with fewer observations. CONCLUSIONS: The SurvMaximin method is well suited for both federated and transfer learning in the high-dimensional survival analysis setting. SurvMaximin only requires a one-time summary information exchange from participating centers. Estimated regression vectors can be very heterogeneous. SurvMaximin provides robust Cox feature coefficient estimates without outcome information in the target population and is privacy-preserving.

International electronic health record-derived post-acute sequelae profiles of COVID-19 patients.

The risk profiles of post-acute sequelae of COVID-19 (PASC) have not been well characterized in multi-national settings with appropriate controls. We leveraged electronic health record (EHR) data from 277 international hospitals representing 414,602 patients with COVID-19, 2.3 million control patients without COVID-19 in the inpatient and outpatient settings, and over 221 million diagnosis codes to systematically identify new-onset conditions enriched among patients with COVID-19 during the post-acute period. Compared to inpatient controls, inpatient COVID-19 cases were at significant risk for angina pectoris (RR 1.30, 95% CI 1.09-1.55), heart failure (RR 1.22, 95% CI 1.10-1.35), cognitive dysfunctions (RR 1.18, 95% CI 1.07-1.31), and fatigue (RR 1.18, 95% CI 1.07-1.30). Relative to outpatient controls, outpatient COVID-19 cases were at risk for pulmonary embolism (RR 2.10, 95% CI 1.58-2.76), venous embolism (RR 1.34, 95% CI 1.17-1.54), atrial fibrillation (RR 1.30, 95% CI 1.13-1.50), type 2 diabetes (RR 1.26, 95% CI 1.16-1.36) and vitamin D deficiency (RR 1.19, 95% CI 1.09-1.30). Outpatient COVID-19 cases were also at risk for loss of smell and taste (RR 2.42, 95% CI 1.90-3.06), inflammatory neuropathy (RR 1.66, 95% CI 1.21-2.27), and cognitive dysfunction (RR 1.18, 95% CI 1.04-1.33). The incidence of post-acute cardiovascular and pulmonary conditions decreased across time among inpatient cases while the incidence of cardiovascular, digestive, and metabolic conditions increased among outpatient cases. Our study, based on a federated international network, systematically identified robust conditions associated with PASC compared to control groups, underscoring the multifaceted cardiovascular and neurological phenotype profiles of PASC.

International comparisons of laboratory values from the 4CE collaborative to predict COVID-19 mortality.

Given the growing number of prediction algorithms developed to predict COVID-19 mortality, we evaluated the transportability of a mortality prediction algorithm using a multi-national network of healthcare systems. We predicted COVID-19 mortality using baseline commonly measured laboratory values and standard demographic and clinical covariates across healthcare systems, countries, and continents. Specifically, we trained a Cox regression model with nine measured laboratory test values, standard demographics at admission, and comorbidity burden pre-admission. These models were compared at site, country, and continent level. Of the 39,969 hospitalized patients with COVID-19 (68.6% male), 5717 (14.3%) died. In the Cox model, age, albumin, AST, creatine, CRP, and white blood cell count are most predictive of mortality. The baseline covariates are more predictive of mortality during the early days of COVID-19 hospitalization. Models trained at healthcare systems with larger cohort size largely retain good transportability performance when porting to different sites. The combination of routine laboratory test values at admission along with basic demographic features can predict mortality in patients hospitalized with COVID-19. Importantly, this potentially deployable model differs from prior work by demonstrating not only consistent performance but also reliable transportability across healthcare systems in the US and Europe, highlighting the generalizability of this model and the overall approach.

Broadening Access to STEM through the Community College: Investigating the Role of Course-Based Research Experiences (CREs).

Broadening access to science, technology, engineering, and mathematics (STEM) professions through the provision of early-career research experiences for a wide range of demographic groups is important for the diversification of the STEM workforce. The size and diversity of the community college system make it a prime educational site for achieving this aim. However, some evidence shows that women and Black, Latinx, and Native American student groups have been hindered in STEM at the community college level. One option for enhancing persistence in STEM is to incorporate the course-based research experiences (CREs) into the curriculum as a replacement for the prevalent traditional laboratory. This can be achieved through the integration of community colleges within extant, multi-institutional CREs such as the SEA-PHAGES program. Using a propensity score-matching technique, students in a CRE and traditional laboratory were compared on a range of psychosocial variables (project ownership, self-efficacy, science identity, scientific community values, and networking). Results revealed higher ratings for women and persons excluded because of their ethnicity or race (PEERs) in the SEA-PHAGES program on important predictors of persistence such as project ownership and science identity. This suggests that the usage of CREs at community colleges could have positive effects in addressing the gender gap for women and enhance inclusiveness for PEER students in STEM.

Changes in laboratory value improvement and mortality rates over the course of the pandemic: an international retrospective cohort study of hospitalised patients infected with SARS-CoV-2.

OBJECTIVE: To assess changes in international mortality rates and laboratory recovery rates during hospitalisation for patients hospitalised with SARS-CoV-2 between the first wave (1 March to 30 June 2020) and the second wave (1 July 2020 to 31 January 2021) of the COVID-19 pandemic. DESIGN, SETTING AND PARTICIPANTS: This is a retrospective cohort study of 83 178 hospitalised patients admitted between 7 days before or 14 days after PCR-confirmed SARS-CoV-2 infection within the Consortium for Clinical Characterization of COVID-19 by Electronic Health Record, an international multihealthcare system collaborative of 288 hospitals in the USA and Europe. The laboratory recovery rates and mortality rates over time were compared between the two waves of the pandemic. PRIMARY AND SECONDARY OUTCOME MEASURES: The primary outcome was all-cause mortality rate within 28 days after hospitalisation stratified by predicted low, medium and high mortality risk at baseline. The secondary outcome was the average rate of change in laboratory values during the first week of hospitalisation. RESULTS: Baseline Charlson Comorbidity Index and laboratory values at admission were not significantly different between the first and second waves. The improvement in laboratory values over time was faster in the second wave compared with the first. The average C reactive protein rate of change was -4.72 mg/dL vs -4.14 mg/dL per day (p=0.05). The mortality rates within each risk category significantly decreased over time, with the most substantial decrease in the high-risk group (42.3% in March-April 2020 vs 30.8% in November 2020 to January 2021, p<0.001) and a moderate decrease in the intermediate-risk group (21.5% in March-April 2020 vs 14.3% in November 2020 to January 2021, p<0.001). CONCLUSIONS: Admission profiles of patients hospitalised with SARS-CoV-2 infection did not differ greatly between the first and second waves of the pandemic, but there were notable differences in laboratory improvement rates during hospitalisation. Mortality risks among patients with similar risk profiles decreased over the course of the pandemic. The improvement in laboratory values and mortality risk was consistent across multiple countries.

Instructional Models for Course-Based Research Experience (CRE) Teaching.

The course-based research experience (CRE) with its documented educational benefits is increasingly being implemented in science, technology, engineering, and mathematics education. This article reports on a study that was done over a period of 3 years to explicate the instructional processes involved in teaching an undergraduate CRE. One hundred and two instructors from the established and large multi-institutional SEA-PHAGES program were surveyed for their understanding of the aims and practices of CRE teaching. This was followed by large-scale feedback sessions with the cohort of instructors at the annual SEA Faculty Meeting and subsequently with a small focus group of expert CRE instructors. Using a qualitative content analysis approach, the survey data were analyzed for the aims of inquiry instruction and pedagogical practices used to achieve these goals. The results characterize CRE inquiry teaching as involving three instructional models: 1) being a scientist and generating data; 2) teaching procedural knowledge; and 3) fostering project ownership. Each of these models is explicated and visualized in terms of the specific pedagogical practices and their relationships. The models present a complex picture of the ways in which CRE instruction is conducted on a daily basis and can inform instructors and institutions new to CRE teaching.

Design, Implementation, and Usability of the Electronic Medical Record Search Engine (EMERSE) Tool.

Free text forms of clinical documentation stored in electronic health records contain a trove of data for researchers and clinicians alike. However, often these data are challenging to use and not easily accessible. EMERSE, a clinical documentation search and data abstraction tool developed by the University of Michigan, helps users in the task of searching through free text notes in clinical documentation. This study evaluates the usability and user experience of the EMERSE system, and draws inferences for the design of such systems. The study was conducted in 3 phases. In Phase 1, interviews with site administrators investigated factors that facilitate or hinder the implementation and adoption of EMERSE. Phase 2 employed semi-structured interviews to understand the uses, benefits, and limitations of the system from the perspective of experienced users. In Phase 3, system-naive users performed a set of basic workflow tasks, then completed post-activity questions and surveys to evaluate the intuitiveness and usability of the system. Participants rated the system exceptionally high on usability, user interface satisfaction, and perceived usefulness. Feedback also indicated that improvements could be made in visual contrast, affordances, and scope of notes indexed. These results indicate that tools such as EMERSE should be highly intuitive, attractive, and moderately customizable. This paper discusses some aspects of what may contribute to a system having such characteristics.