Seek COVER: using a disease proxy to rapidly develop and validate a personalized risk calculator for COVID-19 outcomes in an international network.

BACKGROUND: We investigated whether we could use influenza data to develop prediction models for COVID-19 to increase the speed at which prediction models can reliably be developed and validated early in a pandemic. We developed COVID-19 Estimated Risk (COVER) scores that quantify a patient's risk of hospital admission with pneumonia (COVER-H), hospitalization with pneumonia requiring intensive services or death (COVER-I), or fatality (COVER-F) in the 30-days following COVID-19 diagnosis using historical data from patients with influenza or flu-like symptoms and tested this in COVID-19 patients. METHODS: We analyzed a federated network of electronic medical records and administrative claims data from 14 data sources and 6 countries containing data collected on or before 4/27/2020. We used a 2-step process to develop 3 scores using historical data from patients with influenza or flu-like symptoms any time prior to 2020. The first step was to create a data-driven model using LASSO regularized logistic regression, the covariates of which were used to develop aggregate covariates for the second step where the COVER scores were developed using a smaller set of features. These 3 COVER scores were then externally validated on patients with 1) influenza or flu-like symptoms and 2) confirmed or suspected COVID-19 diagnosis across 5 databases from South Korea, Spain, and the United States. Outcomes included i) hospitalization with pneumonia, ii) hospitalization with pneumonia requiring intensive services or death, and iii) death in the 30 days after index date. RESULTS: Overall, 44,507 COVID-19 patients were included for model validation. We identified 7 predictors (history of cancer, chronic obstructive pulmonary disease, diabetes, heart disease, hypertension, hyperlipidemia, kidney disease) which combined with age and sex discriminated which patients would experience any of our three outcomes. The models achieved good performance in influenza and COVID-19 cohorts. For COVID-19 the AUC ranges were, COVER-H: 0.69-0.81, COVER-I: 0.73-0.91, and COVER-F: 0.72-0.90. Calibration varied across the validations with some of the COVID-19 validations being less well calibrated than the influenza validations. CONCLUSIONS: This research demonstrated the utility of using a proxy disease to develop a prediction model. The 3 COVER models with 9-predictors that were developed using influenza data perform well for COVID-19 patients for predicting hospitalization, intensive services, and fatality. The scores showed good discriminatory performance which transferred well to the COVID-19 population. There was some miscalibration in the COVID-19 validations, which is potentially due to the difference in symptom severity between the two diseases. A possible solution for this is to recalibrate the models in each location before use.

Cardiac MRI Myocardial Functional and Tissue Characterization Detects Early Cardiac Dysfunction in a Mouse Model of Chemotherapy-Induced Cardiotoxicity.

BACKGROUND: Doxorubicin and doxorubicin-trastuzumab combination chemotherapy have been associated with cardiotoxicity that eventually leads to heart failure and may limit dose-effective cancer treatment. Current diagnostic strategies rely on decreased ejection fraction (EF) to diagnose cardiotoxicity. PURPOSE: The aim of this study is to explore the potential of cardiac MR (CMR) imaging to identify imaging biomarkers in a mouse model of chemotherapy-induced cardiotoxicity. METHODS: A cumulative dose of 25 mg/kg doxorubicin was administered over three weeks using subcutaneous pellets (n = 9, Dox). Another group (n = 9) received same dose of Dox and a total of 10 mg/kg trastuzumab (DT). Mice were imaged at baseline, 5/6 weeks and 10 weeks post-treatment on a 7T MRI system. The protocol included short-axis cine MRI covering the left ventricle (LV) and mid-ventricular short-axis tissue phase mapping (TPM), pre- and post-contrast T1 mapping, T2 mapping and Displacement Encoding with Stimulated Echoes (DENSE) strain encoded MRI. EF, peak myocardial velocities, native T1, T2, extracellular volume (ECV), and myocardial strain were quantified. N = 7 mice were sacrificed for histopathologic assessment of apoptosis at 5/6 weeks. RESULTS: Global peak systolic longitudinal velocity was reduced at 5/6 weeks in Dox (0.6 ± 0.3 vs 0.9 ± 0.3, p = 0.02). In the Dox group, native T1 was reduced at 5/6 weeks (1.3 ± 0.2 ms vs 1.6 ± 0.2 ms, p = 0.02), and relatively normalized at week 10 (1.4 ± 0.1 ms vs 1.6 ± 0.2 ms, p > 0.99). There was no change in EF and other MRI parameters and histopathologic results demonstrated minimal apoptosis in all mice (~1-2 apoptotic cell/high power field), suggesting early-stage cardiotoxicity. CONCLUSIONS: In a mouse model of chemotherapy-induced cardiotoxicity using doxorubicin and trastuzumab, advanced CMR shows promise in identifying treatment-related decrease in myocardial velocity and native T1 prior to the onset of cardiomyocyte apoptosis and reduction of EF.

Does Using a Stacking Ensemble Method to Combine Multiple Base Learners Within a Database Improve Model Transportability?

We investigated a stacking ensemble method that combines multiple base learners within a database. The results on external validation across four large databases suggest a stacking ensemble could improve model transportability.

Correlates of psychological distress among undergraduate women engaged in remote learning through a New York City college during the COVID-19 pandemic.

OBJECTIVE: The study's objective is to explore psychological distress (PD) among remote learners during COVID-19. PARTICIPANTS: Female undergraduates matriculated at an NYC college in Winter 2020. METHODS: Using the Kessler-6 scale, we defined PD as no/low (LPD), mild/moderate (MPD), and severe (SPD) and assessed if residing in/near NYC modified associations. RESULTS: PD was common (MPD: 34.1%, SPD: 38.9%). Students identifying as Other/Multiracial had lower MPD odds (aOR = 0.39 [0.17-0.88]). SPD was associated with identifying as White (aOR = 2.02 [1.02-3.99]), unbalanced meals (aOR = 2.59 [1.06-6.30]), violence experience (aOR = 1.77 [1.06-2.94]), no social support (aOR = 3.24 [1.37-7.64]), and loneliness (aOR = 2.52 [1.29-4.95]). Among students in/near NYC, moderate/high drug use (aOR = 2.76 [1.15-6.61]), no social support (aOR = 3.62 [1.10-1.19]), and loneliness (aOR = 2.92 [1.11-7.63]) were SPD correlates. CONCLUSIONS: PD was high and associated with food insecurity, violence experience, no social support, and loneliness. Living in/near NYC modified drug use, loneliness, and social support associations. Mental health initiatives should address modifiable risk factors to ameliorate pandemic-associated PD.

Building-Level Detection Threshold of SARS-CoV-2 in Wastewater.

We established wastewater surveillance of SARS-CoV-2 in a small, residential, urban college as part of an integrated public health response during the COVID-19 pandemic. Students returned to campus in spring 2021. During the semester, students were required to perform nasal PCR tests twice weekly. At the same time, wastewater monitoring was established in 3 campus dormitory buildings. Two were dedicated dormitories with populations of 188 and 138 students; 1 was an isolation building where students were moved within 2 h of receiving positive test results. Analysis of wastewater from isolation indicated that the amount of viral shedding was highly variable and that viral concentration could not be used to estimate the number of cases at the building level. However, rapid movement of students to isolation enabled determination of predictive power, specificity, and sensitivity from instances in which generally one positive case at a time occurred in a building. Our assay yields effective results with an ~60% positive predictive power, ~90% negative predictive power, and ~90% specificity. Sensitivity, however, is low at ~40%. Detection is improved in the few instances of 2 simultaneous positive cases, with sensitivity of 1 case versus 2 cases increasing from ~20% to 100%. We also measured the appearance of a variant of concern on campus and noted a similarity in timeline with increased prevalence in surrounding New York City. Monitoring SARS-CoV-2 in the sewage outflow of individual buildings can be used with a realistic goal of containing outbreak clusters but not necessarily single cases. IMPORTANCE Diagnostic testing of sewage can detect levels of circulating viruses to help inform public health. Wastewater-based epidemiology has been particularly active during the COVID-19 pandemic to measure the prevalence of SARS-CoV-2. Understanding the technical limitations of diagnostic testing for individual buildings would help inform future surveillance programs. We report our diagnostic and clinical data monitoring of buildings on a college campus in New York City during the spring 2021 semester. Frequent nasal testing, mitigation measures, and public health protocols provided a context in which to study the effectiveness of wastewater-based epidemiology. Our efforts could not consistently detect individual positive COVID-19 cases, but sensitivity is significantly improved in detecting two simultaneous cases. We therefore contend that wastewater surveillance may be more practically suited for the mitigation of outbreak clusters.

Matching theory and evidence on Covid-19 using a stochastic network SIR model.

This paper develops an individual-based stochastic network SIR model for the empirical analysis of the Covid-19 pandemic. It derives moment conditions for the number of infected and active cases for single as well as multigroup epidemic models. These moment conditions are used to investigate the identification and estimation of the transmission rates. The paper then proposes a method that jointly estimates the transmission rate and the magnitude of under-reporting of infected cases. Empirical evidence on six European countries matches the simulated outcomes once the under-reporting of infected cases is addressed. It is estimated that the number of actual cases could be between 4 to 10 times higher than the reported numbers in October 2020 and declined to 2 to 3 times in April 2021. The calibrated models are used in the counterfactual analyses of the impact of social distancing and vaccination on the epidemic evolution and the timing of early interventions in the United Kingdom and Germany.

Use of unstructured text in prognostic clinical prediction models: a systematic review.

OBJECTIVE: This systematic review aims to assess how information from unstructured text is used to develop and validate clinical prognostic prediction models. We summarize the prediction problems and methodological landscape and determine whether using text data in addition to more commonly used structured data improves the prediction performance. MATERIALS AND METHODS: We searched Embase, MEDLINE, Web of Science, and Google Scholar to identify studies that developed prognostic prediction models using information extracted from unstructured text in a data-driven manner, published in the period from January 2005 to March 2021. Data items were extracted, analyzed, and a meta-analysis of the model performance was carried out to assess the added value of text to structured-data models. RESULTS: We identified 126 studies that described 145 clinical prediction problems. Combining text and structured data improved model performance, compared with using only text or only structured data. In these studies, a wide variety of dense and sparse numeric text representations were combined with both deep learning and more traditional machine learning methods. External validation, public availability, and attention for the explainability of the developed models were limited. CONCLUSION: The use of unstructured text in the development of prognostic prediction models has been found beneficial in addition to structured data in most studies. The text data are source of valuable information for prediction model development and should not be neglected. We suggest a future focus on explainability and external validation of the developed models, promoting robust and trustworthy prediction models in clinical practice.

Trends in the conduct and reporting of clinical prediction model development and validation: a systematic review.

OBJECTIVES: This systematic review aims to provide further insights into the conduct and reporting of clinical prediction model development and validation over time. We focus on assessing the reporting of information necessary to enable external validation by other investigators. MATERIALS AND METHODS: We searched Embase, Medline, Web-of-Science, Cochrane Library, and Google Scholar to identify studies that developed 1 or more multivariable prognostic prediction models using electronic health record (EHR) data published in the period 2009-2019. RESULTS: We identified 422 studies that developed a total of 579 clinical prediction models using EHR data. We observed a steep increase over the years in the number of developed models. The percentage of models externally validated in the same paper remained at around 10%. Throughout 2009-2019, for both the target population and the outcome definitions, code lists were provided for less than 20% of the models. For about half of the models that were developed using regression analysis, the final model was not completely presented. DISCUSSION: Overall, we observed limited improvement over time in the conduct and reporting of clinical prediction model development and validation. In particular, the prediction problem definition was often not clearly reported, and the final model was often not completely presented. CONCLUSION: Improvement in the reporting of information necessary to enable external validation by other investigators is still urgently needed to increase clinical adoption of developed models.

Development and external validation of prediction models for adverse health outcomes in rheumatoid arthritis: A multinational real-world cohort analysis.

BACKGROUND: Identification of rheumatoid arthritis (RA) patients at high risk of adverse health outcomes remains a major challenge. We aimed to develop and validate prediction models for a variety of adverse health outcomes in RA patients initiating first-line methotrexate (MTX) monotherapy. METHODS: Data from 15 claims and electronic health record databases across 9 countries were used. Models were developed and internally validated on Optum® De-identified Clinformatics® Data Mart Database using L1-regularized logistic regression to estimate the risk of adverse health outcomes within 3 months (leukopenia, pancytopenia, infection), 2 years (myocardial infarction (MI) and stroke), and 5 years (cancers [colorectal, breast, uterine] after treatment initiation. Candidate predictors included demographic variables and past medical history. Models were externally validated on all other databases. Performance was assessed using the area under the receiver operator characteristic curve (AUC) and calibration plots. FINDINGS: Models were developed and internally validated on 21,547 RA patients and externally validated on 131,928 RA patients. Models for serious infection (AUC: internal 0.74, external ranging from 0.62 to 0.83), MI (AUC: internal 0.76, external ranging from 0.56 to 0.82), and stroke (AUC: internal 0.77, external ranging from 0.63 to 0.95), showed good discrimination and adequate calibration. Models for the other outcomes showed modest internal discrimination (AUC < 0.65) and were not externally validated. INTERPRETATION: We developed and validated prediction models for a variety of adverse health outcomes in RA patients initiating first-line MTX monotherapy. Final models for serious infection, MI, and stroke demonstrated good performance across multiple databases and can be studied for clinical use. FUNDING: This activity under the European Health Data & Evidence Network (EHDEN) has received funding from the Innovative Medicines Initiative 2 Joint Undertaking under grant agreement No 806968. This Joint Undertaking receives support from the European Union's Horizon 2020 research and innovation programme and EFPIA.

A standardized analytics pipeline for reliable and rapid development and validation of prediction models using observational health data.

BACKGROUND AND OBJECTIVE: As a response to the ongoing COVID-19 pandemic, several prediction models in the existing literature were rapidly developed, with the aim of providing evidence-based guidance. However, none of these COVID-19 prediction models have been found to be reliable. Models are commonly assessed to have a risk of bias, often due to insufficient reporting, use of non-representative data, and lack of large-scale external validation. In this paper, we present the Observational Health Data Sciences and Informatics (OHDSI) analytics pipeline for patient-level prediction modeling as a standardized approach for rapid yet reliable development and validation of prediction models. We demonstrate how our analytics pipeline and open-source software tools can be used to answer important prediction questions while limiting potential causes of bias (e.g., by validating phenotypes, specifying the target population, performing large-scale external validation, and publicly providing all analytical source code). METHODS: We show step-by-step how to implement the analytics pipeline for the question: 'In patients hospitalized with COVID-19, what is the risk of death 0 to 30 days after hospitalization?'. We develop models using six different machine learning methods in a USA claims database containing over 20,000 COVID-19 hospitalizations and externally validate the models using data containing over 45,000 COVID-19 hospitalizations from South Korea, Spain, and the USA. RESULTS: Our open-source software tools enabled us to efficiently go end-to-end from problem design to reliable Model Development and evaluation. When predicting death in patients hospitalized with COVID-19, AdaBoost, random forest, gradient boosting machine, and decision tree yielded similar or lower internal and external validation discrimination performance compared to L1-regularized logistic regression, whereas the MLP neural network consistently resulted in lower discrimination. L1-regularized logistic regression models were well calibrated. CONCLUSION: Our results show that following the OHDSI analytics pipeline for patient-level prediction modelling can enable the rapid development towards reliable prediction models. The OHDSI software tools and pipeline are open source and available to researchers from all around the world.

Implementation of the COVID-19 Vulnerability Index Across an International Network of Health Care Data Sets: Collaborative External Validation Study.

BACKGROUND: SARS-CoV-2 is straining health care systems globally. The burden on hospitals during the pandemic could be reduced by implementing prediction models that can discriminate patients who require hospitalization from those who do not. The COVID-19 vulnerability (C-19) index, a model that predicts which patients will be admitted to hospital for treatment of pneumonia or pneumonia proxies, has been developed and proposed as a valuable tool for decision-making during the pandemic. However, the model is at high risk of bias according to the "prediction model risk of bias assessment" criteria, and it has not been externally validated. OBJECTIVE: The aim of this study was to externally validate the C-19 index across a range of health care settings to determine how well it broadly predicts hospitalization due to pneumonia in COVID-19 cases. METHODS: We followed the Observational Health Data Sciences and Informatics (OHDSI) framework for external validation to assess the reliability of the C-19 index. We evaluated the model on two different target populations, 41,381 patients who presented with SARS-CoV-2 at an outpatient or emergency department visit and 9,429,285 patients who presented with influenza or related symptoms during an outpatient or emergency department visit, to predict their risk of hospitalization with pneumonia during the following 0-30 days. In total, we validated the model across a network of 14 databases spanning the United States, Europe, Australia, and Asia. RESULTS: The internal validation performance of the C-19 index had a C statistic of 0.73, and the calibration was not reported by the authors. When we externally validated it by transporting it to SARS-CoV-2 data, the model obtained C statistics of 0.36, 0.53 (0.473-0.584) and 0.56 (0.488-0.636) on Spanish, US, and South Korean data sets, respectively. The calibration was poor, with the model underestimating risk. When validated on 12 data sets containing influenza patients across the OHDSI network, the C statistics ranged between 0.40 and 0.68. CONCLUSIONS: Our results show that the discriminative performance of the C-19 index model is low for influenza cohorts and even worse among patients with COVID-19 in the United States, Spain, and South Korea. These results suggest that C-19 should not be used to aid decision-making during the COVID-19 pandemic. Our findings highlight the importance of performing external validation across a range of settings, especially when a prediction model is being extrapolated to a different population. In the field of prediction, extensive validation is required to create appropriate trust in a model.

Measuring Athletic Mental Energy (AME): Instrument Development and Validation.

Although considerable research indicates that mental energy is an important factor in many domains, including athletic performance (Cook and Davis, 2006), athletic mental energy (AME) has never been conceptualized and measured. Therefore, the aim of this study was to conceptualize and develop a reliable and valid instrument to assess AME. In Study 1, a focus group interview established the initial framework of AME. Study 2 used a survey to collect athletes' experiences of AME and develop a scale draft titled "Athletic Mental Energy Scale (AMES)." In Study 3, we examined the psychometric properties and the underlying structure of AMES via item analysis, internal consistency, and exploratory factor analysis (EFA). In Study 4, we used confirmatory factor analysis (CFA) to examine AMES's factorial validity; and examined concurrent and discriminant validity by examining correlations with athletes' life stress, positive state of mind, and burnout. In study 5, we examined the measurement invariance of the 6-factor, 18-item AMES with Taiwanese and Malaysian samples. Study 6 examined the predictive validity by comparing AMES scores of successful and unsuccessful martial artists. Across these phases, results showed a 6-factor, 18-item AMES had adequate content validity, factorial structure, nomological validity, discriminant validity, predictive validity, measurement invariance, and reliability. We suggest future studies may use AMES to examine its relationships with athletes' cognition, affect, and performance. The application of AMES in sport psychology was also discussed.

Loss of Transcriptional Repression by BCL6 Confers Insulin Sensitivity in the Setting of Obesity.

Accumulation of visceral adiposity is directly linked to the morbidity of obesity, while subcutaneous body fat is considered more benign. We have identified an unexpected role for B cell lymphoma 6 (BCL6), a critical regulator of immunity, in the developmental expansion of subcutaneous adipose tissue. In adipocyte-specific knockout mice (Bcl6AKO), we found that Bcl6 deletion results in strikingly increased inguinal, but not perigonadal, adipocyte size and tissue mass in addition to marked insulin sensitivity. Genome-wide RNA expression and DNA binding analyses revealed that BCL6 controls gene networks involved in cell growth and fatty acid biosynthesis. Using deuterium label incorporation and comprehensive adipokine and lipid profiling, we discovered that ablation of adipocyte Bcl6 enhances subcutaneous adipocyte lipogenesis, increases levels of adiponectin and fatty acid esters of hydroxy fatty acids (FAHFAs), and prevents steatosis. Thus, our studies identify BCL6 as a negative regulator of subcutaneous adipose tissue expansion and metabolic health.

Quantification of the binding potential of cell-surface receptors in fresh excised specimens via dual-probe modeling of SERS nanoparticles.

The complete removal of cancerous tissue is a central aim of surgical oncology, but is difficult to achieve in certain cases, especially when the removal of surrounding normal tissues must be minimized. Therefore, when post-operative pathology identifies residual tumor at the surgical margins, re-excision surgeries are often necessary. An intraoperative approach for tumor-margin assessment, insensitive to nonspecific sources of molecular probe accumulation and contrast, is presented employing kinetic-modeling analysis of dual-probe staining using surface-enhanced Raman scattering nanoparticles (SERS NPs). Human glioma (U251) and epidermoid (A431) tumors were implanted subcutaneously in six athymic mice. Fresh resected tissues were stained with an equimolar mixture of epidermal growth factor receptor (EGFR)-targeted and untargeted SERS NPs. The binding potential (BP; proportional to receptor concentration) of EGFR - a cell-surface receptor associated with cancer - was estimated from kinetic modeling of targeted and untargeted NP concentrations in response to serial rinsing. EGFR BPs in healthy, U251, and A431 tissues were 0.06 ± 0.14, 1.13 ± 0.40, and 2.23 ± 0.86, respectively, which agree with flow-cytometry measurements and published reports. The ability of this approach to quantify the BP of cell-surface biomarkers in fresh tissues opens up an accurate new approach to analyze tumor margins intraoperatively.

Temporal specificity of extinction in autoshaping.

Three experiments investigated the effects of varying the conditioned stimulus (CS) duration between training and extinction. Ring doves (Streptopelia risoria) were autoshaped on a fixed CS-unconditioned stimulus (US) interval and extinguished with CS presentations that were longer, shorter, or the same as the training duration. During a subsequent test session, the training CS duration was reintroduced. Results suggest that the cessation of responding during an extinction session is controlled by generalization of excitation between the training and extinction CSs and by the number of nonreinforced CS presentations. Transfer of extinction to the training CS is controlled by the similarity between the extinction and training CSs. Extinction learning is temporally specific.

The Transcultural Wellness Center: rehabilitation and recovery in Asian and Pacific Islander mental health care.

Asian and Pacific Islander Americans (APIAs) are a diverse group, representing many cultures of origin, a range of immigration experiences, and varying access to economic and other resources. Despite stereotypes such as the "model minority" and cultural values that stigmatize mental illness and complicate mental health help-seeking, APIAs' psychiatric rehabilitation and recovery needs are significant. These needs are inadequately treated within existing systems of care. Passage of California's Mental Health Services Act (MHSA) in 2004 created the opportunity for Sacramento County to fund a full-service mental health clinic designed to meet the needs of the APIA community. The process by which this clinic, the Transcultural Wellness Center, was conceptualized, advocated for, and launched is described. This clinic is considered a best practice model within the MHSA system redesign effort.