Prediction of 30-day, 90-day, and 1-year mortality after colorectal cancer surgery using a data-driven approach.

PURPOSE: To develop prediction models for short-term mortality risk assessment following colorectal cancer surgery. METHODS: Data was harmonized from four Danish observational health databases into the Observational Medical Outcomes Partnership Common Data Model. With a data-driven approach using the Least Absolute Shrinkage and Selection Operator logistic regression on preoperative data, we developed 30-day, 90-day, and 1-year mortality prediction models. We assessed discriminative performance using the area under the receiver operating characteristic and precision-recall curve and calibration using calibration slope, intercept, and calibration-in-the-large. We additionally assessed model performance in subgroups of curative, palliative, elective, and emergency surgery. RESULTS: A total of 57,521 patients were included in the study population, 51.1% male and with a median age of 72 years. The model showed good discrimination with an area under the receiver operating characteristic curve of 0.88, 0.878, and 0.861 for 30-day, 90-day, and 1-year mortality, respectively, and a calibration-in-the-large of 1.01, 0.99, and 0.99. The overall incidence of mortality were 4.48% for 30-day mortality, 6.64% for 90-day mortality, and 12.8% for 1-year mortality, respectively. Subgroup analysis showed no improvement of discrimination or calibration when separating the cohort into cohorts of elective surgery, emergency surgery, curative surgery, and palliative surgery. CONCLUSION: We were able to train prediction models for the risk of short-term mortality on a data set of four combined national health databases with good discrimination and calibration. We found that one cohort including all operated patients resulted in better performing models than cohorts based on several subgroups.

Seek COVER: using a disease proxy to rapidly develop and validate a personalized risk calculator for COVID-19 outcomes in an international network.

BACKGROUND: We investigated whether we could use influenza data to develop prediction models for COVID-19 to increase the speed at which prediction models can reliably be developed and validated early in a pandemic. We developed COVID-19 Estimated Risk (COVER) scores that quantify a patient's risk of hospital admission with pneumonia (COVER-H), hospitalization with pneumonia requiring intensive services or death (COVER-I), or fatality (COVER-F) in the 30-days following COVID-19 diagnosis using historical data from patients with influenza or flu-like symptoms and tested this in COVID-19 patients. METHODS: We analyzed a federated network of electronic medical records and administrative claims data from 14 data sources and 6 countries containing data collected on or before 4/27/2020. We used a 2-step process to develop 3 scores using historical data from patients with influenza or flu-like symptoms any time prior to 2020. The first step was to create a data-driven model using LASSO regularized logistic regression, the covariates of which were used to develop aggregate covariates for the second step where the COVER scores were developed using a smaller set of features. These 3 COVER scores were then externally validated on patients with 1) influenza or flu-like symptoms and 2) confirmed or suspected COVID-19 diagnosis across 5 databases from South Korea, Spain, and the United States. Outcomes included i) hospitalization with pneumonia, ii) hospitalization with pneumonia requiring intensive services or death, and iii) death in the 30 days after index date. RESULTS: Overall, 44,507 COVID-19 patients were included for model validation. We identified 7 predictors (history of cancer, chronic obstructive pulmonary disease, diabetes, heart disease, hypertension, hyperlipidemia, kidney disease) which combined with age and sex discriminated which patients would experience any of our three outcomes. The models achieved good performance in influenza and COVID-19 cohorts. For COVID-19 the AUC ranges were, COVER-H: 0.69-0.81, COVER-I: 0.73-0.91, and COVER-F: 0.72-0.90. Calibration varied across the validations with some of the COVID-19 validations being less well calibrated than the influenza validations. CONCLUSIONS: This research demonstrated the utility of using a proxy disease to develop a prediction model. The 3 COVER models with 9-predictors that were developed using influenza data perform well for COVID-19 patients for predicting hospitalization, intensive services, and fatality. The scores showed good discriminatory performance which transferred well to the COVID-19 population. There was some miscalibration in the COVID-19 validations, which is potentially due to the difference in symptom severity between the two diseases. A possible solution for this is to recalibrate the models in each location before use.

Learning patient-level prediction models across multiple healthcare databases: evaluation of ensembles for increasing model transportability.

BACKGROUND: Prognostic models that are accurate could help aid medical decision making. Large observational databases often contain temporal medical data for large and diverse populations of patients. It may be possible to learn prognostic models using the large observational data. Often the performance of a prognostic model undesirably worsens when transported to a different database (or into a clinical setting). In this study we investigate different ensemble approaches that combine prognostic models independently developed using different databases (a simple federated learning approach) to determine whether ensembles that combine models developed across databases can improve model transportability (perform better in new data than single database models)? METHODS: For a given prediction question we independently trained five single database models each using a different observational healthcare database. We then developed and investigated numerous ensemble models (fusion, stacking and mixture of experts) that combined the different database models. Performance of each model was investigated via discrimination and calibration using a leave one dataset out technique, i.e., hold out one database to use for validation and use the remaining four datasets for model development. The internal validation of a model developed using the hold out database was calculated and presented as the 'internal benchmark' for comparison. RESULTS: In this study the fusion ensembles generally outperformed the single database models when transported to a previously unseen database and the performances were more consistent across unseen databases. Stacking ensembles performed poorly in terms of discrimination when the labels in the unseen database were limited. Calibration was consistently poor when both ensembles and single database models were applied to previously unseen databases. CONCLUSION: A simple federated learning approach that implements ensemble techniques to combine models independently developed across different databases for the same prediction question may improve the discriminative performance in new data (new database or clinical setting) but will need to be recalibrated using the new data. This could help medical decision making by improving prognostic model performance.

90-Day all-cause mortality can be predicted following a total knee replacement: an international, network study to develop and validate a prediction model.

PURPOSE: The purpose of this study was to develop and validate a prediction model for 90-day mortality following a total knee replacement (TKR). TKR is a safe and cost-effective surgical procedure for treating severe knee osteoarthritis (OA). Although complications following surgery are rare, prediction tools could help identify high-risk patients who could be targeted with preventative interventions. The aim was to develop and validate a simple model to help inform treatment choices. METHODS: A mortality prediction model for knee OA patients following TKR was developed and externally validated using a US claims database and a UK general practice database. The target population consisted of patients undergoing a primary TKR for knee OA, aged ≥ 40 years and registered for ≥ 1 year before surgery. LASSO logistic regression models were developed for post-operative (90-day) mortality. A second mortality model was developed with a reduced feature set to increase interpretability and usability. RESULTS: A total of 193,615 patients were included, with 40,950 in The Health Improvement Network (THIN) database and 152,665 in Optum. The full model predicting 90-day mortality yielded AUROC of 0.78 when trained in OPTUM and 0.70 when externally validated on THIN. The 12 variable model achieved internal AUROC of 0.77 and external AUROC of 0.71 in THIN. CONCLUSIONS: A simple prediction model based on sex, age, and 10 comorbidities that can identify patients at high risk of short-term mortality following TKR was developed that demonstrated good, robust performance. The 12-feature mortality model is easily implemented and the performance suggests it could be used to inform evidence based shared decision-making prior to surgery and targeting prophylaxis for those at high risk. LEVEL OF EVIDENCE: III.

Feasibility and evaluation of a large-scale external validation approach for patient-level prediction in an international data network: validation of models predicting stroke in female patients newly diagnosed with atrial fibrillation.

BACKGROUND: To demonstrate how the Observational Healthcare Data Science and Informatics (OHDSI) collaborative network and standardization can be utilized to scale-up external validation of patient-level prediction models by enabling validation across a large number of heterogeneous observational healthcare datasets. METHODS: Five previously published prognostic models (ATRIA, CHADS2, CHADS2VASC, Q-Stroke and Framingham) that predict future risk of stroke in patients with atrial fibrillation were replicated using the OHDSI frameworks. A network study was run that enabled the five models to be externally validated across nine observational healthcare datasets spanning three countries and five independent sites. RESULTS: The five existing models were able to be integrated into the OHDSI framework for patient-level prediction and they obtained mean c-statistics ranging between 0.57-0.63 across the 6 databases with sufficient data to predict stroke within 1 year of initial atrial fibrillation diagnosis for females with atrial fibrillation. This was comparable with existing validation studies. The validation network study was run across nine datasets within 60 days once the models were replicated. An R package for the study was published at https://github.com/OHDSI/StudyProtocolSandbox/tree/master/ExistingStrokeRiskExternalValidation. CONCLUSION: This study demonstrates the ability to scale up external validation of patient-level prediction models using a collaboration of researchers and a data standardization that enable models to be readily shared across data sites. External validation is necessary to understand the transportability or reproducibility of a prediction model, but without collaborative approaches it can take three or more years for a model to be validated by one independent researcher. In this paper we show it is possible to both scale-up and speed-up external validation by showing how validation can be done across multiple databases in less than 2 months. We recommend that researchers developing new prediction models use the OHDSI network to externally validate their models.

Intercomparison of the Radio-Chronometric Ages of Plutonium-Certified Reference Materials with Distinct Isotopic Compositions.

An intercomparison of the radio-chronometric ages of four distinct plutonium-certified reference materials varying in chemical form, isotopic composition, and period of production are presented. The cross-comparison of the different 234U/238Pu, 235U/239Pu, 236U/240Pu, and 241Am/241Pu model purification ages obtained at four independent analytical facilities covering a range of laboratory environments from bulk sample processing to clean facilities dedicated to nuclear forensic investigation of environmental samples enables a true assessment of the state-of-practice in "age dating capabilities" for nuclear materials. The analytical techniques evaluated used modern mass spectrometer instrumentation including thermal ionization mass spectrometers and inductively coupled plasma mass spectrometers for isotopic abundance measurements. Both multicollector and single collector instruments were utilized to generate the data presented here. Consensus values established in this study make it possible to use these isotopic standards as quality control standards for radio-chronometry applications. Results highlight the need for plutonium isotopic standards that are certified for 234U/238Pu, 235U/239Pu, 236U/240Pu, and 241Am/241Pu model purification ages as well as other multigenerational radio-chronometers such as 237Np/241Pu. Due to the capabilities of modern analytical instrumentation, analytical laboratories that focus on trace level analyses can obtain model ages with marginally larger uncertainties than laboratories that handle bulk samples. When isotope ratio measurement techniques like thermal ionization mass spectrometry and inductively coupled plasma mass spectrometry with comparable precision are utilized, model purification ages with similar uncertainties are obtained.

The prognostic and predictive value of vascular response parameters measured by dynamic contrast-enhanced-CT, -MRI and -US in patients with metastatic renal cell carcinoma receiving sunitinib.

OBJECTIVES: To identify dynamic contrast-enhanced (DCE) imaging parameters from MRI, CT and US that are prognostic and predictive in patients with metastatic renal cell cancer (mRCC) receiving sunitinib. METHODS: Thirty-four patients were monitored by DCE imaging on day 0 and 14 of the first course of sunitinib treatment. Additional scans were performed with DCE-US only (day 7 or 28 and 2 weeks after the treatment break). Perfusion parameters that demonstrated a significant correlation (Spearman p < 0.05) with progression-free survival (PFS) and overall survival (OS) were investigated using Cox proportional hazard models/ratios (HR) and Kaplan-Meier survival analysis. RESULTS: A higher baseline and day 14 value for Ktrans (DCE-MRI) and a lower pre-treatment vascular heterogeneity (DCE-US) were significantly associated with a longer PFS (HR, 0.62, 0.37 and 5.5, respectively). A larger per cent decrease in blood volume on day 14 (DCE-US) predicted a longer OS (HR, 1.45). We did not find significant correlations between any of the DCE-CT parameters and PFS/OS, unless a cut-off analysis was used. CONCLUSIONS: DCE-MRI, -CT and ultrasound produce complementary parameters that reflect the prognosis of patients receiving sunitinib for mRCC. Blood volume measured by DCE-US was the only parameter whose change during early anti-angiogenic therapy predicted for OS and PFS. KEY POINTS: • DCE-CT, -MRI and ultrasound are complementary modalities for monitoring anti-angiogenic therapy. • The change in blood volume measured by DCE-US was predictive of OS/PFS. • Baseline vascular heterogeneity by DCE-US has the strongest prognostic value for PFS.

Plutonium(IV) and (V) Sorption to Goethite at Sub-Femtomolar to Micromolar Concentrations: Redox Transformations and Surface Precipitation.

Pu(IV) and Pu(V) sorption to goethite was investigated over a concentration range of 10(-15)-10(-5) M at pH 8. Experiments with initial Pu concentrations of 10(-15) - 10(-8) M produced linear Pu sorption isotherms, demonstrating that Pu sorption to goethite is not concentration-dependent across this concentration range. Equivalent Pu(IV) and Pu(V) sorption Kd values obtained at 1 and 2-week sampling time points indicated that Pu(V) is rapidly reduced to Pu(IV) on the goethite surface. Further, it suggested that Pu surface redox transformations are sufficiently rapid to achieve an equilibrium state within 1 week, regardless of the initial Pu oxidation state. At initial concentrations >10(-8) M, both Pu oxidation states exhibited deviations from linear sorption behavior and less Pu was adsorbed than at lower concentrations. NanoSIMS and HRTEM analysis of samples with initial Pu concentrations of 10(-8) - 10(-6) M indicated that Pu surface and/or bulk precipitation was likely responsible for this deviation. In 10(-6) M Pu(IV) and Pu(V) samples, HRTEM analysis showed the formation of a body centered cubic (bcc) Pu4O7 structure on the goethite surface, confirming that reduction of Pu(V) had occurred on the mineral surface and that epitaxial distortion previously observed for Pu(IV) sorption occurs with Pu(V) as well.

Observation of a superfluid Hall effect.

Measurement techniques based upon the Hall effect are invaluable tools in condensed-matter physics. When an electric current flows perpendicular to a magnetic field, a Hall voltage develops in the direction transverse to both the current and the field. In semiconductors, this behavior is routinely used to measure the density and charge of the current carriers (electrons in conduction bands or holes in valence bands)--internal properties of the system that are not accessible from measurements of the conventional resistance. For strongly interacting electron systems, whose behavior can be very different from the free electron gas, the Hall effect's sensitivity to internal properties makes it a powerful tool; indeed, the quantum Hall effects are named after the tool by which they are most distinctly measured instead of the physics from which the phenomena originate. Here we report the first observation of a Hall effect in an ultracold gas of neutral atoms, revealed by measuring a Bose-Einstein condensate's transport properties perpendicular to a synthetic magnetic field. Our observations in this vortex-free superfluid are in good agreement with hydrodynamic predictions, demonstrating that the system's global irrotationality influences this superfluid Hall signal.

Isotope effect of mercury diffusion in air.

Identifying and reducing impacts from mercury sources in the environment remains a considerable challenge and requires process based models to quantify mercury stocks and flows. The stable isotope composition of mercury in environmental samples can help address this challenge by serving as a tracer of specific sources and processes. Mercury isotope variations are small and result only from isotope fractionation during transport, equilibrium, and transformation processes. Because these processes occur in both industrial and environmental settings, knowledge of their associated isotope effects is required to interpret mercury isotope data. To improve the mechanistic modeling of mercury isotope effects during gas phase diffusion, an experimental program tested the applicability of kinetic gas theory. Gas-phase elemental mercury diffusion through small bore needles from finite sources demonstrated mass dependent diffusivities leading to isotope fractionation described by a Rayleigh distillation model. The measured relative atomic diffusivities among mercury isotopes in air are large and in agreement with kinetic gas theory. Mercury diffusion in air offers a reasonable explanation of recent field results reported in the literature.

Comparing penalization methods for linear models on large observational health data.

OBJECTIVE: This study evaluates regularization variants in logistic regression (L1, L2, ElasticNet, Adaptive L1, Adaptive ElasticNet, Broken adaptive ridge [BAR], and Iterative hard thresholding [IHT]) for discrimination and calibration performance, focusing on both internal and external validation. MATERIALS AND METHODS: We use data from 5 US claims and electronic health record databases and develop models for various outcomes in a major depressive disorder patient population. We externally validate all models in the other databases. We use a train-test split of 75%/25% and evaluate performance with discrimination and calibration. Statistical analysis for difference in performance uses Friedman's test and critical difference diagrams. RESULTS: Of the 840 models we develop, L1 and ElasticNet emerge as superior in both internal and external discrimination, with a notable AUC difference. BAR and IHT show the best internal calibration, without a clear external calibration leader. ElasticNet typically has larger model sizes than L1. Methods like IHT and BAR, while slightly less discriminative, significantly reduce model complexity. CONCLUSION: L1 and ElasticNet offer the best discriminative performance in logistic regression for healthcare predictions, maintaining robustness across validations. For simpler, more interpretable models, L0-based methods (IHT and BAR) are advantageous, providing greater parsimony and calibration with fewer features. This study aids in selecting suitable regularization techniques for healthcare prediction models, balancing performance, complexity, and interpretability.

Health-Analytics Data to Evidence Suite (HADES): Open-Source Software for Observational Research.

The Health-Analytics Data to Evidence Suite (HADES) is an open-source software collection developed by Observational Health Data Sciences and Informatics (OHDSI). It executes directly against healthcare data such as electronic health records and administrative claims, that have been converted to the Observational Medical Outcomes Partnership (OMOP) Common Data Model. Using advanced analytics, HADES performs characterization, population-level causal effect estimation, and patient-level prediction, potentially across a federated data network, allowing patient-level data to remain locally while only aggregated statistics are shared. Designed to run across a wide array of technical environments, including different operating systems and database platforms, HADES uses continuous integration with a large set of unit tests to maintain reliability. HADES implements OHDSI best practices, and is used in almost all published OHDSI studies, including some that have directly informed regulatory decisions.

Quantifying Global Origin-Diagnostic Features and Patterns in Biotic and Abiotic Acyclic Lipids for Life Detection.

Lipids are a geologically robust class of organics ubiquitous to life as we know it. Lipid-like soluble organics are synthesized abiotically and have been identified in carbonaceous meteorites and on Mars. Ascertaining the origin of lipids on Mars would be a profound astrobiological achievement. We enumerate origin-diagnostic features and patterns in two acyclic lipid classes, fatty acids (i.e., carboxylic acids) and acyclic hydrocarbons, by collecting and analyzing molecular data reported in over 1500 samples from previously published studies of terrestrial and meteoritic organics. We identify 27 combined (15 for fatty acids, 12 for acyclic hydrocarbons) molecular patterns and structural features that can aid in distinguishing biotic from abiotic synthesis. Principal component analysis (PCA) demonstrates that multivariate analyses of molecular features (16 for fatty acids, 14 for acyclic hydrocarbons) can potentially indicate sample origin. Terrestrial lipids are dominated by longer straight-chain molecules (C4-C34 fatty acids, C14-C46 acyclic hydrocarbons), with predominance for specific branched and unsaturated isomers. Lipid-like meteoritic soluble organics are shorter, with random configurations. Organic solvent-extraction techniques are most commonly reported, motivating the design of our novel instrument, the Extractor for Chemical Analysis of Lipid Biomarkers in Regolith (ExCALiBR), which extracts lipids while preserving origin-diagnostic features that can indicate biogenicity.

The DELPHI Library: Improving Model Validation, Transparency and Dissemination Through a Centralised Library of Prediction Models.

The Deposit, Evaluate and Lookup Predictive Healthcare Information (DELPHI) library provides a centralised location for the depositing, exploring and analysing of patient-level prediction models that are compatible with data mapped to the observational medical outcomes partnership common data model.

The role of estrogen in female skeletal muscle aging: A systematic review.

Aging is associated with a loss of skeletal muscle mass and function that negatively impacts the independence and quality of life of older individuals. Females demonstrate a distinct pattern of muscle aging compared to males, potentially due to menopause, when the production of endogenous sex hormones declines. This systematic review aims to investigate the current knowledge about the role of estrogen in female skeletal muscle aging. A systematic search of MEDLINE Complete, Global Health, Embase, PubMed, SPORTDiscus, and CINHAL was conducted. Studies were considered eligible if they compared a state of estrogen deficiency (e.g. postmenopausal females) or supplementation (e.g. estrogen therapy) to normal estrogen conditions (e.g. premenopausal females or no supplementation). Outcome variables of interest included measures of skeletal muscle mass, function, damage/repair, and energy metabolism. Quality assessment was completed with the relevant Johanna Briggs critical appraisal tool, and data were synthesized in a narrative manner. Thirty-two studies were included in the review. Compared to premenopausal women, postmenopausal women had reduced muscle mass and strength, but the effect of menopause on markers of muscle damage and expression of the genes involved in metabolic signaling pathways remains unclear. Some studies suggest a beneficial effect of estrogen therapy on muscle size and strength, but evidence is largely conflicting and inconclusive, potentially due to large variations in the reporting and status of exposure and outcomes. The findings from this review point toward a potential negative effect of estrogen deficiency on aging skeletal muscle, but further mechanistic evidence is needed to clarify its role.

Does Using a Stacking Ensemble Method to Combine Multiple Base Learners Within a Database Improve Model Transportability?

We investigated a stacking ensemble method that combines multiple base learners within a database. The results on external validation across four large databases suggest a stacking ensemble could improve model transportability.

What determines the success of states in reducing alcohol related crash fatalities? A longitudinal analysis of alcohol related crashes in the U.S. from 1985 to 2019.

In the United States, nearly 28 people die in alcohol-related motor vehicle crashes every day (1 fatality every 52 min). Over decades, states have enacted multiple laws to reduce such fatalities. From 1982 to 2019, the proportion of drivers in fatal crashes with a blood alcohol concentration (BAC) above 0.01 g/dl declined from 41% to 22%. States vary in terms of their success in reducing alcohol-related crash fatalities. The purpose of this study was to examine factors associated with changes in fatalities related to alcohol-impaired driving at the state level. We created a panel dataset of 50 states from 1985 to 2019 by merging different data sources and used fixed-effect linear regression models to analyze the data. Our two outcome variables were the ratio of drivers in fatal crashes with BAC ≥ 0.01 g/dl to those with BAC = 0.00, and the ratio of those with BAC ≥ 0.08 g/dl to those with BAC < 0.08 g/dl. Our independent variables included four laws (0.08 g/dl BAC per se law, administrative license revocation law, minimum legal drinking age law, and zero tolerance law), number of arrests due to impaired driving, alcohol consumption per capita, unemployment rate, and vehicle miles traveled. We found that the 0.08 g/dl per se law was significantly associated with lower alcohol-related crash fatalities while alcohol consumption per capita was significantly and positively associated with crash-related fatalities. Arrests due to driving under the influence (DUI) and crash fatalities were nonlinearly correlated. In addition, interaction of DUI arrests and two laws (0.08 g/dl BAC per se law, and zero tolerance) were significantly associated with lower crash-related fatalities. Our findings suggest that states which have more restrictive laws and enforce them are more likely to significantly reduce alcohol-related crash fatalities.

Using Iterative Pairwise External Validation to Contextualize Prediction Model Performance: A Use Case Predicting 1-Year Heart Failure Risk in Patients with Diabetes Across Five Data Sources.

INTRODUCTION: External validation of prediction models is increasingly being seen as a minimum requirement for acceptance in clinical practice. However, the lack of interoperability of healthcare databases has been the biggest barrier to this occurring on a large scale. Recent improvements in database interoperability enable a standardized analytical framework for model development and external validation. External validation of a model in a new database lacks context, whereby the external validation can be compared with a benchmark in this database. Iterative pairwise external validation (IPEV) is a framework that uses a rotating model development and validation approach to contextualize the assessment of performance across a network of databases. As a use case, we predicted 1-year risk of heart failure in patients with type 2 diabetes mellitus. METHODS: The method follows a two-step process involving (1) development of baseline and data-driven models in each database according to best practices and (2) validation of these models across the remaining databases. We introduce a heatmap visualization that supports the assessment of the internal and external model performance in all available databases. As a use case, we developed and validated models to predict 1-year risk of heart failure in patients initializing a second pharmacological intervention for type 2 diabetes mellitus. We leveraged the power of the Observational Medical Outcomes Partnership common data model to create an open-source software package to increase the consistency, speed, and transparency of this process. RESULTS: A total of 403,187 patients from five databases were included in the study. We developed five models that, when assessed internally, had a discriminative performance ranging from 0.73 to 0.81 area under the receiver operating characteristic curve with acceptable calibration. When we externally validated these models in a new database, three models achieved consistent performance and in context often performed similarly to models developed in the database itself. The visualization of IPEV provided valuable insights. From this, we identified the model developed in the Commercial Claims and Encounters (CCAE) database as the best performing model overall. CONCLUSION: Using IPEV lends weight to the model development process. The rotation of development through multiple databases provides context to model assessment, leading to improved understanding of transportability and generalizability. The inclusion of a baseline model in all modelling steps provides further context to the performance gains of increasing model complexity. The CCAE model was identified as a candidate for clinical use. The use case demonstrates that IPEV provides a huge opportunity in a new era of standardised data and analytics to improve insight into and trust in prediction models at an unprecedented scale.

Use of unstructured text in prognostic clinical prediction models: a systematic review.

OBJECTIVE: This systematic review aims to assess how information from unstructured text is used to develop and validate clinical prognostic prediction models. We summarize the prediction problems and methodological landscape and determine whether using text data in addition to more commonly used structured data improves the prediction performance. MATERIALS AND METHODS: We searched Embase, MEDLINE, Web of Science, and Google Scholar to identify studies that developed prognostic prediction models using information extracted from unstructured text in a data-driven manner, published in the period from January 2005 to March 2021. Data items were extracted, analyzed, and a meta-analysis of the model performance was carried out to assess the added value of text to structured-data models. RESULTS: We identified 126 studies that described 145 clinical prediction problems. Combining text and structured data improved model performance, compared with using only text or only structured data. In these studies, a wide variety of dense and sparse numeric text representations were combined with both deep learning and more traditional machine learning methods. External validation, public availability, and attention for the explainability of the developed models were limited. CONCLUSION: The use of unstructured text in the development of prognostic prediction models has been found beneficial in addition to structured data in most studies. The text data are source of valuable information for prediction model development and should not be neglected. We suggest a future focus on explainability and external validation of the developed models, promoting robust and trustworthy prediction models in clinical practice.