Linked shrinkage to improve estimation of interaction effects in regression models.

Objectives: The addition of two-way interactions is a classic problem in statistics, and comes with the challenge of quadratically increasing dimension. We aim to a) devise an estimation method that can handle this challenge and b) to aid interpretation of the resulting model by developing computational tools for quantifying variable importance. Methods: Existing strategies typically overcome the dimensionality problem by only allowing interactions between relevant main effects. Building on this philosophy, and aiming for settings with moderate n to p ratio, we develop a local shrinkage model that links the shrinkage of interaction effects to the shrinkage of their corresponding main effects. In addition, we derive a new analytical formula for the Shapley value, which allows rapid assessment of individual-specific variable importance scores and their uncertainties. Results: We empirically demonstrate that our approach provides accurate estimates of the model parameters and very competitive predictive accuracy. In our Bayesian framework, estimation inherently comes with inference, which facilitates variable selection. Comparisons with key competitors are provided. Large-scale cohort data are used to provide realistic illustrations and evaluations. The implementation of our method in RStan is relatively straightforward and flexible, allowing for adaptation to specific needs. Conclusions: Our method is an attractive alternative for existing strategies to handle interactions in epidemiological and/or clinical studies, as its linked local shrinkage can improve parameter accuracy, prediction and variable selection. Moreover, it provides appropriate inference and interpretation, and may compete well with less interpretable machine learners in terms of prediction.

Predicting Cognitive Decline in Amyloid-Positive Patients With Mild Cognitive Impairment or Mild Dementia.

BACKGROUND AND OBJECTIVES: Cognitive decline rates in Alzheimer disease (AD) vary greatly. Disease-modifying treatments may alter cognitive decline trajectories, rendering their prediction increasingly relevant. We aimed to construct clinically applicable prediction models of cognitive decline in amyloid-positive patients with mild cognitive impairment (MCI) or mild dementia. METHODS: From the Amsterdam Dementia Cohort, we selected amyloid-positive participants with MCI or mild dementia and at least 2 longitudinal Mini-Mental State Examination (MMSE) measurements. Amyloid positivity was based on CSF AD biomarker concentrations or amyloid PET. We used linear mixed modeling to predict MMSE over time, describing trajectories using a cubic time curve and interactions between linear time and the baseline predictors age, sex, baseline MMSE, APOE ε4 dose, CSF ß-amyloid (Aß) 1-42 and pTau, and MRI total brain and hippocampal volume. Backward selection was used to reduce model complexity. These models can predict MMSE over follow-up or the time to an MMSE value. MCI and mild dementia were modeled separately. Internal 5-fold cross-validation was performed to calculate the explained variance (R2). RESULTS: In total, 961 participants were included (age 65 ± 7 years, 49% female), 310 had MCI (MMSE 26 ± 2) and 651 had mild dementia (MMSE 22 ± 4), with 4 ± 2 measurements over 2 (interquartile range 1-4) years. Cognitive decline rates increased over time for both MCI and mild dementia (model comparisons linear vs squared vs cubic time fit; p < 0.05 favoring a cubic fit). For MCI, backward selection retained age, sex, and CSF Aß1-42 and pTau concentrations as time-varying effects altering the MMSE trajectory. For mild dementia, retained time-varying effects were Aß1-42, age, APOE ε4, and baseline MMSE. R2 was 0.15 for the MCI model and 0.26 for mild dementia in internal cross-validation. A hypothetical patient with MCI, baseline MMSE 28, and CSF Aß1-42 of 925 pg/mL was predicted to reach an MMSE of 20 after 6.0 years (95% CI 5.4-6.7) and after 8.6 years with a hypothetical treatment reducing decline by 30%. DISCUSSION: We constructed models for MCI and mild dementia that predict MMSE over time. These models could inform patients about their potential cognitive trajectory and the remaining uncertainty and aid in conversations about individualized potential treatment effects.

Optimising personal continuity for older patients in general practice: a cluster randomised stepped wedge pragmatic trial.

AIM: To evaluate the effectiveness, feasibility and acceptability of a multicomponent intervention for improving personal continuity for older patients in general practice. DESIGN: A cluster randomised three-wedged, pragmatic trial during 18 months. SETTING: 32 general practices in the Netherlands. PARTICIPANTS: 221 general practitioners (GPs), practice assistants and other practice staff were included. Practices were instructed to include a random sample of 1050 patients aged 65 or older at baseline and 12-month follow-up. INTERVENTION: The intervention took place at practice level and included opTimise persOnal cOntinuity for oLder (TOOL)-kit: a toolbox containing 34 strategies to improve personal continuity. OUTCOMES: Data were collected at baseline and at six 3-monthly follow-up measurements. Primary outcome measure was experienced continuity of care at the patient level measured by the Nijmegen Continuity Questionnaire (NCQ) with subscales for personal continuity (GP knows me and GP shows commitment) and team/cross-boundary continuity at 12-month follow-up. Secondary outcomes were measured in GPs, practice assistants and other practice staff and included work stress and satisfaction and perceived level of personal continuity. In addition, a process evaluation was undertaken among GPs, practice assistants and other practice staff to assess the acceptability and feasibility of the intervention. RESULTS: No significant effect of the intervention was observed on NCQ subscales GP knows me (adjusted mean difference: 0.05 (95% CI -0.05 to 0.15), p=0.383), GP shows commitment (0.03 (95% CI -0.08 to 0.14), p=0.668) and team/cross-boundary (0.01 (95% CI -0.06 to 0.08), p=0.911). All secondary outcomes did not change significantly during follow-up. Process evaluation among GPs, practice assistants and other practice staff showed adequate acceptability of the intervention and partial implementation due to the COVID-19 pandemic and a high perceived workload. CONCLUSION: Although participants viewed TOOL-kit as a practical and accessible toolbox, it did not improve personal continuity as measured with the NCQ. The absence of an effect may be explained by the incomplete implementation of TOOL-kit into practice and the choice of general outcome measures instead of outcomes more specific for the intervention. TRIAL REGISTRATION NUMBER: International Clinical Trials registry Platform (ICTRP), trial NL8132 (URL: ICTRP Search Portal (who.int).

Performance on clinical outcomes, activities of daily living and user experience on head-mounted displays for people with vision impairment.

PURPOSE: To compare the objective performance, acceptance and usability of head-mounted displays (HMDs) to provide evidence-based data that could be used to increase the efficiency of device referrals based upon a person's vision loss and functional needs. METHODS: A cross-sectional, counterbalanced, individually controlled crossover study was performed on 15 adults with various eye conditions. Performance was measured when using four HMDs: eSight4, Eyedaptic EYE3, Eyedaptic EYE4 and IrisVision Inspire. Performance on clinical visual acuity tests and contrast were assessed, as well as vision-related activities of daily living (ADL) which were divided into three categories: Reading, Searching & Identifying and Eye-hand Coordination. User-experience was also assessed. Logistic regression analyses, Friedman one-way repeated measure analyses of variance by ranks and multivariate permutation testing were used for analysis. RESULTS: There was a significant improvement in visual acuity when using all devices. For contrast tasks, only the eSight4 and Eyedaptic EYE3 improved performance relative to baseline. For most Reading and Searching & Identifying tasks, the odds of being able to perform the tasks were significantly higher while using the devices. However, the actual performance with most devices (e.g., number of words read or reading speed) did not improve significantly over baseline for most tasks. For the Eye-hand Coordination tasks, participants performed equivalent to or significantly poorer than baseline when using the devices. No demographic or clinical predictors of outcomes were identified. Participants expressed dissatisfaction with the devices' effectiveness, acceptability and usability. CONCLUSIONS: While performance on clinical tests was better when using the devices, performance on most real-world ADLs was equal to or worse than baseline. No single device improved performance on all tasks, and performance on any one task was not improved with all the devices. The overall dissatisfaction with the devices paralleled the lack of objective improvement in the performance of real-world tasks.

Long-Term Follow-Up of the LEAP Study: Early Versus Delayed Levodopa in Early Parkinson's Disease.

BACKGROUND AND OBJECTIVE: The Levodopa in EArly Parkinson's disease study showed no effect of earlier versus later levodopa initiation on Parkinson's disease (PD) progression over 80 weeks. We now report the effects over 5 years. METHODS: The Levodopa in EArly Parkinson's disease study randomly assigned patients to levodopa/carbidopa 300/75 mg daily for 80 weeks (early start) or to placebo for 40 weeks followed by levodopa/carbidopa 300/75 mg daily for 40 weeks (delayed start). Follow-up visits were performed 3 and 5 years after baseline. We assessed the between-group differences in terms of square root transformed total Unified Parkinson's Disease Rating Scale score at 3 and 5 years with linear regression. We compared the prevalence of dyskinesia, prevalence of wearing off, and the levodopa equivalent daily dose. RESULTS: A total of 321 patients completed the 5-year visit. The adjusted square root transformed total Unified Parkinson's Disease Rating Scale did not differ between treatment groups at 3 (estimated difference, 0.17; standard error, 0.13; P = 0.18) and 5 years (estimated difference, 0.24; standard error, 0.13; P = 0.07). At 5 years, 46 of 160 patients in the early-start group and 62 of 161 patients in the delayed-start group experienced dyskinesia (P = 0.06). The prevalence of wearing off and the levodopa equivalent daily dose were not significantly different between groups. CONCLUSIONS: We did not find a difference in disease progression or in prevalence of motor complications between patients with early PD starting treatment with a low dose of levodopa 40 weeks earlier versus 40 weeks later over the subsequent 5 years. © 2024 The Authors. Movement Disorders published by Wiley Periodicals LLC on behalf of International Parkinson and Movement Disorder Society.

Feasibility and potential effectiveness of the IdentifEYE training programmes to address mental health problems in adults with vision impairment.

PURPOSE: Two training programmes about depression and anxiety in adults with vision impairment were developed to support eye care practitioners (ECPs) and low vision service (LVS) workers in identifying and discussing mental health problems. The purpose of this study was to evaluate the training programmes' potential effectiveness and feasibility. METHODS: The training programmes were offered to ECPs (n = 9) and LVS workers (n = 17). All participants completed surveys pre-, mid- and post-training, and at a 4 week follow-up about the training programmes' content, effectiveness, feasibility and implementation. The Kirkpatrick model was used as a theoretical framework; linear mixed models were used to determine the potential effectiveness, and outcomes were explored during three focus group meetings. RESULTS: Expectations were met in the majority of the participants (84.6%). Post-training, both ECPs and LVS workers reported increased confidence (ß = 3.67, confidence interval (CI): 0.53-6.80; ß = 4.35, CI: 1.57 to 7.14, respectively) and less barriers (ß = -3.67, CI: -6.45 to -0.89; ß = -1.82, CI: -4.56 to 0.91). Mental health problems were more likely addressed in both the groups (ECP ß = 2.22, CI: -0.17 to 4.62; LVS ß = 4.18, CI: 2.67 to 5.68), but these effects did not last in ECPs (ß = -3.22, CI: -7.37 to 0.92). Variations of these learning effects between individual participants were found within both the groups, and LVS workers indicated a need to focus on their own profession. Participants provided information on how to improve the training programmes' feasibility, effectiveness and implementation. CONCLUSION: The training programmes seemed feasible and potentially effective. Transfer of the lessons learned into daily practice could be enhanced by, for example, specifying the training programmes for healthcare providers with the same profession, introducing microlearning and incorporating mental health management into organisation policies.

Prognosis and prediction of antibiotic benefit in adults with clinically diagnosed acute rhinosinusitis: an individual participant data meta-analysis.

BACKGROUND: A previous individual participant data meta-analysis (IPD-MA) of antibiotics for adults with clinically diagnosed acute rhinosinusitis (ARS) showed a marginal overall effect of antibiotics, but was unable to identify patients that are most likely to benefit from antibiotics when applying conventional (i.e. univariable or one-variable-at-a-time) subgroup analysis. We updated the systematic review and investigated whether multivariable prediction of patient-level prognosis and antibiotic treatment effect may lead to more tailored treatment assignment in adults presenting to primary care with ARS. METHODS: An IPD-MA of nine double-blind placebo-controlled trials of antibiotic treatment (n=2539) was conducted, with the probability of being cured at 8-15 days as the primary outcome. A logistic mixed effects model was developed to predict the probability of being cured based on demographic characteristics, signs and symptoms, and antibiotic treatment assignment. Predictive performance was quantified based on internal-external cross-validation in terms of calibration and discrimination performance, overall model fit, and the accuracy of individual predictions. RESULTS: Results indicate that the prognosis with respect to risk of cure could not be reliably predicted (c-statistic 0.58 and Brier score 0.24). Similarly, patient-level treatment effect predictions did not reliably distinguish between those that did and did not benefit from antibiotics (c-for-benefit 0.50). CONCLUSIONS: In conclusion, multivariable prediction based on patient demographics and common signs and symptoms did not reliably predict the patient-level probability of cure and antibiotic effect in this IPD-MA. Therefore, these characteristics cannot be expected to reliably distinguish those that do and do not benefit from antibiotics in adults presenting to primary care with ARS.

Propensity-based standardization to enhance the validation and interpretation of prediction model discrimination for a target population.

External validation of the discriminative ability of prediction models is of key importance. However, the interpretation of such evaluations is challenging, as the ability to discriminate depends on both the sample characteristics (ie, case-mix) and the generalizability of predictor coefficients, but most discrimination indices do not provide any insight into their respective contributions. To disentangle differences in discriminative ability across external validation samples due to a lack of model generalizability from differences in sample characteristics, we propose propensity-weighted measures of discrimination. These weighted metrics, which are derived from propensity scores for sample membership, are standardized for case-mix differences between the model development and validation samples, allowing for a fair comparison of discriminative ability in terms of model characteristics in a target population of interest. We illustrate our methods with the validation of eight prediction models for deep vein thrombosis in 12 external validation data sets and assess our methods in a simulation study. In the illustrative example, propensity score standardization reduced between-study heterogeneity of discrimination, indicating that between-study variability was partially attributable to case-mix. The simulation study showed that only flexible propensity-score methods (allowing for non-linear effects) produced unbiased estimates of model discrimination in the target population, and only when the positivity assumption was met. Propensity score-based standardization may facilitate the interpretation of (heterogeneity in) discriminative ability of a prediction model as observed across multiple studies, and may guide model updating strategies for a particular target population. Careful propensity score modeling with attention for non-linear relations is recommended.

Measuring the performance of prediction models to personalize treatment choice.

When data are available from individual patients receiving either a treatment or a control intervention in a randomized trial, various statistical and machine learning methods can be used to develop models for predicting future outcomes under the two conditions, and thus to predict treatment effect at the patient level. These predictions can subsequently guide personalized treatment choices. Although several methods for validating prediction models are available, little attention has been given to measuring the performance of predictions of personalized treatment effect. In this article, we propose a range of measures that can be used to this end. We start by defining two dimensions of model accuracy for treatment effects, for a single outcome: discrimination for benefit and calibration for benefit. We then amalgamate these two dimensions into an additional concept, decision accuracy, which quantifies the model's ability to identify patients for whom the benefit from treatment exceeds a given threshold. Subsequently, we propose a series of performance measures related to these dimensions and discuss estimating procedures, focusing on randomized data. Our methods are applicable for continuous or binary outcomes, for any type of prediction model, as long as it uses baseline covariates to predict outcomes under treatment and control. We illustrate all methods using two simulated datasets and a real dataset from a trial in depression. We implement all methods in the R package predieval. Results suggest that the proposed measures can be useful in evaluating and comparing the performance of competing models in predicting individualized treatment effect.

The relation between prediction model performance measures and patient selection outcomes for proton therapy in head and neck cancer.

BACKGROUND: Normal-tissue complication probability (NTCP) models predict complication risk in patients receiving radiotherapy, considering radiation dose to healthy tissues, and are used to select patients for proton therapy, based on their expected reduction in risk after proton therapy versus photon radiotherapy (ΔNTCP). Recommended model evaluation measures include area under the receiver operating characteristic curve (AUC), overall calibration (CITL), and calibration slope (CS), whose precise relation to patient selection is still unclear. We investigated how each measure relates to patient selection outcomes. METHODS: The model validation and consequent patient selection process was simulated within empirical head and neck cancer patient data. By manipulating performance measures independently via model perturbations, the relation between model performance and patient selection was studied. RESULTS: Small reductions in AUC (-0.02) yielded mean changes in ΔNTCP between 0.9-3.2 %, and single-model patient selection differences between 2-19 %. Deviations (-0.2 or +0.2) in CITL or CS yielded mean changes in ΔNTCP between 0.3-1.4 %, and single-model patient selection differences between 1-10 %. CONCLUSIONS: Each measure independently impacts ΔNTCP and patient selection and should thus be assessed in a representative sufficiently large external sample. Our suggested practical model selection approach is considering the model with the highest AUC, and recalibrating it if needed.

Diagnostic prediction models for CT-confirmed and bacterial rhinosinusitis in primary care: individual participant data meta-analysis.

BACKGROUND: Antibiotics are overused in patients with acute rhinosinusitis (ARS) as it is difficult to identify those who benefit from antibiotic treatment. AIM: To develop prediction models for computed tomography (CT)-confirmed ARS and culture-confirmed acute bacterial rhinosinusitis (ABRS) in adults presenting to primary care with symptoms suggestive of ARS. DESIGN AND SETTING: This was a systematic review and individual participant data meta-analysis. METHOD: CT-confirmed ARS was defined as the presence of fluid level or total opacification in any maxillary sinuses, whereas culture-confirmed ABRS was defined by culture of fluid from antral puncture. Prediction models were derived using logistic regression modelling. RESULTS: Among 426 patients from three studies, 140 patients (32.9%) had CT-confirmed ARS. A model consisting of seven variables: previous diagnosis of ARS, preceding upper respiratory tract infection, anosmia, double sickening, purulent nasal discharge on examination, need for antibiotics as judged by a physician, and C-reactive protein (CRP) showed an optimism-corrected c-statistic of 0.73 (95% confidence interval [CI] = 0.69 to 0.78) and a calibration slope of 0.99 (95% CI = 0.72 to 1.19). Among 225 patients from two studies, 68 patients (30.2%) had culture-confirmed ABRS. A model consisting of three variables: pain in teeth, purulent nasal discharge, and CRP showed an optimism-corrected c-statistic of 0.70 (95% CI = 0.63 to 0.77) and a calibration slope of 1.00 (95% CI = 0.66 to 1.52). Clinical utility analysis showed that both models could be useful to rule out the target condition. CONCLUSION: Simple prediction models for CT-confirmed ARS and culture-confirmed ABRS can be useful to safely reduce antibiotic use in adults with ARS in high-prescribing countries.

Accuracy of approximations to recover incompletely reported logistic regression models depended on other available information.

OBJECTIVE: To provide approximations to recover the full regression equation across different scenarios of incompletely reported prediction models that were developed from binary logistic regression. STUDY DESIGN AND SETTING: In a case study, we considered four common scenarios and illustrated their corresponding approximations: (A) Missing: the intercept, Available: the regression coefficients of predictors, overall frequency of the outcome and descriptive statistics of the predictors; (B) Missing: regression coefficients and the intercept, Available: a simplified score; (C) Missing: regression coefficients and the intercept, Available: a nomogram; (D) Missing: regression coefficients and the intercept, Available: a web calculator. RESULTS: In the scenario A, a simplified approach based on the predicted probability corresponding to the average linear predictor was inaccurate. An approximation based on the overall outcome frequency and an approximation of the linear predictor distribution was more accurate, however, the appropriateness of the underlying assumptions cannot be verified in practice. In the scenario B, the recovered equation was inaccurate due to rounding and categorization of risk scores. In the scenarios C and D, the full regression equation could be recovered with minimal error. CONCLUSION: The accuracy of the approximations in recovering the regression equation varied depending on the available information.

A tutorial on individualized treatment effect prediction from randomized trials with a binary endpoint.

Randomized trials typically estimate average relative treatment effects, but decisions on the benefit of a treatment are possibly better informed by more individualized predictions of the absolute treatment effect. In case of a binary outcome, these predictions of absolute individualized treatment effect require knowledge of the individual's risk without treatment and incorporation of a possibly differential treatment effect (ie, varying with patient characteristics). In this article, we lay out the causal structure of individualized treatment effect in terms of potential outcomes and describe the required assumptions that underlie a causal interpretation of its prediction. Subsequently, we describe regression models and model estimation techniques that can be used to move from average to more individualized treatment effect predictions. We focus mainly on logistic regression-based methods that are both well-known and naturally provide the required probabilistic estimates. We incorporate key components from both causal inference and prediction research to arrive at individualized treatment effect predictions. While the separate components are well known, their successful amalgamation is very much an ongoing field of research. We cut the problem down to its essentials in the setting of a randomized trial, discuss the importance of a clear definition of the estimand of interest, provide insight into the required assumptions, and give guidance with respect to modeling and estimation options. Simulated data illustrate the potential of different modeling options across scenarios that vary both average treatment effect and treatment effect heterogeneity. Two applied examples illustrate individualized treatment effect prediction in randomized trial data.

Identifying adults with acute rhinosinusitis in primary care that benefit most from antibiotics: protocol of an individual patient data meta-analysis using multivariable risk prediction modelling.

INTRODUCTION: Acute rhinosinusitis (ARS) is a prime reason for doctor visits and among the conditions with highest antibiotic overprescribing rates in adults. To reduce inappropriate prescribing, we aim to predict the absolute benefit of antibiotic treatment for individual adult patients with ARS by applying multivariable risk prediction methods to individual patient data (IPD) of multiple randomised placebo-controlled trials. METHODS AND ANALYSIS: This is an update and re-analysis of a 2008 IPD meta-analysis on antibiotics for adults with clinically diagnosed ARS. First, the reference list of the 2018 Cochrane review on antibiotics for ARS will be reviewed for relevant studies published since 2008. Next, the systematic searches of CENTRAL, MEDLINE and Embase of the Cochrane review will be updated to 1 September 2020. Methodological quality of eligible studies will be assessed using the Cochrane Risk of Bias 2 tool. The primary outcome is cure at 8-15 days. Regression-based methods will be used to model the risk of being cured based on relevant predictors and treatment, while accounting for clustering. Such model allows for risk predictions as a function of treatment and individual patient characteristics and hence gives insight into individualised absolute benefit. Candidate predictors will be based on literature, clinical reasoning and availability. Calibration and discrimination will be evaluated to assess model performance. Resampling techniques will be used to assess internal validation. In addition, internal-external cross-validation procedures will be used to inform on between-study differences and estimate out-of-sample model performance. Secondarily, we will study possible heterogeneity of treatment effect as a function of outcome risk. ETHICS AND DISSEMINATION: In this study, no identifiable patient data will be used. As such, the Medical Research Involving Humans Subject Act (WMO) does not apply and official ethical approval is not required. Results will be submitted for publication in international peer-reviewed journals. PROSPERO REGISTRATION NUMBER: CRD42020220108.

Real-time imputation of missing predictor values improved the application of prediction models in daily practice.

OBJECTIVES: In clinical practice, many prediction models cannot be used when predictor values are missing. We, therefore, propose and evaluate methods for real-time imputation. STUDY DESIGN AND SETTING: We describe (i) mean imputation (where missing values are replaced by the sample mean), (ii) joint modeling imputation (JMI, where we use a multivariate normal approximation to generate patient-specific imputations), and (iii) conditional modeling imputation (CMI, where a multivariable imputation model is derived for each predictor from a population). We compared these methods in a case study evaluating the root mean squared error (RMSE) and coverage of the 95% confidence intervals (i.e., the proportion of confidence intervals that contain the true predictor value) of imputed predictor values. RESULTS: -RMSE was lowest when adopting JMI or CMI, although imputation of individual predictors did not always lead to substantial improvements as compared to mean imputation. JMI and CMI appeared particularly useful when the values of multiple predictors of the model were missing. Coverage reached the nominal level (i.e., 95%) for both CMI and JMI. CONCLUSION: Multiple imputations using either CMI or JMI is recommended when dealing with missing predictor values in real-time settings.

Real-time imputation of missing predictor values in clinical practice.

Aims: Use of prediction models is widely recommended by clinical guidelines, but usually requires complete information on all predictors, which is not always available in daily practice. We aim to describe two methods for real-time handling of missing predictor values when using prediction models in practice. Methods and results: We compare the widely used method of mean imputation (M-imp) to a method that personalizes the imputations by taking advantage of the observed patient characteristics. These characteristics may include both prediction model variables and other characteristics (auxiliary variables). The method was implemented using imputation from a joint multivariate normal model of the patient characteristics (joint modelling imputation; JMI). Data from two different cardiovascular cohorts with cardiovascular predictors and outcome were used to evaluate the real-time imputation methods. We quantified the prediction model's overall performance [mean squared error (MSE) of linear predictor], discrimination (c-index), calibration (intercept and slope), and net benefit (decision curve analysis). When compared with mean imputation, JMI substantially improved the MSE (0.10 vs. 0.13), c-index (0.70 vs. 0.68), and calibration (calibration-in-the-large: 0.04 vs. 0.06; calibration slope: 1.01 vs. 0.92), especially when incorporating auxiliary variables. When the imputation method was based on an external cohort, calibration deteriorated, but discrimination remained similar. Conclusions: We recommend JMI with auxiliary variables for real-time imputation of missing values, and to update imputation models when implementing them in new settings or (sub)populations.

Handling missing predictor values when validating and applying a prediction model to new patients.

Missing data present challenges for development and real-world application of clinical prediction models. While these challenges have received considerable attention in the development setting, there is only sparse research on the handling of missing data in applied settings. The main unique feature of handling missing data in these settings is that missing data methods have to be performed for a single new individual, precluding direct application of mainstay methods used during model development. Correspondingly, we propose that it is desirable to perform model validation using missing data methods that transfer to practice in single new patients. This article compares existing and new methods to account for missing data for a new individual in the context of prediction. These methods are based on (i) submodels based on observed data only, (ii) marginalization over the missing variables, or (iii) imputation based on fully conditional specification (also known as chained equations). They were compared in an internal validation setting to highlight the use of missing data methods that transfer to practice while validating a model. As a reference, they were compared to the use of multiple imputation by chained equations in a set of test patients, because this has been used in validation studies in the past. The methods were evaluated in a simulation study where performance was measured by means of optimism corrected C-statistic and mean squared prediction error. Furthermore, they were applied in data from a large Dutch cohort of prophylactic implantable cardioverter defibrillator patients.

Individual participant data meta-analysis to examine interactions between treatment effect and participant-level covariates: Statistical recommendations for conduct and planning.

Precision medicine research often searches for treatment-covariate interactions, which refers to when a treatment effect (eg, measured as a mean difference, odds ratio, hazard ratio) changes across values of a participant-level covariate (eg, age, gender, biomarker). Single trials do not usually have sufficient power to detect genuine treatment-covariate interactions, which motivate the sharing of individual participant data (IPD) from multiple trials for meta-analysis. Here, we provide statistical recommendations for conducting and planning an IPD meta-analysis of randomized trials to examine treatment-covariate interactions. For conduct, two-stage and one-stage statistical models are described, and we recommend: (i) interactions should be estimated directly, and not by calculating differences in meta-analysis results for subgroups; (ii) interaction estimates should be based solely on within-study information; (iii) continuous covariates and outcomes should be analyzed on their continuous scale; (iv) nonlinear relationships should be examined for continuous covariates, using a multivariate meta-analysis of the trend (eg, using restricted cubic spline functions); and (v) translation of interactions into clinical practice is nontrivial, requiring individualized treatment effect prediction. For planning, we describe first why the decision to initiate an IPD meta-analysis project should not be based on between-study heterogeneity in the overall treatment effect; and second, how to calculate the power of a potential IPD meta-analysis project in advance of IPD collection, conditional on characteristics (eg, number of participants, standard deviation of covariates) of the trials (potentially) promising their IPD. Real IPD meta-analysis projects are used for illustration throughout.

Added value of inflammatory markers to vital signs to predict mortality in patients suspected of severe infection.

OBJECTIVE: To evaluate the added value of inflammatory markers to vital signs to predict mortality in patients suspected of severe infection. METHODS: This study was conducted at an acute care hospital (471-bed capacity). Consecutive adult patients suspected of severe infection who presented to either ambulatory care or the emergency department from April 2015 to March 2017 were retrospectively evaluated. A prognostic model for predicting 30-day in-hospital mortality based on previously established vital signs (systolic blood pressure, respiratory rate, and mental status) was compared with an extended model that also included four inflammatory markers (C-reactive protein, neutrophil-lymphocyte ratio, mean platelet volume, and red cell distribution width). Measures of interest were model fit, discrimination, and the net percentage of correctly reclassified individuals at the pre-specified threshold of 10% risk. RESULTS: Of the 1015 patients included, 66 (6.5%) died. The extended model including inflammatory markers performed significantly better than the vital sign model (likelihood ratio test: p < 0.001), and the c-index increased from 0.69 (range 0.67-0.70) to 0.76 (range 0.75-0.77) (p = 0.01). All included markers except C-reactive protein showed significant contribution to the model improvement. Among those who died, 9.1% (95% CI -2.8-21.8) were correctly reclassified by the extended model at the 10% threshold. CONCLUSIONS: The inflammatory markers except C-reactive protein showed added predictive value to vital signs. Future studies should focus on developing and validating prediction models for use in individualized predictions including both vital signs and the significant markers.