Adjusting for partial verification or workup bias in meta-analyses of diagnostic accuracy studies.

A key requirement in the design of diagnostic accuracy studies is that all study participants receive both the test under evaluation and the reference standard test. For a variety of practical and ethical reasons, sometimes only a proportion of patients receive the reference standard, which can bias the accuracy estimates. Numerous methods have been described for correcting this partial verification bias or workup bias in individual studies. In this article, the authors describe a Bayesian method for obtaining adjusted results from a diagnostic meta-analysis when partial verification or workup bias is present in a subset of the primary studies. The method corrects for verification bias without having to exclude primary studies with verification bias, thus preserving the main advantages of a meta-analysis: increased precision and better generalizability. The results of this method are compared with the existing methods for dealing with verification bias in diagnostic meta-analyses. For illustration, the authors use empirical data from a systematic review of studies of the accuracy of the immunohistochemistry test for diagnosis of human epidermal growth factor receptor 2 status in breast cancer patients.

Adjusting for differential-verification bias in diagnostic-accuracy studies: a Bayesian approach.

In studies of diagnostic accuracy, the performance of an index test is assessed by verifying its results against those of a reference standard. If verification of index-test results by the preferred reference standard can be performed only in a subset of subjects, an alternative reference test could be given to the remainder. The drawback of this so-called differential-verification design is that the second reference test is often of lesser quality, or defines the target condition in a different way. Incorrectly treating results of the 2 reference standards as equivalent will lead to differential-verification bias. The Bayesian methods presented in this paper use a single model to (1) acknowledge the different nature of the 2 reference standards and (2) make simultaneous inferences about the population prevalence and the sensitivity, specificity, and predictive values of the index test with respect to both reference tests, in relation to latent disease status. We illustrate this approach using data from a study on the accuracy of the elbow extension test for diagnosis of elbow fractures in patients with elbow injury, using either radiography or follow-up as reference standards.

Diagnostic accuracy and user-friendliness of 5 point-of-care D-dimer tests for the exclusion of deep vein thrombosis.

BACKGROUND: Point-of-care D-dimer tests have recently been introduced to enable rapid exclusion of deep venous thrombosis (DVT) without the need to refer a patient for conventional laboratory-based D-dimer testing. Before implementation in practice, however, the diagnostic accuracy of each test should be validated. METHODS: We analyzed data of 577 prospectively identified consecutive primary care patients suspected to have DVT, who underwent 5 point-of-care D-dimer tests-4 quantitative (Vidas®, Pathfast™, Cardiac®, and Triage®) and 1 qualitative (Clearview Simplify®)-and ultrasonography as the reference method. We evaluated the tests for the accuracy of their measurements and submitted a questionnaire to 20 users to assess the user-friendliness of each test. RESULTS: All D-dimer tests showed negative predictive values higher than 98%. Sensitivity was high for all point-of-care tests, with a range of 0.91 (Clearview Simplify) to 0.99 (Vidas). Specificity varied between 0.39 (Pathfast) and 0.64 (Clearview Simplify). The quantitative point-of-care tests showed similar and high discriminative power for DVT, according to calculated areas under the ROC curves (range 0.88-0.89). The quantitative Vidas and Pathfast devices showed limited user-friendliness for primary care, owing to a laborious calibration process and long analyzer warm-up time compared to the Cardiac and Triage. For the qualitative Clearview Simplify assay, no analyzer or calibration was needed, but interpretation of a test result was sometimes difficult because of poor color contrast. CONCLUSIONS: Point-of-care D-dimer assays show good and similar diagnostic accuracy. The quantitative Cardiac and Triage and the qualitative Clearview Simplify D-dimer seem most user-friendly for excluding DVT in the doctor's office.

Dealing with missing predictor values when applying clinical prediction models.

BACKGROUND: Prediction models combine patient characteristics and test results to predict the presence of a disease or the occurrence of an event in the future. In the event that test results (predictor) are unavailable, a strategy is needed to help users applying a prediction model to deal with such missing values. We evaluated 6 strategies to deal with missing values. METHODS: We developed and validated (in 1295 and 532 primary care patients, respectively) a prediction model to predict the risk of deep venous thrombosis. In an application set (259 patients), we mimicked 3 situations in which (1) an important predictor (D-dimer test), (2) a weaker predictor (difference in calf circumference), and (3) both predictors simultaneously were missing. The 6 strategies to deal with missing values were (1) ignoring the predictor, (2) overall mean imputation, (3) subgroup mean imputation, (4) multiple imputation, (5) applying a submodel including only the observed predictors as derived from the development set, or (6) the "one-step-sweep" method. We compared the model's discriminative ability (expressed by the ROC area) with the true ROC area (no missing values) and the model's estimated calibration slope and intercept with the ideal values of 1 and 0, respectively. RESULTS: Ignoring the predictor led to the worst and multiple imputation to the best discrimination. Multiple imputation led to calibration intercepts closest to the true value. The effect of the strategies on the slope differed between the 3 scenarios. CONCLUSIONS: Multiple imputation is preferred if a predictor value is missing.

Prediction rule for cardiovascular events and mortality in peripheral arterial disease patients: data from the prospective Second Manifestations of ARTerial disease (SMART) cohort study.

BACKGROUND: Patients with peripheral arterial disease (PAD) are at high risk of secondary cardiovascular death and events such as myocardial infarction or stroke. To minimize this elevated risk, cardiovascular risk factors should be treated in all PAD patients. Secondary risk management may benefit from a prediction tool to identify PAD patients at the highest risk who could be referred for an additional extensive workup. Stratifying PAD patients according to their risk of secondary events could aid in achieving optimal therapy compliance. To this end we developed a prediction model for secondary cardiovascular events in PAD patients. METHODS: The model was developed using data from 800 PAD patients who participated in the Second Manifestations of ARTerial disease (SMART) cohort study. From the baseline characteristics, 13 candidate predictors were selected for the model development. Missing values were imputed by means of single regression imputation. Continuous predictors were truncated and transformed where necessary, followed by model reduction by means of backward stepwise selection. To correct for over-fitting, a bootstrapping technique was applied. Finally, a score chart was created that divides patients in four risk categories that have been linked to the risk of a cardiovascular event during 1- and 5-year follow-up. RESULTS: During a mean follow-up of 4.7 years, 120 events occurred (27% nonfatal myocardial infarction, 21% nonfatal stroke, and 52% mortality from vascular causes), corresponding to a 1- and 5-year cumulative incidence of 3.1% and 13.2%, respectively. Important predictors for the secondary risk of a cardiovascular event are age, history of symptomatic cardiovascular disease, systolic blood pressure, high-density lipoprotein cholesterol, smoking behavior, ankle-brachial pressure index, and creatinine level. The risk of a cardiovascular event in a patient as predicted by the model was 0% to 10% and 1% to 28% for the four risk categories at 1- and 5-year follow-up, respectively. The discriminating capacity of the prediction model, indicated by the c statistic, was 0.76 (95% confidence interval, 0.71-0.80). CONCLUSION: A prediction model can be used to predict secondary cardiovascular risk in PAD patients. We propose such a prediction model to allow for the identification of PAD patients at the highest risk of a cardiovascular event or cardiovascular death, which may be a viable tool in vascular secondary health care practice.

The risk of severe postoperative pain: modification and validation of a clinical prediction rule.

BACKGROUND: Recently, a prediction rule was developed to preoperatively predict the risk of severe pain in the first postoperative hour in surgical inpatients. We aimed to modify the rule to enhance its use in both surgical inpatients and outpatients (ambulatory patients). Subsequently, we prospectively tested the modified rule in patients who underwent surgery later in time and in another hospital (external validation). METHODS: The rule was originally developed from the data of 1395 adult inpatients. We modified the rule with the data of 549 outpatients who underwent surgery between 1997 and 1999 in the same center (Academic Medical Center Amsterdam, The Netherlands). Furthermore, we tested the performance of the modified rule in 1035 in- and outpatients who underwent surgery in 2004, in the University Medical Center Utrecht, The Netherlands (external validation). Performance was quantified by the rule's calibration (agreement between observed frequencies and predicted risks) and discrimination (ability to distinguish between patients at high and low risk). RESULTS: Modification of the original rule to enhance prediction in outpatients included reclassification of the predictor "type of surgery," addition of the predictor "surgical setting" (ambulatory surgery: yes/no) and addition of interaction terms between surgical setting and the other predictors. One-third of the patients in the Utrecht cohort reported severe postoperative pain (36%), compared to 62% of the patients in the Amsterdam cohort. The distribution of most predictors was similar in the two cohorts, although the patients in the Utrecht cohort were slightly older, more often underwent ambulatory surgery and had large expected incision sizes less often than patients in the Amsterdam cohort. The modified prediction rule showed good calibration, when an adjusted intercept was used for the lower incidence in the Utrecht cohort. The discrimination was reasonable (area under the Receiver Operating Characteristic curve 0.65 [95% confidence interval 0.57-0.73]). CONCLUSIONS: A previously developed prediction rule to predict severe postoperative pain was modified to allow use in both inpatients and outpatients. By validating the rule in patients who underwent surgery several years later in another hospital, it was shown that the rule could be generalized in time and place. We demonstrated that, instead of deriving new prediction rules for new populations, a simple adjustment may be enough to recalibrate prediction rules for new populations. This is in line with the perception that external validation and updating of prediction rules is a continuing and multistage process.

Reporting of Bayesian analysis in epidemiologic research should become more transparent.

OBJECTIVES: The objective of this systematic review is to investigate the use of Bayesian data analysis in epidemiology in the past decade and particularly to evaluate the quality of research papers reporting the results of these analyses. STUDY DESIGN AND SETTING: Complete volumes of five major epidemiological journals in the period 2005-2015 were searched via PubMed. In addition, we performed an extensive within-manuscript search using a specialized Java application. Details of reporting on Bayesian statistics were examined in the original research papers with primary Bayesian data analyses. RESULTS: The number of studies in which Bayesian techniques were used for primary data analysis remains constant over the years. Though many authors presented thorough descriptions of the analyses they performed and the results they obtained, several reports presented incomplete method sections and even some incomplete result sections. Especially, information on the process of prior elicitation, specification, and evaluation was often lacking. CONCLUSION: Though available guidance papers concerned with reporting of Bayesian analyses emphasize the importance of transparent prior specification, the results obtained in this systematic review show that these guidance papers are often not used. Additional efforts should be made to increase the awareness of the existence and importance of these checklists to overcome the controversy with respect to the use of Bayesian techniques. The reporting quality in epidemiological literature could be improved by updating existing guidelines on the reporting of frequentist analyses to address issues that are important for Bayesian data analyses.

Diagnostic accuracy of conventional or age adjusted D-dimer cut-off values in older patients with suspected venous thromboembolism: systematic review and meta-analysis.

OBJECTIVE: To review the diagnostic accuracy of D-dimer testing in older patients (>50 years) with suspected venous thromboembolism, using conventional or age adjusted D-dimer cut-off values. DESIGN: Systematic review and bivariate random effects meta-analysis. DATA SOURCES: We searched Medline and Embase for studies published before 21 June 2012 and we contacted the authors of primary studies. STUDY SELECTION: Primary studies that enrolled older patients with suspected venous thromboembolism in whom D-dimer testing, using both conventional (500 µg/L) and age adjusted (age × 10 µg/L) cut-off values, and reference testing were performed. For patients with a non-high clinical probability, 2 × 2 tables were reconstructed and stratified by age category and applied D-dimer cut-off level. RESULTS: 13 cohorts including 12,497 patients with a non-high clinical probability were included in the meta-analysis. The specificity of the conventional cut-off value decreased with increasing age, from 57.6% (95% confidence interval 51.4% to 63.6%) in patients aged 51-60 years to 39.4% (33.5% to 45.6%) in those aged 61-70, 24.5% (20.0% to 29.7% in those aged 71-80, and 14.7% (11.3% to 18.6%) in those aged >80. Age adjusted cut-off values revealed higher specificities over all age categories: 62.3% (56.2% to 68.0%), 49.5% (43.2% to 55.8%), 44.2% (38.0% to 50.5%), and 35.2% (29.4% to 41.5%), respectively. Sensitivities of the age adjusted cut-off remained above 97% in all age categories. CONCLUSIONS: The application of age adjusted cut-off values for D-dimer tests substantially increases specificity without modifying sensitivity, thereby improving the clinical utility of D-dimer testing in patients aged 50 or more with a non-high clinical probability.

A simple adaptation method improved the interpretability of prediction models for composite end points.

OBJECTIVE: The pros and cons of composite end points in prognostic research are discussed, and an adaptation method, designed to accurately adjust absolute risks for a composite end point to risks for the individual component outcomes, is presented. STUDY DESIGN AND SETTING: An example prediction model for recurrent cardiovascular events (composite end point) was used to evaluate the performance regarding the individual component outcomes (cardiovascular death, myocardial infarction, and stroke) before and after the adaptation method. RESULTS: Discrimination for the individual component outcomes (concordance index for myocardial infarction, 0.68; concordance index for stroke, 0.70) was very similar to discrimination for the original composite end point (concordance index, 0.70). For cardiovascular death, it even increased substantially (concordance index, 0.78). After adaptation, calibration plots for the component outcomes also improved, with visible convergence of the predicted risks and the observed incidences. CONCLUSION: In sum, these findings show that the adaptation method is useful when validating or applying a composite end point prediction model to the individual component outcomes. Following from this, recommendations concerning reporting of composite end points in future research are also included. Without the need for extra data, composite end point prediction models can easily be directly expanded to allow for the estimation of risk for each individual component outcome, improving the interpretability for clinicians and patients.

Validation of two age dependent D-dimer cut-off values for exclusion of deep vein thrombosis in suspected elderly patients in primary care: retrospective, cross sectional, diagnostic analysis.

OBJECTIVE: To determine whether the use of age adapted D-dimer cut-off values can be translated to primary care patients who are suspected of deep vein thrombosis. DESIGN: Retrospective, cross sectional diagnostic study. SETTING: 110 primary care doctors affiliated with three hospitals in the Netherlands. PARTICIPANTS: 1374 consecutive patients (936 (68.1%) aged >50 years) with clinically suspected deep vein thrombosis. MAIN OUTCOME MEASURES: Proportion of patients with D-dimer values below two proposed age adapted cut-off levels (age in years × 10 µg/L in patients aged >50 years, or 750 µg/L in patients aged ≥ 60 years), in whom deep vein thrombosis could be excluded; and the number of false negative results. RESULTS: Using the Wells score, 647 patients had an unlikely clinical probability of deep vein thrombosis. In these patients (at all ages), deep vein thrombosis could be excluded in 309 (47.8%) using the age dependent cut-off value compared with 272 (42.0%) using the conventional cut-off value of 500 µg/L (increase 5.7%, 95% confidence interval 4.1% to 7.8%). This exclusion rate resulted in 0.5% and 0.3% false negative cases, respectively (increase 0.2%, 0.004% to 8.6%).The increase in exclusion rate by using the age dependent cut-off value was highest in the oldest patients. In patients older than 80 years, deep vein thrombosis could be safely excluded in 22 (35.5%) patients using the age dependent cut-off value compared with 13 (21.0%) using the conventional cut-off value (increase 14.5%, 6.8% to 25.8%). Compared with the age dependent cut-off value, the cut-off value of 750 µg/L had a similar exclusion rate (307 (47.4%) patients) and false negative rate (0.3%). CONCLUSIONS: Combined with a low clinical probability of deep vein thrombosis, use of the age dependent D-dimer cut-off value for patients older than 50 years or the cut-off value of 750 µg/L for patients aged 60 years and older resulted in a considerable increase in the proportion of patients in primary care in whom deep vein thrombosis could be safely excluded, compared with the conventional cut-off value of 500 µg/L.

Development and validation of clinical prediction models: marginal differences between logistic regression, penalized maximum likelihood estimation, and genetic programming.

OBJECTIVE: Many prediction models are developed by multivariable logistic regression. However, there are several alternative methods to develop prediction models. We compared the accuracy of a model that predicts the presence of deep venous thrombosis (DVT) when developed by four different methods. STUDY DESIGN AND SETTING: We used the data of 2,086 primary care patients suspected of DVT, which included 21 candidate predictors. The cohort was split into a derivation set (1,668 patients, 329 with DVT) and a validation set (418 patients, 86 with DVT). Also, 100 cross-validations were conducted in the full cohort. The models were developed by logistic regression, logistic regression with shrinkage by bootstrapping techniques, logistic regression with shrinkage by penalized maximum likelihood estimation, and genetic programming. The accuracy of the models was tested by assessing discrimination and calibration. RESULTS: There were only marginal differences in the discrimination and calibration of the models in the validation set and cross-validations. CONCLUSION: The accuracy measures of the models developed by the four different methods were only slightly different, and the 95% confidence intervals were mostly overlapped. We have shown that models with good predictive accuracy are most likely developed by sensible modeling strategies rather than by complex development methods.

Accuracy of symptoms, signs, and C-reactive protein for early chronic obstructive pulmonary disease.

BACKGROUND: Guidelines recommend detection of early chronic obstructive pulmonary disease (COPD), but evidence on the diagnostic work-up for COPD only concerns advanced and established COPD. AIM: To quantify the accuracy of symptoms and signs for early COPD, and the added value of C-reactive protein (CRP), in primary care patients presenting with cough. DESIGN AND SETTING: Cross-sectional diagnostic study of 73 primary care practices in the Netherlands. METHOD: Four hundred primary care patients (182 males, mean age 63 years) older than 50 years, presenting with persistent cough (>14 days) without established COPD participated, of whom 382 completed the study. They underwent a systematic diagnostic work-up of symptoms, signs, conventional laboratory CRP level, and hospital lung functions tests, including body plethysmography, and an expert panel decided whether COPD was present (reference test). The independent value of all items was estimated by multivariable logistic regression analysis. RESULTS: According to the expert panel, 118 patients had COPD (30%). Symptoms and signs with independent diagnostic value were age, sex, current smoking, smoking more than 20 pack-years, cardiovascular comorbidity, wheezing complaints, diminished breath sounds, and wheezing on auscultation. Combining these items resulted in an area under the receiver operating characteristic curve (ROC area) of 0.79 (95% confidence interval = 0.74 to 0.83) after internal validation. The proportion of subjects with elevated CRP was higher in those with early COPD, but CRP added no relevant diagnostic information above symptoms and signs. CONCLUSION: In subjects presenting with persistent cough, the CRP level has no added value for detection of early COPD.

Incorporation of historical data in the analysis of randomized therapeutic trials.

Historical studies provide a valuable source of information for the motivation and design of later trials. Bayesian techniques offer possibilities for the quantitative inclusion of prior knowledge within the analysis of current trial data. Combining information from previous studies into an informative prior distribution is, however, a delicate case. The power prior distribution is a tool to estimate the effect of an intervention in a current study sample, while accounting for the information provided by previous research. In this study we evaluate the use of the power prior distribution, illustrated with data from a large randomized clinical trial on the effect of ST-wave analysis in intrapartum fetal monitoring. We advocate the use of a power prior distribution with pre-specified fixed study weights based on differences in study characteristics. We propose obtaining a ranking of the historical studies via expert elicitation, based on relevance for the current study, and specify study weights accordingly.

Correcting for partial verification bias: a comparison of methods.

PURPOSE: A common problem in diagnostic research is that the reference standard has not been carried out in all patients. This partial verification may lead to biased accuracy measures of the test under study. The authors studied the performance of multiple imputation and the conventional correction method proposed by Begg and Greenes under a range of different situations of partial verification. METHODS: In a series of simulations, using a previously published deep venous thrombosis data set (n = 1292), the authors set the outcome of the reference standard to missing based on various underlying mechanisms and by varying the total number of missing values. They then compared the performance of the different correction methods. RESULTS: The results of the study show that when the mechanism of missing reference data is known, accuracy measures can easily be correctly adjusted using either the Begg and Greenes method, or multiple imputation. In situations where the mechanism of missing reference data is complex or unknown, we recommend using multiple imputation methods to correct. CONCLUSIONS: These methods can easily apply for both continuous and categorical variables, are readily available in statistical software and give reliable estimates of the missing reference data.

Optimisation of the diagnostic strategy for suspected deep-vein thrombosis in primary care.

Recently, a diagnostic score was developed to safely exclude deep-vein thrombosis (DVT) in primary care. A large prospective study, in which general practitioners used this diagnostic score to decide which patients needed referral, revealed that the number of referrals for ultrasound measurements was reduced by almost 50%, at the cost of an acceptably low risk (1.4%, 95% confidence interval [CI] 0.6% to 2.9%) of venous thromboembolic events in non-referred patients. However, simple adjustments to the diagnostic score (so-called updating) might further improve the accuracy; i.e. reduce the proportion of missed diagnoses (safety) or increase the proportion of patients who do not need to be referred (efficiency). We applied two updating methods to determine whether adjusting the weights of the predictors or adding new predictors could further improve the accuracy of the diagnostic score. The weights of the predictors did not need to be adjusted, but inclusion of 'history of DVT' and 'prolonged travelling' significantly added predictive value (p-values 0.014 and 0.023, respectively). However, adding these predictors to the diagnostic score did not improve the safety and efficiency: at equal safety (1.4% missed diagnoses among the non-referred patients), the efficiency was lower (43.5%, 95% CI 40.4% to 46.6% compared to 49.4%, 95% CI 46.3% to 52.5%). The diagnostic score for excluding DVT in primary care has good accuracy in its original form and could not be improved by including additional predictors. This suggests that the original diagnostic score can be used to safely exclude clinically suspected DVT in primary care.

A model for preconceptional prediction of recurrent early-onset preeclampsia: derivation and internal validation.

OBJECTIVE: To develop a model to identify women at very low risk of recurrent early-onset preeclampsia. METHODS: We enrolled 407 women who had experienced early-onset preeclampsia in their first pregnancy, resulting in a delivery before 34 weeks' gestation. Preeclampsia was defined as hypertension (systolic blood pressure ≥140 mm Hg and/or diastolic blood pressure ≥90 mm Hg) after 20 weeks' gestation with de novo proteinuria (≥300 mg urinary protein excretion/day). Based on the previous published evidence and expert opinion, 5 predictors (gestational age at previous birth, prior small-for-gestational-age newborn, fasting blood glucose, body mass index, and hypertension) were entered in a logistic regression model. Discrimination and calibration were evaluated after adjusting for overfitting by bootstrapping techniques. RESULTS: Early-onset disease recurred in 28 (6.9%) of 407 women. The area under the receiver operating characteristic (ROC) curve of the model was 0.65 (95% CI: 0.56-0.74). Calibration was good, indicated by a nonsignificant Hosmer-Lemeshow test (P = .11). Using a predicted absolute risk threshold of, for example, 4.6% (ie, women identified with an estimated risk either above or below 4.6%), the sensitivity was 100%, with a specificity of 26%. In such a strategy, no women who developed preeclampsia were missed, while 98 of the 407 women would be regarded as low risk of recurrent early-onset preeclampsia, not necessarily requiring intensified antenatal care. CONCLUSION: Our model may be helpful in the identification of women at very low risk of recurrent early-onset preeclampsia. Before widespread application, our model should be validated in other populations.

Prediction model for hypertension in pregnancy in nulliparous women using information obtained at the first antenatal visit.

OBJECTIVE: To develop a prediction model for the occurrence of hypertension in pregnancy using clinical variables obtained routinely at the antenatal booking visit prior to 16 weeks gestation. METHODS: We studied 2334 nulliparous pregnant women participating in two population-based prospective cohort studies. Potential predictors included maternal age, blood pressure, body weight, height, previous miscarriage and smoking history, assessed at the visit booking prior to 16 weeks gestation. The outcome measure was the development of hypertension before 36 weeks of gestation. A prediction model was derived from the regression model using discrimination, calibration, bootstrapping approaches and transformed into a prediction model usable in clinical practice. RESULTS: One hundred and forty-one of 2334 women (6.0%) developed hypertension. Main predictors were systolic and diastolic blood pressure, and weight. The area under the receiver operating characteristic curve of the model was 0.78, 95% confidence interval (CI) 0.75-0.82. Among women with a very low score (19% of the population) the risk of hypertension was 0.5%. In those with a high score (13% of the population) the risk was 22.9%. CONCLUSION: Among nulliparous and initially normotensive women, the use of three simple clinical variables obtained routinely at the antenatal booking visit prior to 16 weeks, can accurately identify women at very low and very high risk of becoming hypertensive before 36 weeks of gestation. When confirmed in recent cohorts, application of the prediction model may lead to a reduction in frequency of antenatal visits for low-risk and increased surveillance for high-risk women.

Missing covariate data in medical research: to impute is better than to ignore.

OBJECTIVE: We compared popular methods to handle missing data with multiple imputation (a more sophisticated method that preserves data). STUDY DESIGN AND SETTING: We used data of 804 patients with a suspicion of deep venous thrombosis (DVT). We studied three covariates to predict the presence of DVT: d-dimer level, difference in calf circumference, and history of leg trauma. We introduced missing values (missing at random) ranging from 10% to 90%. The risk of DVT was modeled with logistic regression for the three methods, that is, complete case analysis, exclusion of d-dimer level from the model, and multiple imputation. RESULTS: Multiple imputation showed less bias in the regression coefficients of the three variables and more accurate coverage of the corresponding 90% confidence intervals than complete case analysis and dropping d-dimer level from the analysis. Multiple imputation showed unbiased estimates of the area under the receiver operating characteristic curve (0.88) compared with complete case analysis (0.77) and when the variable with missing values was dropped (0.65). CONCLUSION: As this study shows that simple methods to deal with missing data can lead to seriously misleading results, we advise to consider multiple imputation. The purpose of multiple imputation is not to create data, but to prevent the exclusion of observed data.

Unpredictable bias when using the missing indicator method or complete case analysis for missing confounder values: an empirical example.

OBJECTIVE: Missing indicator method (MIM) and complete case analysis (CC) are frequently used to handle missing confounder data. Using empirical data, we demonstrated the degree and direction of bias in the effect estimate when using these methods compared with multiple imputation (MI). STUDY DESIGN AND SETTING: From a cohort study, we selected an exposure (marital status), outcome (depression), and confounders (age, sex, and income). Missing values in "income" were created according to different patterns of missingness: missing values were created completely at random and depending on exposure and outcome values. Percentages of missing values ranged from 2.5% to 30%. RESULTS: When missing values were completely random, MIM gave an overestimation of the odds ratio, whereas CC and MI gave unbiased results. MIM and CC gave under- or overestimations when missing values depended on observed values. Magnitude and direction of bias depended on how the missing values were related to exposure and outcome. Bias increased with increasing percentage of missing values. CONCLUSION: MIM should not be used in handling missing confounder data because it gives unpredictable bias of the odds ratio even with small percentages of missing values. CC can be used when missing values are completely random, but it gives loss of statistical power.