Radiologists' Self-Assessment Versus Peer Assessment of Perceived Probability of Recommending Additional Imaging.

OBJECTIVE: Determine radiologist ability to accurately select the probability of recommendation of additional imaging (RAI) for themselves and colleagues when arrayed in a feedback report. METHODS: In this institutional review board-approved study, we analyzed 318,366 diagnostic imaging reports from examinations performed in the radiology department of a large quaternary teaching hospital during calendar year 2016. A validated machine learning algorithm identified reports containing RAI. A multivariable logistic regression model was then used to determine the probability of RAI. In 2018, an e-mailed survey asked radiologists to identify their own RAI probability and that of their colleagues from a report arrayed lowest to highest. Radiologists were grouped into quartiles based on their RAI probability. χ2 Analysis compared self-assessment and assessment of colleagues between quartiles. RESULTS: Forty-eight of 57 radiologists completed the survey (84.2%). Fourteen (29.2%) accurately self-identified their RAI probability (chose the correct quartile); 34 (70.8%) did not. There was no statistically significant difference between quartiles of radiologists and their ability to self-identify their RAI probability (ie, radiologists in the bottom or top quartile of RAI probabilities did not correctly predict their RAI probability). However, radiologists were better able to identify the RAI probability of their colleagues who were in the top and bottom quartiles. DISCUSSION: Radiologists were unable to estimate their own RAI probability but were better at predicting the RAI probability of colleagues. Given that radiologists, and physicians in general, may be poor evaluators of their own performance, objective assessment tools are likely needed to help reduce unwarranted variation.

The acute care diagnostics collaboration: Performance assessment of contrast-enhanced ultrasound compared to abdominal computed tomography and conventional ultrasound in an emergency trauma score bayesian clinical decision scheme.

BACKGROUND: Bayes' theorem describes the probability of an event, based on conditions that might be related to the event.[1] We developed the Bayesian Diagnostic Gains (BDG) method as a simple tool for interpreting diagnostic impact.[234567]. AIM: We aimed to evaluate the clinical diagnostic impact of contrast-enhanced ultrasound (CEUS) compared to traditional abdominal computed tomography (CT) and standard ultrasound (US) in a Bayesian Clinical Decision Scheme. MATERIALS AND METHODS: Our mathematical method uses Bayesian Diagnostic Gains (BDG) model. For the purposes of our model, the EMTRAS was used as pretest probability and stratified as low risk (0-3 points = 10%), moderate risk (4-6 points = 42%), and high risk (7-12 points = 80%) based on mortality risk. Sensitivity and specificity for US, CT, and CEUS were obtained from pooled data and used to calculate LR- and LR+. Bayesian/Fagan nomogram was used to attain posttest probabilities using baseline probability of an event on the first axis (PRE), with LR on the second axis, and read off the pos-test probability (POST) on the third axis. For the nomogram analysis, the pretest probability (Pre) scoring for the EMTRAS score was obtained using the original EMTRAS data. Posttest probabilities were obtained based on the Bayes/Fagan Nomgram. Relative diagnostic gain (RDG) and absolute diagnostic gain (ADG) were calculated based on the differences deducted from pre- and post-test probabilities. IBM® SPSS® Statistics 20 was used for analysis and modeling. ANOVA was used for association between EMTRAS, CT scan, and CEUS, where P value set at 0.05. RESULTS: Pooled data for Sensitivity (Se), Specificity (Sp), LR+, and LR- were obtained for US (Se = 45.7%, Sp = 91.8%, LR+ = 5.57, and LR- = 0.59), CEUS (Se 91.4%, Sp 100%, LR+ 91, and LR-0.09), and CT (Se = 94.8%, SP = 98.7%, LR+ = 73, and LR- =0.05). ANOVA analysis for LR+ and LR- showed no significant difference (P < 0.8745 and P < 0.9841). Comparison of CT and CEUS did not yield statistically significant differences for LR+ (P < 0.1). CONCLUSION: In this Bayesian model, the diagnostic performance of CEUS was found to be similar to traditional abdominal CT. The greatest diagnostic gain was observed in low pretest positive LR groups.

Incremental diagnostic quality gain of CTA over V/Q scan in the assessment of pulmonary embolism by means of a Wells score Bayesian model: results from the ACDC collaboration.

OBJECTIVE: Our objective was to evaluate the diagnostic value of computed tomography angiography (CTA) and ventilation perfusion (V/Q) scan in the assessment of pulmonary embolism (PE) by means of a Bayesian statistical model. METHODS: Wells criteria defined pretest probability. Sensitivity and specificity of CTA and V/Q scan for PE were derived from pooled meta-analysis data. Likelihood ratios calculated for CTA and V/Q were inserted in the nomogram. Absolute (ADG) and relative diagnostic gains (RDG) were analyzed comparing post- and pretest probability. Comparative gain difference was calculated for CTA ADG over V/Q scan integrating ANOVA p value set at 0.05. RESULTS: The sensitivity for CT was 86.0% (95% CI: 80.2%, 92.1%) and specificity of 93.7% (95% CI: 91.1%, 96.3%). The V/Q scan yielded a sensitivity of 96% (95% CI: 95%, 97%) and a specificity of 97% (95% CI: 96%, 98%). Bayes nomogram results for CTA were low risk and yielded a posttest probability of 71.1%, an ADG of 56.1%, and an RDG of 374%, moderate-risk posttest probability was 85.1%, an ADG of 56.1%, and an RDG of 193.4%, and high-risk posttest probability was 95.2%, an ADG of 36.2%, and an RDG of 61.35%. The comparative gain difference for low-risk population was 46.1%; in moderate-risk 41.6%; and in high-risk a 22.1% superiority. ANOVA analysis for LR+ and LR- showed no significant difference (p = 0.8745, p = 0.9841 respectively). CONCLUSIONS: This Bayesian model demonstrated a superiority of CTA when compared to V/Q scan for the diagnosis of pulmonary embolism. Low-risk patients are recognized to have a superior overall comparative gain favoring CTA.

Acute Care Diagnostic Collaboration: Bayesian modeling comparative diagnostic assessment of lactate, procalcitonin and CRP in risk stratified population by Mortality in ED (MEDS) score.

OBJECTIVE: To assess and compare the diagnostic value of lactate, procalcitonin (PCT) and C-reactive protein (CRP) in low, moderate, and high-risk stratified population applying Mortality in Emergency Department (MEDS) risk score using Bayesian statistical modeling. METHODS: MEDS criteria was used to risk stratify into low, moderate and high risk. Each population was attributed a percentage risk, and used as pre-test probability in the Bayesian nomogram. Sensitivity and specificity lactate, PCT and CRP were attained from pooled meta-analysis data. Absolute and relative diagnostic gains were calculated. RESULTS: Pooled diagnostic quality data obtained from a meta-analysis reflected sensitivity for PCT of 77% and specificity of 79%, for lactate sensitivity 49.1% and specificity 74.3% and CRP yielded a sensitivity of 75% and specificity 67%. likelihood ratios (LR) calculations for PCT were LR+ 3.67 and LR- 0.29; for lactate LR+ 1.88 and LR- 0.69; CRP LR+ 2.27 and LR- 0.37. When computed in Bayesian nomogram post-test probabilities for LR+ were as follows: for PCT low risk absolute gain of 11.7% and relative gain of 220%; moderate absolute gain 25.7% relative gain 148.5%; for high risk absolute gain 25.1% and relative gain 42.6%. Lactate LR+ results for low risk absolute gain of 4.7% and relative gain of 88.6%; moderate absolute gain 10.7% and relative gain 61.8%; high risk relative gain 14.1% and relative gain 23.9%. CRP results for low population and LR+ absolute gain 5.7% and relative gain 107.5%; moderate risk 14.7% absolute gain and 84.9% relative gain; high risk 77% post-test 18.1% absolute gain and 30.7% relative gain. CONCLUSION: Bayesian statistical model demonstrated the superior diagnostic quality of PCT. For ruling out severe disease, lactate yielded a higher benefit with increased relative gain with negative LR.

Bayesian comparative assessment of diagnostic accuracy of low-dose CT scan and ultrasonography in the diagnosis of urolithiasis after the application of the STONE score.

OBJECTIVE: The objective of our study was to assess the diagnostic quality of low-dose computed tomography (CT) when compared to ultrasound (US) in diagnosis of urolithiasis using STONE score as a predictor of pre-test probability and the Bayesian statistical model to calculate post-test probabilities (POST) for both diagnostic tests. METHODS: STONE score was used to form risk groups to obtain pre-test probabilities. Likelihood ratios (LR) were calculated from external data for low-dose CT and US. POST were obtained using pre-test probabilities and likelihood ratios with Bayesian nomogram. Absolute (ADG) and relative (RDG) gains in diagnostic value were calculated. RESULTS: Calculated +LR for US was 12 and -LR was 0.32; for CT, +LR was 19 and -LR 0.04. +LR and low STONE for US yielded POST 57% and RDG 470%; intermediate STONE POST 92% and RDG 84%; and high STONE POST 99% and RDG 10%. -LR and low STONE for US POST 3% and RDG -70%; intermediate POST 24% and RDG -52%; and high STONE POST 74% and RDG -17.7%. +LR and low STONE for CT POST 68% and RDG 580%; moderate STONE POST 95% and RDG 90%; and high STONE POST 99% and RDG 10%. -LR and low STONE for CT POST 0% and RDG -100%; intermediate POST 4% and RDG -92%; and high STONE POST 26% and RDG -71.1%. ANOVA calculations comparing CT vs US for +LR showed no statistical significance (P value = 0.9893; LR- P value = 0.5488). CONCLUSION: Bayesian statistical analysis demonstrated slight superiority of CT scan over US on STONE score low- and moderate-risk stratified subtypes, whereas no significant advantage was seen when evaluating high-probability patients.

Bayesian comparative model of CT scan and ultrasonography in the assessment of acute appendicitis: results from the Acute Care Diagnostic Collaboration project.

The objective of this study was to develop a comparative diagnostic model for computed tomography (CT) and ultrasound (US) in the assessment of acute appendicitis using Alvarado risk score as a predictor of pretest probability and Bayesian statistical model as a tool to calculate posttest probabilities for both diagnostic test. Stratification was made by applying the Alvarado score for the prediction of acute appendicitis. Likelihood ratios were calculated using sensitivity and specificity of both CT and US from a Meta-analysis. Posttest probabilities were obtained after inserting Alvarado score and likelihood ratios into Bayesian nomogram. Absolute and relative gains were calculated. ANOVA was used to assess statistical association. 4341 patients from 31 studies yielded a pooled sensitivity and specificity US of 83% (95% CI, 78%-87%) and 93% (95% CI, 90%-96%) and 94% (95% CI, 92%-95%) and 94% (95% CI, 94%-96%), respectively, for CT studies. Positive likelihood ratios (LR) for US were 12 and negative LR was 0.18; for CT +LR was 16 and -LR 0.06. Bayesian statistical modeling posttest probabilities for +LR and low Alvarado risk results yielded a posttest probability for US of 83.72% and 87.27% for CT, intermediate risk gave 95.88% and 96.88%, high risk 99.37% and 99.53 respectively. No statistical differences were found between Ultrasound and CT. This Bayesian analysis demonstrated slight superiority of CT scan over US low-risk patients, whereas no significant advantage was seen when evaluating intermediate and high risk patients. This study also favored elevated accuracy of the Alvarado score.

Acute Care Diagnostics Collaboration: Assessment of a Bayesian clinical decision model integrating the Prehospital Sepsis Score and point-of-care lactate.

UNLABELLED: Previous research demonstrated that shock index and respiratory rate are highly predictive of intensive care unit admissions. OBJECTIVE: The objective of the study is to evaluate the integration of the prehospital sepsis project score (PSP-S) and point-of-care lactate in assisting prediction of severity of illness using Bayesian statistical modeling. METHODS: The PSP-S incorporates fever (38°C [100.4°F]) allotted with 1 point, shock index greater than or equal to 0.7 given 2 points, and a respiratory rate greater than or equal to 22 breaths per minute given 1 point for a total maximum score of 4 points. The patient population was stratified based on the PSP-S: 1 point is low risk, 2 points is moderate risk, and 3 to 4 points is high risk. Percentage risk was obtained based on intensive care unit admissions and used as pretest probability. Prehospital lactate pooled data were obtained and used to calculate likelihood ratio (LR). Percentage risk used as pretest probability and LRs for prehospital lactate were charted into the Bayesian nomogram to obtain posttest probabilities. Absolute diagnostic gain (ADG) and relative diagnostic gains (RDG) were then calculated. RESULTS: Pooled data for prehospital point of care lactate demonstrated a positive LR of 1.6 and negative LR of 0.44. Posttest probability for low risk was 16% with an ADG of 6% and RDG of 160%. Moderate risk population yielded a posttest probability of 47%, ADG of 12.5%, and RDG of 136.2%. High-risk population resulted in a posttest probability of 72%, ADG of 12%, and RDG of 120%. CONCLUSION: We found that PSP-S can be clinically complemented with the use of point-of-care lactate.