Interrater agreement in the evaluation of discrepant imaging findings with the Radpeer system.

OBJECTIVE: The Radpeer system is central to the quality assurance process in many radiology practices. Previous studies have shown poor agreement between physicians in the evaluation of their peers. The purpose of this study was to assess the reliability of the Radpeer scoring system. MATERIALS AND METHODS: A sample of 25 discrepant cases was extracted from our quality assurance database. Images were made anonymous; associated reports and identities of interpreting radiologists were removed. Indications for the studies and descriptions of the discrepancies were provided. Twenty-one subspecialist attending radiologists rated the cases using the Radpeer scoring system. Multirater kappa statistics were used to assess interrater agreement, both with the standard scoring system and with dichotomized scores to reflect the practice of further review for cases rated 3 and 4. Subgroup analyses were conducted to assess subspecialist evaluation of cases. RESULTS: Interrater agreement was slight to fair compared with that expected by chance. For the group of 21 raters, the kappa values were 0.11 (95% CI, 0.06-0.16) with the standard scoring system and 0.20 (95% CI, 0.13-0.27) with dichotomized scores. There was disagreement about whether a discrepancy had occurred in 20 cases. Subgroup analyses did not reveal significant differences in the degree of interrater agreement. CONCLUSION: The identification of discrepant interpretations is valuable for the education of individual radiologists and for larger-scale quality assurance and quality improvement efforts. Our results show that a ratings-based peer review system is unreliable and subjective for the evaluation of discrepant interpretations. Resources should be devoted to developing more robust and objective assessment procedures, particularly those with clear quality improvement goals.

Positron Emission Mammography Image Interpretation for Reduced Image Count Levels.

UNLABELLED: We studied the effects of reduced (18)F-FDG injection activity on interpretation of positron emission mammography (PEM) images and compared image interpretation between 2 postinjection imaging times. METHODS: We performed a receiver-operating-characteristic (ROC) study using PEM images reconstructed with different count levels expected from injected activities between 23 and 185 MBq. Thirty patients received 2 PEM scans at postinjection times of 60 and 120 min. Half of the patients were scanned with a standard protocol; the others received one-half of the standard activity. Images were reconstructed using 100%, 50%, and 25% of the total counts acquired. Eight radiologists used a 5-point confidence scale to score 232 PEM images for the presence of up to 3 malignant lesions. Paired images were analyzed with conditional logistic regression and ROC analysis to investigate changes in interpretation. RESULTS: There was a trend for increasing lesion detection sensitivity with increased image counts: odds ratios were 2.2 (P = 0.01) and 1.9 (P = 0.04) per doubling of image counts for 60- and 120-min uptake images, respectively, without significant difference between time points (P = 0.7). The area under the ROC curve (AUC) was highest for the 100%-count, 60-min images (0.83 vs. 0.75 for 50%-counts, P = 0.02). The 120-min images had a similar trend but did not reach statistical significance (AUC = 0.79 vs. 0.73, P = 0.1). Our data did not yield significant trends between specificity and image counts. Lesion-to-background ratios increased between 60- and 120-min scans (P < 0.001). CONCLUSION: Reducing the image counts relative to the standard protocol decreased diagnostic accuracy. The increase in lesion-to-background ratio between 60- and 120-min uptake times was not enough to improve detection sensitivity in this study, perhaps in part due to fewer counts in the later scan.

The impacted ureteral stone.

Urinary tract calculi and infections are common causes of presentation to the emergency department. Computed tomography kidney-ureter-bladder is the initial imaging study of choice in patients presenting with symptoms of urinary tract calculi. As clinical evidence of superimposed infection can be atypical or absent, it is crucial to identify subtle imaging findings that suggest this complication. We report here a case of acute ureterolithiasis with missed evidence of superimposed infection on the initial computed tomography kidney-ureter-bladder. The patient subsequently developed urosepsis complicated by critical limb ischemia requiring amputation of all 4 extremities. The case became the topic of a malpractice suit.

Anonymous group peer review in surgery morbidity and mortality conference.

BACKGROUND: Surgical peer review might be characterized by assessment heterogeneity. METHODS: We performed a prospective, anonymous, peer review of surgeon and system performance during a morbidity and mortality conference. RESULTS: Twenty-two cases were reviewed by a mean of 48.9 respondents each, including attendings, fellows, and residents. A mean of 50% (standard deviation, 23%) of respondents identified some quality issue in each case, reflecting high heterogeneity. The mean percentage in identifying a system issue was 27%, and in identifying a physician issue was 37%. When identifying a physician issue, physician care was judged as appropriate by 72%, as controversial by 26%, or as inappropriate by 2%. Residents were more likely than attendings to identify system issues (odds ratio, 2.23) and physician issues (odds ratio, 3.58), but attendings were more likely to rate care controversial or inappropriate (odds ratio, 2.53). CONCLUSIONS: Surgical peer reviews, even after group discussion, display substantial heterogeneity. Review methods should account for this heterogeneity.