Objective determination of standard of care: use of blind readings by external radiologists.

OBJECTIVE: The purpose of this study was to determine whether specific findings determined to be critical and standard of care by expert witnesses in a legal case are identifiable by radiologists blinded to clinical outcome and litigation. SUBJECTS AND METHODS: Images from six CT studies were sent to radiologists for interpretation. Two studies were performed for screening after major trauma, one of the cases being the subject of a settled legal action; three were randomly selected from studies performed in the evaluation of emergency department patients; and one was the control. The cases were selected to simulate a typical emergency department caseload. In the medicolegal case, four plaintiff expert witness radiologists had identified three findings in the CT study that were not described by the radiologist of record (primary reader). One of these findings was considered critical and was the basis for the legal case. RESULTS: Thirty-one radiologists participated in the study. The three findings made by the expert witnesses-T3 and T10 vertebral body fractures and 1-mm symmetric widening of the facet joints at T10-were made by none, 19 (61.3%), and none of the 31 radiologists in this study. CONCLUSION: Thirty-one radiologists who had no knowledge of the clinical outcome or litigation did not confirm the expert witness interpretation. This finding prompts questions about the current method of determining standard of care in legal cases, that is, use of paid medical expert witnesses. Our findings suggest that use of radiologists blinded to clinical outcome may be a more objective method of evaluating legal cases.

Evaluation of radiologist interpretive performance using blinded reads by multiple external readers.

OBJECTIVE: To demonstrate a method of assessing radiologic diagnostic agreement utilizing multiple blinded external readers. METHODS: Six body CT studies interpreted by one reader (primary reader) at the host institution were compiled with patient identifiers removed. Brief clinical histories that were available to the primary reader were provided. Radiologists at 22 centers participated and the interpretations were analyzed in aggregate with the consensus majority that served as the surrogate gold standard for each case. RESULTS: A total of 31 radiologists formed the group of secondary readers with two-thirds in academic practice averaging 8 years of experience (range: 1-25 years). The average findings per reader for cases A to F include: 1.9 (range: 1-5), 6.3 (range: 2-10), 10.4 (range: 7-14), 5.7 (range: 3-10), 4.2 (range: 2-8), and 3.8 (range: 1-7), respectively. There was agreement of the primary interpretation and the surrogate gold standard for each case. CONCLUSIONS: The results of our study demonstrate a wide range of interpretation, with wider ranges observed in more complex cases and with vague clinical complaints. Comparison to the primary reader required the use of aggregate analysis and an agreement percentage cutoff to minimize bias and the limitations of this type of study. An intensive evaluation of radiologist performance such as this could be considered in various settings such as a quality assurance program, intense scrutiny of an individual radiologist whether competency is in question, or for medicolegal purpose to ascertain standard of care.