Inter-rater agreement in the characterization of cystic renal lesions on contrast-enhanced MRI.

ABSTRACT

PURPOSE: To evaluate inter-observer agreement of MRI features and classification of cystic renal masses among radiologist and radiology trainees. METHODS: Four readers (two radiologists and two radiology trainees) retrospectively reviewed 100 cystic renal lesions on gadolinium enhanced MRI and assigned each a Bosniak classification (1, 2, 2F, 3, and 4). Lesions were also assessed on their individual features including size, presence of nodules, septations, and enhancement. Readers ranked their level of confidence regarding Bosniak classifications. Inter-observer variability of lesion classification and features was evaluated between raters at both radiologist and radiology trainee levels as well as the level of agreement of all four readers using weighted Kappa and intraclass correlation coefficient (ICC). RESULTS: One hundred cystic renal lesions were evaluated. There was moderate and substantial classification agreement between trainees and radiologists (ICC 0.59 and 0.63, respectively). There was substantial classification agreement among all four readers (0.66) with the lowest level of agreement for Bosniak 2F lesions (ICC 0.14). There was moderate-substantial agreement for the presence of nodular component, septations, and enhancement. Staff demonstrated highest agreement when assessing for nodular components (0.73). Agreement for the presence of enhancement was lowest (0.37 and 0.42 for radiologists and trainees, respectively). Reported confidence was higher among radiologists compared with trainees. CONCLUSION: There is substantial overall inter-observer agreement in the MRI classification of cystic renal lesions. Confidence increases as rater experience increases.