Effect of Prostate MRI Interpretation Experience on PPV Using PI-RADS Version 2: A 6-Year Assessment Among Eight Fellowship-Trained Radiologists.

BACKGROUND. Understanding the effect of specific experience in prostate MRI interpretation on diagnostic performance would help inform the minimum interpretation volume to establish proficiency. OBJECTIVE. The purpose of this article is to assess for an association between increasing experience in prostate MRI interpretation and change in radiologist-level PPVs for PI-RADS version 2 (v2) categories 3, 4, and 5. METHODS. This retrospective study included prostate MRI examinations performed between July 1, 2015, and August 13, 2021, that were assigned a PI-RADS v2 category of 3, 4, or 5 and with an MRI-ultrasound fusion biopsy available as the reference standard. All examinations were among the first 100-200 prostate MRI examinations interpreted using PI-RADS v2 by fellowship-trained abdominal radiologists. Radiologists received feedback through a quality assurance program. Radiologists' experience levels were classified using progressive subsets of 50 interpreted examinations. Change with increasing experience in distribution of individual radiologists' whole-gland PPVs for Gleason sum score 7 or greater prostate cancer, stratified by PI-RADS category, was assessed by hierarchic linear mixed models. RESULTS. The study included 1300 prostate MRI examinations in 1037 patients (mean age, 66 ± 7 [SD] years), interpreted by eight radiologists (median, 13 years of postfellow-ship experience; range, 5-22 years). Aggregate PPVs were 20% (68/340) for PI-RADS category 3, 49% (318/652) for category 4, and 71% (220/308) for category 5. Interquartile ranges (IQRs) of PPVs overlapped for category 4 (51%; IQR, 42-60%) and category 5 (70%; IQR, 54-75%) for radiologists' first 50 examinations. IQRs of PPVs did not overlap between categories of greater experience; for example, at the 101-150 examination level, PPV for category 3 was 24% (IQR, 20-29%), category 4 was 55% (IQR, 54-63%), and category 5 was 81% (IQR, 77-82%). Hierarchic modeling showed no change in radiologists' absolute PPV with increasing experience (category 3, p = .27; category 4, p = .71; category 5, p = .38). CONCLUSION. Absolute PPVs at specific PI-RADS categories did not change during radiologists' first 200 included examinations. However, resolution of initial overlap in IQRs indicates improved precision of PPVs after the first 50 examinations. CLINICAL IMPACT. If implementing a minimum training threshold for fellowship-trained abdominal radiologists, 50 prostate MRI examinations may be sufficient in the context of a quality assurance program with feedback.

Prostate Imaging and Data Reporting System Version 2 as a Radiology Performance Metric: An Analysis of 18 Abdominal Radiologists.

PURPOSE: To determine expected trained provider performance dispersion in Prostate Imaging and Data Reporting System version 2 (PI-RADS v2) positive predictive values (PPVs). METHODS: This single-center quality assurance retrospective cohort study evaluated 5,556 consecutive prostate MRIs performed on 4,593 patients. Studies were prospectively interpreted from October 8, 2016, to July 31, 2020, by 18 subspecialty-trained abdominal radiologists (1-22 years' experience; median MRIs per radiologist: 232, first-to-third quartile range [Q1-Q3]: 128-440; 13 interpreted at least 30 MRIs with a reference standard). Maximum prospectively reported whole-gland PI-RADS v2 score was compared to post-MRI biopsy histopathology obtained within 2 years. The primary outcome was PPV of MRI by provider stratified by maximum whole-gland PI-RADS v2 score. RESULTS: Median provider-level PPVs for the radiologists who interpreted ≥30 MRIs with a reference standard were PI-RADS 3 (22.1%; Q1-Q3: 10.0%-28.6%), PI-RADS 4 (49.2%; Q1-Q3: 41.4%-50.0%), PI-RADS 5 (81.8%; Q1-Q3: 77.1%-84.4%). Overall, the maximum whole-gland PI-RADS v2 score was PI-RADS 1 to 2 (34.6% [1,925]), PI-RADS 3 (8.5% [474]), PI-RADS 4 (21.0% [1,166]), PI-RADS 5 (18.3% [1,018]), no PI-RADS score (17.5% [973]). System-level (all providers) PPVs for maximum PI-RADS v2 scores were 20.0% (95% confidence interval [CI]: 15.7%-24.9%) for PI-RADS 3, 48.5% (95% CI: 44.8%-52.2%) for PI-RADS 4, and 80.1% for PI-RADS 5 (95% CI: 75.7%-83.9%). CONCLUSION: Subspecialty-trained abdominal radiologists with a wide range of experience can obtain consistent positive predictive values for PI-RADS v2 scores of 3 to 5. These data can be used for quality assurance benchmarking.