Radiologists' Contribution to Variation in Detecting Clinically Significant Prostate Cancer in Men With Prostate MRI.

OBJECTIVE: Assess radiologists' contribution to variation in clinically significant prostate cancer (csPCa) detection in patients with elevated prostate-specific antigen (PSA) and multiparametric MRI (mpMRI). METHODS: This institutional review board-approved, retrospective cohort study was performed at a tertiary, academic, National Cancer Institute-designated Comprehensive Cancer Center with a multidisciplinary prostate cancer program. Men undergoing mpMRI examinations from January 1, 2015, to December 31, 2019, with elevated PSA (≥4 ng/mL) and biopsy within 6 months pre- or post-MRI or prostatectomy within 6 months post-mpMRI were included. Univariate and multivariable hierarchical logistic regression assessed impact of patient, provider, mpMRI examination, mpMRI report, and pathology factors on the diagnosis of Grade Group ≥ 2 csPCa. RESULTS: Study cohort included 960 MRIs in 928 men, mean age 64.0 years (SD ± 7.4), and 59.8% (555 of 928) had csPCa. Interpreting radiologist was not significant individually (P > .999) or combined with mpMRI ordering physician and physician performing biopsy or prostatectomy (P = .41). Prostate Imaging Reporting and Data System (PI-RADS) category 2 (odds ratio [OR] 0.18, P = .04), PI-RADS category 4 (OR 2.52, P < .001), and PI-RADS category 5 (OR 4.99, P < .001) assessment compared with no focal lesion; PSA density of 0.1 to 0.15 ng/mL/cc (OR 2.46, P < .001), 0.15 to 0.2 ng/mL/cc (OR 2.77, P < .001), or ≥0.2 ng/mL/cc (OR 4.52, P < .001); private insurance (reference = Medicare, OR 0.52, P = .001), and unambiguous extraprostatic extension on mpMRI (OR 2.94, P = .01) were independently associated with csPCa. PI-RADS 3 assessment (OR 1.18, P = .56), age (OR 0.99, P = .39), and African American race (OR 0.90, P = .75) were not. DISCUSSION: Although there is known in-practice variation in radiologists' interpretation of mpMRI, in our multidisciplinary prostate cancer program we found no significant radiologist-attributable variation in csPCa detection.

Assessment of a Large-Scale Peer Learning Program's Value by Manual Review of Case Submissions.

OBJECTIVE: Prior studies used submission numbers or report addendum rates to measure peer learning programs' (PLP) impact. We assessed the educational value of a PLP by manually reviewing cases submitted to identify factors correlating with meaningful learning opportunities (MLOs). METHODS: This institutional review board-exempted, retrospective study was performed in a large academic radiology department generating >800,000 reports annually. A PLP facilitating radiologist-to-radiologist feedback was implemented May 1, 2017, with subsequent pay-for-performance initiatives encouraging increasing submissions, >18,000 by 2019. Two radiologists blinded to submitter and receiver identity categorized 336 randomly selected submissions as a MLO, not meaningful, or equivocal, resolving disagreements in consensus review. Primary outcome was proportion of MLOs. Secondary outcomes included percent engagement by subspecialty clinical division and comparing MLO and report addendum rates via Fisher's exact tests. We assessed association between peer learning category, pay-for-performance interventions, and subspecialty division with MLOs using logistic regression. RESULTS: Of 336 PLP submissions, 65.2% (219 of 336) were categorized as meaningful, 27.4% (92 of 336) not meaningful, and 7.4% (25 of 336) equivocal, with substantial reviewer agreement (86.0% [289 of 336], κ = 0.71, 95% confidence interval 0.64-0.78). MLO rate (65.2% [219 of 336]) was five times higher than addendum rate (12.9% [43 of 333]) for the cohort. MLO proportion (adjusted odds ratios 0.05-1.09) and percent engagement (0.5%-3.6%) varied between subspecialty divisions, some submitting significantly fewer MLOs (P < .01). MLO proportion did not vary between peer learning categories. CONCLUSION: Educational value of a large-scale PLP, estimated through manual review of case submissions, is likely a more accurate measure of program impact. Incentives to enhance PLP use did not diminish the program's educational value.

Peer Learning in Radiology: Effect of a Pay-for-Performance Initiative on Clinical Impact and Usage.

OBJECTIVE. The purpose of this article is to assess the effects of a pay-for-performance (PFP) initiative on clinical impact and usage of a radiology peer learning tool. MATERIALS AND METHODS. This retrospective study was performed at a large academic hospital. On May 1, 2017, a peer learning tool was implemented to facilitate radiologist peer feedback including clinical follow-up, positive feedback, and consultation. Subsequently, PFP target numbers for peer learning tool alerts by subspecialty divisions (October 1, 2017) and individual radiologists (October 1, 2018) were set. The primary outcome was report addendum rate (percent of clinical follow-up alerts with addenda), which was a proxy for peer learning tool clinical impact. Secondary outcomes were peer learning tool usage rate (number of peer learning tool alerts per 1000 radiology reports) and proportion of clinical follow-up alerts (percent of clinical follow-ups among all peer learning tool alerts). Outcomes were assessed biweekly using ANOVA and statistical process control analyses. RESULTS. Among 1,265,839 radiology reports from May 1, 2017, to September 29, 2019, a total of 20,902 peer learning tool alerts were generated. The clinical follow-up alert addendum rate was not significantly different between the period before the PFP initiative (9.9%) and the periods including division-wide (8.3%) and individual (7.9%) PFP initiatives (p = .55; ANOVA). Peer learning tool usage increased from 2.2 alerts per 1000 reports before the PFP initiative to 12.6 per 1000 during the division-wide PFP period (5.7-fold increase; 12.6/2.2), to 25.2 in the individual PFP period (11.5-fold increase vs before PFP; twofold increase vs division-wide) (p < .001). The clinical follow-up alert proportion decreased from 37.5% before the PFP initiative, to 34.4% in the division-wide period, to 31.3% in the individual PFP period. CONCLUSION. A PFP initiative improved radiologist engagement in peer learning by marked increase in peer learning tool usage rate without a change in report addendum rate as a proxy for clinical impact.

Radiology Patient Outcome Measures: Impact of a Departmental Pay-for-Performance Initiative on Key Quality and Safety Measures.

OBJECTIVE: Assess impact of a multifaceted pay-for-performance (PFP) initiative on radiologists' behavior regarding key quality and safety measures. METHODS: This institutional review board-approved prospective study was performed at a large, 12-division urban academic radiology department. Radiology patient outcome measures were implemented October 1, 2017, measuring report signature timeliness, critical results communication, and generation of peer-learning communications between radiologists. Subspecialty division-wide and individual radiologist targets were specified, performance was transparently communicated on an intranet dashboard updated daily, and performance was financially incentivized (5% of salary) quarterly. We compared outcomes 12 months pre- versus 12 months post-PFP implementation. Primary outcome was monthly 90th percentile time from scan completion to final report signature (CtoF). Secondary outcomes were percentage timely closed-loop communication of critical results and number of division-wide peer-learning communications. Statistical process control analysis and parallel coordinates charts were used to assess for temporal trends. RESULTS: In all, 144 radiologists generated 1,255,771 reports (613,273 pre-PFP) during the study period. Monthly 90th percentile CtoF exhibited an absolute decrease of 4.4 hours (from 21.1 to 16.7 hours) and a 20.9% relative decrease post-PFP. Statistical process control analysis demonstrated significant decreases in 90th percentile CtoF post-PFP, sustained throughout the study period (P < .003). Between 95% (119 of 125, July 1, 2018, to September 30, 2018) and 98.4% (126 of 128, October 1, 2017, to December 31, 2017) of radiologists achieved >90% timely closure of critical alerts; all divisions exceeded the target of 90 peer-learning communications each quarter (range: 97-472) after January 1, 2018. DISCUSSION: Implementation of a multifaceted PFP initiative using well-defined radiology patient outcome measures correlated with measurable improvements in radiologist behavior regarding key quality and safety parameters.