The average receiver operating characteristic curve in multireader multicase imaging studies.

OBJECTIVE: In multireader, multicase (MRMC) receiver operating characteristic (ROC) studies for evaluating medical imaging systems, the area under the ROC curve (AUC) is often used as a summary metric. Owing to the limitations of AUC, plotting the average ROC curve to accompany the rigorous statistical inference on AUC is recommended. The objective of this article is to investigate methods for generating the average ROC curve from ROC curves of individual readers. METHODS: We present both a non-parametric method and a parametric method for averaging ROC curves that produce a ROC curve, the area under which is equal to the average AUC of individual readers (a property we call area preserving). We use hypothetical examples, simulated data and a real-world imaging data set to illustrate these methods and their properties. RESULTS: We show that our proposed methods are area preserving. We also show that the method of averaging the ROC parameters, either the conventional bi-normal parameters (a, b) or the proper bi-normal parameters (c, da), is generally not area preserving and may produce a ROC curve that is intuitively not an average of multiple curves. CONCLUSION: Our proposed methods are useful for making plots of average ROC curves in MRMC studies as a companion to the rigorous statistical inference on the AUC end point. The software implementing these methods is freely available from the authors. ADVANCES IN KNOWLEDGE: METHODS for generating the average ROC curve in MRMC ROC studies are formally investigated. The area-preserving criterion we defined is useful to evaluate such methods.

Evidence for TeV gamma-ray emission from a region of the galactic plane.

Gamma-ray emission from a narrow band at the galactic equator has previously been detected up to 30 GeV. We report evidence for a TeV gamma-ray signal from a region of the galactic plane by Milagro, a large-field-of-view water Cherenkov detector for extensive air showers. An excess with a significance of 4.5 standard deviations has been observed from the region of galactic longitude l E (40 degrees, 100 degrees) and latitude /b/ < 5 degrees. Under the assumption of a simple power law spectrum, with no cutoff in the EGRET-Milagro energy range, the measured integral flux is phi gamma(>3.5 TeV) = (6.4 +/- 1.4 +/- 2.1) x 10(-11) cm(-2) s(-1) sr(-1). This flux is consistent with an extrapolation of the EGRET spectrum between 1 and 30 GeV in this galactic region.