Time-varying risks of second events following a DCIS diagnosis in the population-based Vermont DCIS cohort.

PURPOSE: Long-term disease-free survival patterns following surgical, radiation, and endocrine therapy treatments for ductal carcinoma in situ (DCIS) are not well characterized in general US practice. METHODS: We identified 1252 women diagnosed with DCIS in Vermont during 1994-2012 using data from the Vermont Breast Cancer Surveillance System, a statewide registry of breast imaging and pathology records. Poisson regression and Cox regression with time-varying hazards were used to evaluate disease-free survival among self-selected treatment groups. RESULTS: With 7.8 years median follow-up, 192 cases experienced a second breast cancer diagnosis. For women treated with breast-conserving surgery (BCS) alone, the annual rate of second events decreased from 3.1% (95% CI 2.2-4.2%) during follow-up years 1-5 to 1.7% (95% CI 0.7-3.5%) after 10 years. In contrast, the annual rate of second events among women treated with BCS plus adjuvant radiation therapy increased from 1.8% (95% CI 1.1-2.6%) during years 1-5 to 2.8% (95% CI 1.6-4.7%) after 10 years (P < 0.05 for difference in trend compared to BCS alone). Annual rates of second events also increased over time among women treated with BCS plus adjuvant radiation and endocrine therapy (P = 0.01 for difference in trend compared to BCS alone). The rate of contralateral events increased after 10 years for all groups with adjuvant treatments. The rate of second events did not vary over time among women who underwent ipsilateral mastectomy (P = 0.62). CONCLUSIONS: Long-term risk of a second event after DCIS varies over time in a manner dependent on initial treatment.

Pathologists' Use of Second Opinions in Interpretation of Melanocytic Cutaneous Lesions: Policies, Practices, and Perceptions.

BACKGROUND: Research examining the role of second opinions in pathology for diagnosis of melanocytic lesions is limited. OBJECTIVE: To assess current laboratory policies, clinical use of second opinions, and pathologists' perceptions of second opinions for melanocytic lesions. MATERIALS AND METHODS: Cross-sectional data collected from 207 pathologists in 10 US states who diagnose melanocytic lesions. The web-based survey ascertained pathologists' professional information, laboratory second opinion policy, use of second opinions, and perceptions of second opinion value for melanocytic lesions. RESULTS: Laboratory policies required second opinions for 31% of pathologists and most commonly required for melanoma in situ (26%) and invasive melanoma (30%). In practice, most pathologists reported requesting second opinions for melanocytic tumors of uncertain malignant potential (85%) and atypical Spitzoid lesions (88%). Most pathologists perceived that second opinions increased interpretive accuracy (78%) and protected them from malpractice lawsuits (62%). CONCLUSION: Use of second opinions in clinical practice is greater than that required by laboratory policies, especially for melanocytic tumors of uncertain malignant potential and atypical Spitzoid lesions. Quality of care in surgical interventions for atypical melanocytic proliferations critically depends on the accuracy of diagnosis in pathology reporting. Future research should examine the extent to which second opinions improve accuracy of melanocytic lesion diagnosis.

Diagnostic Reproducibility: What Happens When the Same Pathologist Interprets the Same Breast Biopsy Specimen at Two Points in Time?

BACKGROUND: Surgeons may receive a different diagnosis when a breast biopsy is interpreted by a second pathologist. The extent to which diagnostic agreement by the same pathologist varies at two time points is unknown. METHODS: Pathologists from eight U.S. states independently interpreted 60 breast specimens, one glass slide per case, on two occasions separated by ≥9 months. Reproducibility was assessed by comparing interpretations between the two time points; associations between reproducibility (intraobserver agreement rates); and characteristics of pathologists and cases were determined and also compared with interobserver agreement of baseline interpretations. RESULTS: Sixty-five percent of invited, responding pathologists were eligible and consented; 49 interpreted glass slides in both study phases, resulting in 2940 interpretations. Intraobserver agreement rates between the two phases were 92% [95% confidence interval (CI) 88-95] for invasive breast cancer, 84% (95% CI 81-87) for ductal carcinoma-in-situ, 53% (95% CI 47-59) for atypia, and 84% (95% CI 81-86) for benign without atypia. When comparing all study participants' case interpretations at baseline, interobserver agreement rates were 89% (95% CI 84-92) for invasive cancer, 79% (95% CI 76-81) for ductal carcinoma-in-situ, 43% (95% CI 41-45) for atypia, and 77% (95% CI 74-79) for benign without atypia. CONCLUSIONS: Interpretive agreement between two time points by the same individual pathologist was low for atypia and was similar to observed rates of agreement for atypia between different pathologists. Physicians and patients should be aware of the diagnostic challenges associated with a breast biopsy diagnosis of atypia when considering treatment and surveillance decisions.

Ages at menarche- and menopause-related genetic variants in relation to terminal duct lobular unit involution in normal breast tissue.

Reduced levels of terminal duct lobular unit (TDLU) involution, as reflected by higher numbers of TDLUs and acini per TDLU, have been associated with higher breast cancer risk. Younger age at menarche and older age at menopause have been previously related to lower levels of TDLU involution. To determine a possible genetic link, we examined whether single-nucleotide polymorphisms (SNPs) previously established in genome-wide association studies (GWAS) for ages at menarche and menopause are associated with TDLU involution. We conducted a pooled analysis of 862 women from two studies. H&E tissue sections were assessed for numbers of TDLUs and acini/TDLU. Poisson regression models were used to estimate associations of 36 menarche- and 21 menopause-SNPs with TDLU counts, acini counts/TDLU, and the product of these two measures, adjusting for age and study site. Fourteen percent of evaluated SNPs (eight SNPs) were associated with TDLU counts at p < 0.05, suggesting an enrichment of associations with TDLU counts. However, only menopause-SNPs had >50 % that were either significantly or nonsignificantly associated with TDLU measures in the directions consistent with their relationships shown in GWAS. Among ten SNPs that were statistically significantly associated with at least one TDLU involution measure (p < 0.05), seven SNPs (rs466639: RXRG; rs2243803: SLC14A2; rs2292573: GAB2; rs6438424: 3q13.32; rs7606918: METAP1D; rs11668344: TMEM150B; rs1635501: EXO1) were associated in the consistent directions. Our data suggest that the loci associated with ages at menarche and menopause may influence TDLU involution, suggesting some shared genetic mechanisms. However, larger studies are needed to confirm the results.

Identifying and processing the gap between perceived and actual agreement in breast pathology interpretation.

We examined how pathologists' process their perceptions of how their interpretations on diagnoses for breast pathology cases agree with a reference standard. To accomplish this, we created an individualized self-directed continuing medical education program that showed pathologists interpreting breast specimens how their interpretations on a test set compared with a reference diagnosis developed by a consensus panel of experienced breast pathologists. After interpreting a test set of 60 cases, 92 participating pathologists were asked to estimate how their interpretations compared with the standard for benign without atypia, atypia, ductal carcinoma in situ and invasive cancer. We then asked pathologists their thoughts about learning about differences in their perceptions compared with actual agreement. Overall, participants tended to overestimate their agreement with the reference standard, with a mean difference of 5.5% (75.9% actual agreement; 81.4% estimated agreement), especially for atypia and were least likely to overestimate it for invasive breast cancer. Non-academic affiliated pathologists were more likely to more closely estimate their performance relative to academic affiliated pathologists (77.6 vs 48%; P=0.001), whereas participants affiliated with an academic medical center were more likely to underestimate agreement with their diagnoses compared with non-academic affiliated pathologists (40 vs 6%). Before the continuing medical education program, nearly 55% (54.9%) of participants could not estimate whether they would overinterpret the cases or underinterpret them relative to the reference diagnosis. Nearly 80% (79.8%) reported learning new information from this individualized web-based continuing medical education program, and 23.9% of pathologists identified strategies they would change their practice to improve. In conclusion, when evaluating breast pathology specimens, pathologists do a good job of estimating their diagnostic agreement with a reference standard, but for atypia cases, pathologists tend to overestimate diagnostic agreement. Many participants were able to identify ways to improve.

How concerns and experiences with medical malpractice affect dermatopathologists' perceptions of their diagnostic practices when interpreting cutaneous melanocytic lesions.

OBJECTIVE: We sought to identify characteristics associated with past malpractice lawsuits and how malpractice concerns may affect interpretive practices. METHODS: We surveyed 207 of 301 (68.8%) eligible dermatopathologists who interpret melanocytic skin lesions in 10 states. The survey assessed dermatopathologists' demographic and clinical practice characteristics, perceptions of how medical malpractice concerns could influence their interpretive practices, and past malpractice lawsuits. RESULTS: Of dermatopathologists, 33% reported past malpractice experiences. Factors associated with being sued included older age (57 vs 48 years, P < .001), lack of board certification or fellowship training in dermatopathology (76.5% vs 53.2%, P = .001), and greater number of years interpreting melanocytic lesions (>20 years: 52.9% vs 20.1%, P < .001). Of participants, 64% reported being moderately or extremely confident in their melanocytic interpretations. Although most dermatopathologists believed that malpractice concerns increased their likelihood of ordering specialized pathology tests, obtaining recuts, and seeking a second opinion, none of these practices were associated with past malpractice. Most dermatopathologists reported concerns about potential harms to patients that may result from their assessments of melanocytic lesions. LIMITATIONS: Limitations of this study include lack of validation of and details about the malpractice suits experienced by participating dermatopathologists. In addition, the study assessed perceptions of practice rather than actual practices that might be associated with malpractice incidents. CONCLUSIONS: Most dermatopathologists reported apprehension about how malpractice affects their clinical practice and are concerned about patient safety irrespective of whether they had actually experienced a medical malpractice suit.

Achieving consensus for the histopathologic diagnosis of melanocytic lesions: use of the modified Delphi method.

OBJECTIVE: To understand the sophisticated nature of coming to consensus when diagnosing complex melanocytic lesions among a panel of experienced dermatopathologists. METHODS: A total of 240 melanocytic lesions were assessed independently by three experienced dermatopathologists with their diagnoses mapped into one of five Melanocytic Pathology Assessment Tool and Hierarchy for Diagnosis (MPATH-DX) categories: (I) nevus/mild atypia, (II) moderate atypia, (III) severe atypia/melanoma in situ, (IV) T1a invasive melanoma and (V) ≥ T1b invasive melanoma. The dermatopathologists then discussed the cases, using a modified Delphi method to facilitated consensus building for cases with discordant diagnoses. RESULTS: For most cases, a majority of interpretations (two or three of three) agreed with the consensus diagnosis in 95% of Category I, 64% of Category II, 84% of Category III, 88% for Category IV and 100% of Category V cases. Disagreements were typically due to diagnostic threshold differences (64.5%), differing contents on slides even though the slides were sequential cuts (18.5%), and missed findings (15.3%). Disagreements were resolved via discussion of histopathologic features and their significance while reviewing the slides using a multi-headed microscope, considering treatment recommendations, citing existing literature, reviewing additional slides for a case, and choosing a provisional/borderline diagnosis to capture diverse opinions. All experienced pathologists participating in this study reported that the process of coming to consensus was challenging for borderline cases and may have represented compromise rather than consensus. They also reported the process changed their approaches to diagnosing complex melanocytic lesions. CONCLUSIONS: The most frequent reason for disagreement of experienced dermatopathologists was differences in diagnostic thresholds related to observer viewpoints. A range of approaches was needed to come to consensus, and this may guide pathology groups who do not currently hold consensus conferences.

Use of Digital Whole Slide Imaging in Dermatopathology.

Digital whole slide imaging (WSI) is an emerging technology for pathology interpretation, with specific challenges for dermatopathology, yet little is known about pathologists' practice patterns or perceptions regarding WSI for interpretation of melanocytic lesions. A national sample of pathologists (N = 207) was recruited from 864 invited pathologists from ten US states (CA, CT, HI, IA, KY, LA, NJ, NM, UT, and WA). Pathologists who had interpreted melanocytic lesions in the past year were surveyed in this cross-sectional study. The survey included questions on pathologists' experience, WSI practice patterns and perceptions using a 6-point Likert scale. Agreement was summarized with descriptive statistics to characterize pathologists' use and perceptions of WSI. The majority of participating pathologists were between 40 and 59 years of age (62%) and not affiliated with an academic medical center (71%). Use of WSI was seen more often among dermatopathologists and participants affiliated with an academic medical center. Experience with WSI was reported by 41%, with the most common type of use being for education and testing (CME, board exams, and teaching in general, 71%), and clinical use at tumor boards and conferences (44%). Most respondents (77%) agreed that accurate diagnoses can be made with this technology, and 59% agreed that benefits of WSI outweigh concerns. However, 78% of pathologists reported that digital slides are too slow for routine clinical interpretation. The respondents were equally split as to whether they would like to adopt WSI (49%) or not (51%). The majority of pathologists who interpret melanocytic lesions do not use WSI, but among pathologists who do, use is largely for CME, licensure/board exams, and teaching. Positive perceptions regarding WSI slightly outweigh negative perceptions. Understanding practice patterns with WSI as dissemination advances may facilitate concordance of perceptions with adoption of the technology.

Trends in breast biopsy pathology diagnoses among women undergoing mammography in the United States: a report from the Breast Cancer Surveillance Consortium.

BACKGROUND: Current data on the pathologic diagnoses of breast biopsy after mammography can inform patients, clinicians, and researchers about important population trends. METHODS: Breast Cancer Surveillance Consortium data on 4,020,140 mammograms between 1996 and 2008 were linked to 76,567 pathology specimens. Trends in diagnoses in biopsies by time and risk factors (patient age, breast density, and family history of breast cancer) were examined for screening and diagnostic mammography (performed for a breast symptom or short-interval follow-up). RESULTS: Of the total mammograms, 88.5% were screening and 11.5% diagnostic; 1.2% of screening and 6.8% of diagnostic mammograms were followed by biopsies. The frequency of biopsies over time was stable after screening mammograms, but increased after diagnostic mammograms. For biopsies obtained after screening, frequencies of invasive carcinoma increased over time for women ages 40-49 and 60-69, Ductal carcinoma in situ (DCIS) increased for those ages 40-69, whereas benign diagnoses decreased for all ages. No trends in pathology diagnoses were found following diagnostic mammograms. Dense breast tissue was associated with high-risk lesions and DCIS relative to nondense breast tissue. Family history of breast cancer was associated with DCIS and invasive cancer. CONCLUSIONS: Although the frequency of breast biopsy after screening mammography has not changed over time, the percentages of biopsies with DCIS and invasive cancer diagnoses have increased. Among biopsies following mammography, women with dense breasts or family history of breast cancer were more likely to have high-risk lesions or invasive cancer. These findings are relevant to breast cancer screening and diagnostic practices.

Breast cancer detection with short-interval follow-up compared with return to annual screening in patients with benign stereotactic or US-guided breast biopsy results.

PURPOSE: To compare the cancer detection rate and stage after benign stereotactic or ultrasonography (US)-guided core breast biopsy between patients with short-interval follow-up (SIFU) and those who return to annual screening. MATERIALS AND METHODS: The Breast Cancer Surveillance Consortium (BCSC) registry and the BCSC Statistical Coordinating Center received institutional review board approval for active and passive consent processes and a waiver of consent. All procedures were HIPAA compliant. BCSC data for 1994-2010 were used to compare ipsilateral breast cancer detection rates and tumor characteristics for diagnoses within 3 months after SIFU (3-8 months) versus return to annual screening (RTAS) mammography (9-18 months) after receiving a benign pathology result from image-guided breast biopsy. RESULTS: In total, 17 631 biopsies with benign findings were identified with SIFU or RTAS imaging. In the SIFU group, 27 ipsilateral breast cancers were diagnosed in 10 715 mammographic examinations (2.5 cancers per 1000 examinations) compared with 16 cancers in 6916 mammographic examinations in the RTAS group (2.3 cancers per 1000 examinations) (P = .88). Sixteen cancers after SIFU (59%; 95% confidence interval [CI]: 39%, 78%) were invasive versus 12 after RTAS (75%; 95% CI: 48%, 93%). The invasive cancer rate was 1.5 per 1000 examinations after SIFU (95% CI: 0.9, 2.4) and 1.7 per 1000 examinations (95% CI: 0.9, 3.0) after RTAS (P = .70). Among invasive cancers, 25% were late stage (stage 2B, 3, or 4) in the SIFU group (95% CI: 7%, 52%) versus 27% in the RTAS group (95% CI: 6%, 61%). Positive lymph nodes were found in seven (44%; 95% CI: 20%, 70%) invasive cancers after SIFU and in three (25%; 95% CI: 5%, 57%) invasive cancers after RTAS. CONCLUSION: Similar rates of cancer detection were found between SIFU and RTAS after benign breast biopsy with no significant differences in stage, tumor size, or nodal status, although the present study was limited by sample size. These findings suggest that patients with benign radiologic-pathologic-concordant percutaneous breast biopsy results could return to annual screening.

Diagnostic concordance among pathologists interpreting breast biopsy specimens.

IMPORTANCE: A breast pathology diagnosis provides the basis for clinical treatment and management decisions; however, its accuracy is inadequately understood. OBJECTIVES: To quantify the magnitude of diagnostic disagreement among pathologists compared with a consensus panel reference diagnosis and to evaluate associated patient and pathologist characteristics. DESIGN, SETTING, AND PARTICIPANTS: Study of pathologists who interpret breast biopsies in clinical practices in 8 US states. EXPOSURES: Participants independently interpreted slides between November 2011 and May 2014 from test sets of 60 breast biopsies (240 total cases, 1 slide per case), including 23 cases of invasive breast cancer, 73 ductal carcinoma in situ (DCIS), 72 with atypical hyperplasia (atypia), and 72 benign cases without atypia. Participants were blinded to the interpretations of other study pathologists and consensus panel members. Among the 3 consensus panel members, unanimous agreement of their independent diagnoses was 75%, and concordance with the consensus-derived reference diagnoses was 90.3%. MAIN OUTCOMES AND MEASURES: The proportions of diagnoses overinterpreted and underinterpreted relative to the consensus-derived reference diagnoses were assessed. RESULTS: Sixty-five percent of invited, responding pathologists were eligible and consented to participate. Of these, 91% (N = 115) completed the study, providing 6900 individual case diagnoses. Compared with the consensus-derived reference diagnosis, the overall concordance rate of diagnostic interpretations of participating pathologists was 75.3% (95% CI, 73.4%-77.0%; 5194 of 6900 interpretations). Among invasive carcinoma cases (663 interpretations), 96% (95% CI, 94%-97%) were concordant, and 4% (95% CI, 3%-6%) were underinterpreted; among DCIS cases (2097 interpretations), 84% (95% CI, 82%-86%) were concordant, 3% (95% CI, 2%-4%) were overinterpreted, and 13% (95% CI, 12%-15%) were underinterpreted; among atypia cases (2070 interpretations), 48% (95% CI, 44%-52%) were concordant, 17% (95% CI, 15%-21%) were overinterpreted, and 35% (95% CI, 31%-39%) were underinterpreted; and among benign cases without atypia (2070 interpretations), 87% (95% CI, 85%-89%) were concordant and 13% (95% CI, 11%-15%) were overinterpreted. Disagreement with the reference diagnosis was statistically significantly higher among biopsies from women with higher (n = 122) vs lower (n = 118) breast density on prior mammograms (overall concordance rate, 73% [95% CI, 71%-75%] for higher vs 77% [95% CI, 75%-80%] for lower, P < .001), and among pathologists who interpreted lower weekly case volumes (P < .001) or worked in smaller practices (P = .034) or nonacademic settings (P = .007). CONCLUSIONS AND RELEVANCE: In this study of pathologists, in which diagnostic interpretation was based on a single breast biopsy slide, overall agreement between the individual pathologists' interpretations and the expert consensus-derived reference diagnoses was 75.3%, with the highest level of concordance for invasive carcinoma and lower levels of concordance for DCIS and atypia. Further research is needed to understand the relationship of these findings with patient management.

Registry-based study of trends in breast cancer screening mammography before and after the 2009 U.S. Preventive Services Task Force recommendations.

PURPOSE: To determine whether the 2009 U.S. Preventive Services Task Force (USPSTF) guidelines for breast cancer mammography screening were followed by changes in screening utilization in the state of Vermont. MATERIALS AND METHODS: This retrospective study was HIPAA compliant and approved by the institutional review board, with waiver of informed consent. Trends in screening mammography utilization during 1997-2011 were examined among approximately 150,000 women aged 40 years and older in the state of Vermont using statewide mammography registry data. RESULTS: The percentage of Vermont women aged 40 years and older screened in the past year declined from 45.3% in 2009% to 41.6% in 2011 (an absolute decrease of -3.7 percentage points; 95% confidence interval [CI]: -3.3, -4.1). The largest decline in utilization was among women aged 40-49 years (-4.8 percentage points; 95% CI: -4.1, -5.4), although substantial declines were also observed among women aged 50-74 years (-3.0 percentage points; 95% CI: -2.6, -3.5) and women aged 75 years and older (-3.1 percentage points; 95% CI: -2.3, -4.0). The percentage of women aged 50-74 years screened within the past 2 years declined by -3.4 percentage points (95% CI: -3.0, -3.9) from 65.4% in 2009 to 61.9% in 2011. CONCLUSION: After years of increasing screening mammography utilization in Vermont, there was a decline in screening, which coincided with the release of the 2009 USPSTF recommendations. The age-specific patterns in utilization were generally consistent with the USPSTF recommendations, although there was also evidence that the percentage of women aged 50-74 years screened in the past 2 years declined since 2009.

Effect of radiologists' diagnostic work-up volume on interpretive performance.

PURPOSE: To examine radiologists' screening performance in relation to the number of diagnostic work-ups performed after abnormal findings are discovered at screening mammography by the same radiologist or by different radiologists. MATERIALS AND METHODS: In an institutional review board-approved HIPAA-compliant study, the authors linked 651 671 screening mammograms interpreted from 2002 to 2006 by 96 radiologists in the Breast Cancer Surveillance Consortium to cancer registries (standard of reference) to evaluate the performance of screening mammography (sensitivity, false-positive rate [ FPR false-positive rate ], and cancer detection rate [ CDR cancer detection rate ]). Logistic regression was used to assess the association between the volume of recalled screening mammograms ("own" mammograms, where the radiologist who interpreted the diagnostic image was the same radiologist who had interpreted the screening image, and "any" mammograms, where the radiologist who interpreted the diagnostic image may or may not have been the radiologist who interpreted the screening image) and screening performance and whether the association between total annual volume and performance differed according to the volume of diagnostic work-up. RESULTS: Annually, 38% of radiologists performed the diagnostic work-up for 25 or fewer of their own recalled screening mammograms, 24% performed the work-up for 0-50, and 39% performed the work-up for more than 50. For the work-up of recalled screening mammograms from any radiologist, 24% of radiologists performed the work-up for 0-50 mammograms, 32% performed the work-up for 51-125, and 44% performed the work-up for more than 125. With increasing numbers of radiologist work-ups for their own recalled mammograms, the sensitivity (P = .039), FPR false-positive rate (P = .004), and CDR cancer detection rate (P < .001) of screening mammography increased, yielding a stepped increase in women recalled per cancer detected from 17.4 for 25 or fewer mammograms to 24.6 for more than 50 mammograms. Increases in work-ups for any radiologist yielded significant increases in FPR false-positive rate (P = .011) and CDR cancer detection rate (P = .001) and a nonsignificant increase in sensitivity (P = .15). Radiologists with a lower annual volume of any work-ups had consistently lower FPR false-positive rate , sensitivity, and CDR cancer detection rate at all annual interpretive volumes. CONCLUSION: These findings support the hypothesis that radiologists may improve their screening performance by performing the diagnostic work-up for their own recalled screening mammograms and directly receiving feedback afforded by means of the outcomes associated with their initial decision to recall. Arranging for radiologists to work up a minimum number of their own recalled cases could improve screening performance but would need systems to facilitate this workflow.

Understanding diagnostic variability in breast pathology: lessons learned from an expert consensus review panel.

AIMS: To gain a better understanding of the reasons for diagnostic variability, with the aim of reducing the phenomenon. METHODS AND RESULTS: In preparation for a study on the interpretation of breast specimens (B-PATH), a panel of three experienced breast pathologists reviewed 336 cases to develop consensus reference diagnoses. After independent assessment, cases coded as diagnostically discordant were discussed at consensus meetings. By the use of qualitative data analysis techniques, transcripts of 16 h of consensus meetings for a subset of 201 cases were analysed. Diagnostic variability could be attributed to three overall root causes: (i) pathologist-related; (ii) diagnostic coding/study methodology-related; and (iii) specimen-related. Most pathologist-related root causes were attributable to professional differences in pathologists' opinions about whether the diagnostic criteria for a specific diagnosis were met, most frequently in cases of atypia. Diagnostic coding/study methodology-related root causes were primarily miscategorizations of descriptive text diagnoses, which led to the development of a standardized electronic diagnostic form (BPATH-Dx). Specimen-related root causes included artefacts, limited diagnostic material, and poor slide quality. After re-review and discussion, a consensus diagnosis could be assigned in all cases. CONCLUSIONS: Diagnostic variability is related to multiple factors, but consensus conferences, standardized electronic reporting formats and comments on suboptimal specimen quality can be used to reduce diagnostic variability.

Educational interventions to improve screening mammography interpretation: a randomized controlled trial.

OBJECTIVE: The objective of our study was to conduct a randomized controlled trial of educational interventions that were created to improve performance of screening mammography interpretation. MATERIALS AND METHODS: We randomly assigned physicians who interpret mammography to one of three groups: self-paced DVD, live expert-led educational seminar, or control. The DVD and seminar interventions used mammography cases of varying difficulty and provided associated teaching points. Interpretive performance was compared using a pretest-posttest design. Sensitivity, specificity, and positive predictive value (PPV) were calculated relative to two outcomes: cancer status and consensus of three experts about recall. The performance measures for each group were compared using logistic regression adjusting for pretest performance. RESULTS: One hundred two radiologists completed all aspects of the trial. After adjustment for preintervention performance, the odds of improved sensitivity for correctly identifying a lesion relative to expert recall were 1.34 times higher for DVD participants than for control subjects (95% CI, 1.00-1.81; p = 0.050). The odds of an improved PPV for correctly identifying a lesion relative to both expert recall (odds ratio [OR] = 1.94; 95% CI, 1.24-3.05; p = 0.004) and cancer status (OR = 1.81; 95% CI, 1.01-3.23; p = 0.045) were significantly improved for DVD participants compared with control subjects, with no significant change in specificity. For the seminar group, specificity was significantly lower than the control group (OR relative to expert recall = 0.80; 95% CI, 0.64-1.00; p = 0.048; OR relative to cancer status = 0.79; 95% CI, 0.65-0.95; p = 0.015). CONCLUSION: In this randomized controlled trial, the DVD educational intervention resulted in a significant improvement in screening mammography interpretive performance on a test set, which could translate into improved interpretative performance in clinical practice.

Development of a diagnostic test set to assess agreement in breast pathology: practical application of the Guidelines for Reporting Reliability and Agreement Studies (GRRAS).

BACKGROUND: Diagnostic test sets are a valuable research tool that contributes importantly to the validity and reliability of studies that assess agreement in breast pathology. In order to fully understand the strengths and weaknesses of any agreement and reliability study, however, the methods should be fully reported. In this paper we provide a step-by-step description of the methods used to create four complex test sets for a study of diagnostic agreement among pathologists interpreting breast biopsy specimens. We use the newly developed Guidelines for Reporting Reliability and Agreement Studies (GRRAS) as a basis to report these methods. METHODS: Breast tissue biopsies were selected from the National Cancer Institute-funded Breast Cancer Surveillance Consortium sites. We used a random sampling stratified according to woman's age (40-49 vs. ≥50), parenchymal breast density (low vs. high) and interpretation of the original pathologist. A 3-member panel of expert breast pathologists first independently interpreted each case using five primary diagnostic categories (non-proliferative changes, proliferative changes without atypia, atypical ductal hyperplasia, ductal carcinoma in situ, and invasive carcinoma). When the experts did not unanimously agree on a case diagnosis a modified Delphi method was used to determine the reference standard consensus diagnosis. The final test cases were stratified and randomly assigned into one of four unique test sets. CONCLUSIONS: We found GRRAS recommendations to be very useful in reporting diagnostic test set development and recommend inclusion of two additional criteria: 1) characterizing the study population and 2) describing the methods for reference diagnosis, when applicable.

Mammographic interpretive volume and diagnostic mammogram interpretation performance in community practice.

PURPOSE: To investigate the association between radiologist interpretive volume and diagnostic mammography performance in community-based settings. MATERIALS AND METHODS: This study received institutional review board approval and was HIPAA compliant. A total of 117,136 diagnostic mammograms that were interpreted by 107 radiologists between 2002 and 2006 in the Breast Cancer Surveillance Consortium were included. Logistic regression analysis was used to estimate the adjusted effect on sensitivity and the rates of false-positive findings and cancer detection of four volume measures: annual diagnostic volume, screening volume, total volume, and diagnostic focus (percentage of total volume that is diagnostic). Analyses were stratified by the indication for imaging: additional imaging after screening mammography or evaluation of a breast concern or problem. RESULTS: Diagnostic volume was associated with sensitivity; the odds of a true-positive finding rose until a diagnostic volume of 1000 mammograms was reached; thereafter, they either leveled off (P < .001 for additional imaging) or decreased (P = .049 for breast concerns or problems) with further volume increases. Diagnostic focus was associated with false-positive rate; the odds of a false-positive finding increased until a diagnostic focus of 20% was reached and decreased thereafter (P < .024 for additional imaging and P < .001 for breast concerns or problems with no self-reported lump). Neither total volume nor screening volume was consistently associated with diagnostic performance. CONCLUSION: Interpretive volume and diagnostic performance have complex multifaceted relationships. Our results suggest that diagnostic interpretive volume is a key determinant in the development of thresholds for considering a diagnostic mammogram to be abnormal. Current volume regulations do not distinguish between screening and diagnostic mammography, and doing so would likely be challenging.

Web-based mammography audit feedback.

OBJECTIVE: Interpreting screening mammography accurately is challenging and requires ongoing education to maintain and improve interpretative skills. Recognizing this, many countries with organized breast screening programs have developed audit and feedback systems using national performance data to help radiologists assess and improve their skills. We developed and tested an interactive Website to provide screening and diagnostic mammography audit feedback with comparisons to national and regional benchmarks. MATERIALS AND METHODS: Radiologists who participate in three Breast Cancer Surveillance Consortium registries in the United States were invited during 2009 and 2010 to use a Website that provides tabular and graphical displays of mammography audit reports with comparisons to national and regional performance measures. We collected data about the use and perceptions of the Website. RESULTS: Thirty-five of 111 invited radiologists used the Website from one to five times in a year. The most popular measure was sensitivity for both screening and diagnostic mammography, whereas a table with all measures was the most visited page. Of the 13 radiologists who completed the postuse survey, all found the Website easy to use and navigate, 11 found the benchmarks useful, and nine reported that they intended to improve a specific outcome measure that year. CONCLUSION: An interactive Website to provide customized mammography audit feedback reports to radiologists has the potential to be a powerful tool in improving interpretive performance. The conceptual framework of customized audit feedback reports can also be generalized to other imaging tests.

Is confidence of mammographic assessment a good predictor of accuracy?

OBJECTIVE: Interpretive accuracy varies among radiologists, especially in mammography. This study examines the relationship between radiologists' confidence in their assessments and their accuracy in interpreting mammograms. MATERIALS AND METHODS: In this study, 119 community radiologists interpreted 109 expert-defined screening mammography examinations in test sets and rated their confidence in their assessment for each case. They also provided a global assessment of their ability to interpret mammograms. Positive predictive value (PPV) and negative predictive value (NPV) were modeled as functions of self-rated confidence on each examination using log-linear regression estimated with generalized estimating equations. Reference measures were cancer status and expert-defined need for recall. Effect modification by weekly mammography volume was examined. RESULTS: Radiologists who self-reported higher global interpretive ability tended to interpret more mammograms per week (p = 0.08), were more likely to specialize (p = 0.02) and to have completed a fellowship in breast or women's imaging (p = 0.05), and had a higher PPV for cancer detection (p = 0.01). Examinations for which low-volume radiologists were "very confident" had a PPV of 2.93 times (95% CI, 2.01-4.27) higher than examinations they rated with neutral confidence. Trends of increasing NPVs with increasing confidence were significant for low-volume radiologists relative to noncancers (p = 0.01) and expert nonrecalls (p < 0.001). A trend of significantly increasing NPVs existed for high-volume radiologists relative to expert nonrecall (p = 0.02) but not relative to noncancer status (p = 0.32). CONCLUSION: Confidence in mammography assessments was associated with better accuracy, especially for low-volume readers. Asking for a second opinion when confidence in an assessment is low may increase accuracy.

Association between time spent interpreting, level of confidence, and accuracy of screening mammography.

OBJECTIVE: The objective of this study was to examine the effect of time spent viewing images and level of confidence on a screening mammography test set on interpretive performance. MATERIALS AND METHODS: Radiologists from six mammography registries participated in this study and were randomized to interpret one of four test sets and complete 12 survey questions. Each test set had 109 cases of digitized four-view screening screen-film mammograms with prior comparison screening views. Viewing time for each case was defined as the cumulative time spent viewing all mammographic images before recording which visible feature, if any, was the "most significant finding." Log-linear regression fit via the generalized estimating equation was used to test the effect of viewing time and level of confidence in the interpretation on test set sensitivity and false-positive rate. RESULTS: One hundred nineteen radiologists completed a test set and contributed data on 11,484 interpretations. The radiologists spent more time viewing cases that had significant findings or cases for which they had less confidence in their interpretation. Each additional minute of viewing time increased the probability of a true-positive interpretation among cancer cases by 1.12 (95% CI, 1.06-1.19; p < 0.001) regardless of confidence in the assessment. Among the radiologists who were very confident in their assessment, each additional minute of viewing time increased the adjusted risk of a false-positive interpretation among noncancer cases by 1.42 (95% CI, 1.21-1.68), and this viewing-time effect diminished with decreasing confidence. CONCLUSION: Longer interpretation times and higher levels of confidence in an interpretation are both associated with higher sensitivity and false-positive rates in mammography screening.