Resident Assessment of Clinician Educators According to Core ACGME Competencies.

BACKGROUND: The Accreditation Council for Graduate Medical Education (ACGME) requires faculty to pursue annual development to enhance their teaching skills. Few studies exist on how to identify and improve the quality of teaching provided by faculty educators. Understanding the correlation between numeric scores assigned to faculty educators and their tangible, practical teaching skills would be beneficial. OBJECTIVE: This study aimed to identify and describe qualities that differentiate numerically highly rated and low-rated physician educators. DESIGN: This observational mixed-methods study evaluated attending physician educators between July 1, 2015, and June 30, 2021. Quantitative analysis involved descriptive statistics, normalization of scores, and stratification of faculty into tertiles based on a summary score. We compared the highest and lowest tertiles during qualitative analyses of residents' comments. PARTICIPANTS: Twenty-five attending physicians and 111 residents in an internal medicine residency program. MAIN MEASURES: Resident evaluations of faculty educators, including 724 individual assessments of faculty educators on 15 variables related to the ACGME core competencies. KEY RESULTS: Quantitative analyses revealed variation in attending physician educators' performance across the ACGME core competencies. The highest-rated teaching qualities were interpersonal and communication skills, medical knowledge, and professionalism, while the lowest-rated teaching quality was systems-based practice. Qualitative analyses identified themes distinguishing high-quality from low-quality attending physician educators, such as balancing autonomy and supervision, role modeling, engagement, availability, compassion, and excellent teaching. CONCLUSIONS: This study provides insights into areas where attending physicians' educational strategies can be improved, emphasizing the importance of role modeling and effective communication. Ongoing efforts are needed to enhance the quality of faculty educators and resident education in internal medicine residency programs.

Factors Associated with Medical Students' Attitudes About Cost-Conscious Care: A Mixed-Methods Multi-school Study.

BACKGROUND: Because physician practices contribute to national healthcare expenditures, initiatives aimed at educating physicians about high-value cost-conscious care (HVCCC) are important. Prior studies suggest that the training environment influences physician attitudes and behaviors towards HVCCC. OBJECTIVE: To explore the relationship between medical student experiences and HVCCC attitudes. DESIGN: Quantitative and qualitative analysis of a multi-institutional survey. PARTICIPANTS: Medical students from nine US medical schools. APPROACH: A 44-item survey that included the Maastricht HVCCC Attitudes Questionnaire, a validated tool for assessing HVCCC attitudes, was administered electronically. Attitudinal domains of high-value care (HVC), cost incorporation (CI), and perceived drawbacks (PD) were compared using one-way ANOVA among students with a range of exposures. Open text responses inviting participants to reflect on their attitudes were analyzed using classical content analysis. KEY RESULTS: A total of 740 students completed the survey (response rate 15%). Students pursuing a "continuity-oriented" specialty held more favorable attitudes towards HVCCC than those pursuing "technique-oriented" specialties (HVC sub-score = 3.20 vs. 3.06; p = 0.005, CI sub-score = 2.83 vs. 2.74; p < 0.001). Qualitative analyses revealed personal, educational, and professional experiences shape students' HVCCC attitudes, with similar experiences interpreted differently leading to both more and less favorable attitudes. CONCLUSION: Students pursuing specialties with longitudinal patient contact may be more enthusiastic about practicing high-value care. Life experiences before and during medical school shape these attitudes, and complex interactions between these forces drive student perceptions of HVCCC.

Development and Pilot Testing of a Programmatic System for Competency Assessment in US Anesthesiology Residency Training.

BACKGROUND: In 2018, a set of entrustable professional activities (EPAs) and procedural skills assessments were developed for anesthesiology training, but they did not assess all the Accreditation Council for Graduate Medical Education (ACGME) milestones. The aims of this study were to (1) remap the 2018 EPA and procedural skills assessments to the revised ACGME Anesthesiology Milestones 2.0, (2) develop new assessments that combined with the original assessments to create a system of assessment that addresses all level 1 to 4 milestones, and (3) provide evidence for the validity of the assessments. METHODS: Using a modified Delphi process, a panel of anesthesiology education experts remapped the original assessments developed in 2018 to the Anesthesiology Milestones 2.0 and developed new assessments to create a system that assessed all level 1 through 4 milestones. Following a 24-month pilot at 7 institutions, the number of EPA and procedural skill assessments and mean scores were computed at the end of the academic year. Milestone achievement and subcompetency data for assessments from a single institution were compared to scores assigned by the institution's clinical competency committee (CCC). RESULTS: New assessment development, 2 months of testing and feedback, and revisions resulted in 5 new EPAs, 11 nontechnical skills assessments (NTSAs), and 6 objective structured clinical examinations (OSCEs). Combined with the original 20 EPAs and procedural skills assessments, the new system of assessment addresses 99% of level 1 to 4 Anesthesiology Milestones 2.0. During the 24-month pilot, aggregate mean EPA and procedural skill scores significantly increased with year in training. System subcompetency scores correlated significantly with 15 of 23 (65.2%) corresponding CCC scores at a single institution, but 8 correlations (36.4%) were <30.0, illustrating poor correlation. CONCLUSIONS: A panel of experts developed a set of EPAs, procedural skill assessment, NTSAs, and OSCEs to form a programmatic system of assessment for anesthesiology residency training in the United States. The method used to develop and pilot test the assessments, the progression of assessment scores with time in training, and the correlation of assessment scores with CCC scoring of milestone achievement provide evidence for the validity of the assessments.

Differences in perceived clinical knowledge uptake among health profession students and licensed clinicians receiving buprenorphine waiver training in Oregon.

Background: Addressing the opioid crisis requires an understanding of how to train both health professional students and practicing clinicians on medications for opioid use disorder (mOUD). We designed a robust evaluation instrument to assess the impact of training on perceived clinical knowledge in these different categories of learners. Methods: We enrolled 3rd and 4th year medical, physician assistant (PA), and nurse practitioner (NP) students, as well as practicing PAs, NPs, and physicians to undertake the Drug Addiction Treatment Act (DATA) Waiver Training for mOUD. We designed and implemented a cross-sectional survey to assess perceived change in clinical knowledge as a result of training in opioid use disorder and satisfaction with training. Results: Twenty-one MD/DO and 45 NP/PA students, and 24 practicing MD/DO and 27 NP/PAs completed the survey. Among health professional students (n = 66) and practicing clinicians (n =51), perceived clinical knowledge scores increased significantly (p < 0.001) for all 13 variables. Program evaluation scores for the buprenorphine waiver training were high with no statistical differences between students and practicing clinicians. Overall, the majority of participants indicated they would recommend the training to a colleague (Students' score = 4.84; practicing clinician scores = 4.53; scale = strongly disagree = 1 to strongly agree = 5). Conclusions: Our novel instrument allowed us to determine that the implementation of buprenorphine waiver trainings for health professional students and practicing clinicians leads to significant increases in perceived knowledge, interest and confidence in diagnosing and treating OUD. Although the buprenorphine waiver can now be obtained without training, many waivered providers still do not prescribe buprenorphine; integrating training into medical, NP, and PA curriculum for students and offering the training to practicing clinicians may increase confidence and uptake of mOUD.

Measuring Coaching in Undergraduate Medical Education: the Development and Psychometric Validation of New Instruments.

BACKGROUND: Coaching is emerging as a novel approach to guide medical students toward becoming competent, reflective physicians and master adaptive learners. However, no instruments currently exist to measure academic coaching at the undergraduate medical education level. OBJECTIVE: To describe the development and psychometric assessments of two instruments designed to assess academic coaching of medical students toward creating a robust measurement model of this educational paradigm. DESIGN: Observational psychometric. PARTICIPANTS: All medical students in the 2014 and 2015 cohorts and all their coaches were invited to complete the instruments being tested, which led to 662 medical student responses from 292 medical students and 468 coaching responses from 22 coaches being included in analyses. Medical student response rates were 75.7% from 2014 and 75.5% from 2015. Overall, the coaches' response rate was 71%. MAIN MEASURES: Two 31-item instruments were initially developed, one for medical students to assess their coach and one for faculty coaches to assess their students, both of which evaluated coaching based on definitions we formulated using existing literature. Each was administered to two cohorts of medical students and coaches in 2015 and 2016. An exploratory factor analysis using principal component analysis as the extraction method and Varimax with Kaiser normalization as the rotation method was conducted. KEY RESULTS: Eighteen items reflecting four domains were retained with eigenvalues higher than 1.0 for medical students' assessment of coaching, which measured promoting self-monitoring, relationship building, promoting reflective behavior, and establishing foundational ground rules. Sixteen items reflecting two domains were retained for the faculty assessment of coaching with eigenvalues higher than 1.0: the Practice of Coaching and Relationship Formation. CONCLUSIONS: We successfully developed and psychometrically validated surveys designed to measure key aspects of the coaching relationship, coaching processes, and reflective outcomes. The new validated instruments offer a robust measurement model for academic coaching.

Indigenizing Academics Through Leadership, Awareness, and Healing: The Impact of a Native American Health Seminar Series for Health Professionals, Students, and Community.

Health disparities have long affected American Indian and Alaska Native (AI/AN) populations. Transformations are needed in academia to help understand Indigenous 'ways of knowing.' Lifting the voices of AI/ANs in telling their stories could improve the education of students, faculty and the lay public. We collaborated to develop, implement and evaluate a Native American Health Seminar Series taught by AI/AN leaders on addressing health disparities among AI/AN people. A quasi-experimental mixed methods design included a 15-item survey to assess the impact of the Seminar Series on knowledge of AI/AN health issues and its influence, among students, on health career choices. During the 2018 academic year, three seminars were held and 243 participants attended. In total, 182 surveys (74.9%) were completed by faculty members, students and members of the lay public. Students (all categories combined) represented the highest participant group (48.4%), followed by the lay public at 30% and faculty at 21.6%. The highest scores on knowledge of Native health issues prior to seminar attendance were reported by those representing the lay public with a mean of 3.96 compared to 3.67 for faculty and 3.43 among students (p = 0.01), which was highly represented by Indigenous people. Increases in knowledge occurred in all participant groups. Among students, 65.6% initially indicated that they were not planning on pursuing a career in Native health. Among these, 56.9% indicated they were somewhat to extremely likely to pursue a career in Native health as a result of having attended the seminar.

Boosting medical diagnostics by pooling independent judgments.

Collective intelligence refers to the ability of groups to outperform individual decision makers when solving complex cognitive problems. Despite its potential to revolutionize decision making in a wide range of domains, including medical, economic, and political decision making, at present, little is known about the conditions underlying collective intelligence in real-world contexts. We here focus on two key areas of medical diagnostics, breast and skin cancer detection. Using a simulation study that draws on large real-world datasets, involving more than 140 doctors making more than 20,000 diagnoses, we investigate when combining the independent judgments of multiple doctors outperforms the best doctor in a group. We find that similarity in diagnostic accuracy is a key condition for collective intelligence: Aggregating the independent judgments of doctors outperforms the best doctor in a group whenever the diagnostic accuracy of doctors is relatively similar, but not when doctors' diagnostic accuracy differs too much. This intriguingly simple result is highly robust and holds across different group sizes, performance levels of the best doctor, and collective intelligence rules. The enabling role of similarity, in turn, is explained by its systematic effects on the number of correct and incorrect decisions of the best doctor that are overruled by the collective. By identifying a key factor underlying collective intelligence in two important real-world contexts, our findings pave the way for innovative and more effective approaches to complex real-world decision making, and to the scientific analyses of those approaches.

Improvements in hospice utilization among patients with advanced-stage lung cancer in an integrated health care system.

BACKGROUND: Hospice, a patient-centered care system for those with limited life expectancy, is important for enhancing quality of life and is understudied in integrated health care systems. METHODS: This was a retrospective cohort study of 21,860 decedents with advanced-stage lung cancer diagnosed from January 2007 to June 2013 in the national US Veterans Affairs Health Care System. Trends over time, geographic regional variability, and patient and tumor characteristics associated with hospice use and the timing of enrollment were examined. Multivariable logistic regression and Cox proportional hazards modeling were used. RESULTS: From 2007 to 2013, 70.3% of decedents with advanced-stage lung cancer were enrolled in hospice. Among patients in hospice, 52.9% were enrolled in the last month of life, and 14.7% were enrolled in the last 3 days of life. Hospice enrollment increased (adjusted odds ratio [AOR], 1.07; P < .001), whereas the mean time from the cancer diagnosis to hospice enrollment decreased by 65 days (relative decrease, 32%; adjusted hazard ratio, 1.04; P < .001). Relative decreases in late hospice enrollment were observed in the last month (7%; AOR, 0.98; P = .04) and last 3 days of life (26%; AOR, 0.95; P < .001). The Southeast region of the United States had both the highest rate of hospice enrollment and the lowest rate of late enrollment. Patient sociodemographic and lung cancer characteristics were associated with hospice enrollment. CONCLUSIONS: Among patients with advanced-stage lung cancer in the Veterans Affairs Health Care System, overall and earlier hospice enrollment increased over time. Considerable regional variability in hospice enrollment and the persistence of late enrollment suggests opportunities for improvement in end-of-life care. Cancer 2018;124:426-33. © 2017 American Cancer Society.

Second opinion strategies in breast pathology: a decision analysis addressing over-treatment, under-treatment, and care costs.

PURPOSE: To estimate the potential near-term population impact of alternative second opinion breast biopsy pathology interpretation strategies. METHODS: Decision analysis examining 12-month outcomes of breast biopsy for nine breast pathology interpretation strategies in the U.S. health system. Diagnoses of 115 practicing pathologists in the Breast Pathology Study were compared to reference-standard-consensus diagnoses with and without second opinions. Interpretation strategies were defined by whether a second opinion was sought universally or selectively (e.g., 2nd opinion if invasive). Main outcomes were the expected proportion of concordant breast biopsy diagnoses, the proportion involving over- or under-interpretation, and cost of care in U.S. dollars within one-year of biopsy. RESULTS: Without a second opinion, 92.2% of biopsies received a concordant diagnosis. Concordance rates increased under all second opinion strategies, and the rate was highest (95.1%) and under-treatment lowest (2.6%) when all biopsies had second opinions. However, over-treatment was lowest when second opinions were sought selectively for initial diagnoses of invasive cancer, DCIS, or atypia (1.8 vs. 4.7% with no 2nd opinions). This strategy also had the lowest projected 12-month care costs ($5.907 billion vs. $6.049 billion with no 2nd opinions). CONCLUSIONS: Second opinion strategies could lower overall care costs while reducing both over- and under-treatment. The most accurate cost-saving strategy required second opinions for initial diagnoses of invasive cancer, DCIS, or atypia.

Pathologist characteristics associated with accuracy and reproducibility of melanocytic skin lesion interpretation.

BACKGROUND: Diagnostic interpretations of melanocytic skin lesions vary widely among pathologists, yet the underlying reasons remain unclear. OBJECTIVE: Identify pathologist characteristics associated with rates of accuracy and reproducibility. METHODS: Pathologists independently interpreted the same set of biopsy specimens from melanocytic lesions on 2 occasions. Diagnoses were categorized into 1 of 5 classes according to the Melanocytic Pathology Assessment Tool and Hierarchy for Diagnosis system. Reproducibility was determined by pathologists' concordance of diagnoses across 2 occasions. Accuracy was defined by concordance with a consensus reference standard. Associations of pathologist characteristics with reproducibility and accuracy were assessed individually and in multivariable logistic regression models. RESULTS: Rates of diagnostic reproducibility and accuracy were highest among pathologists with board certification and/or fellowship training in dermatopathology and in those with 5 or more years of experience. In addition, accuracy was high among pathologists with a higher proportion of melanocytic lesions in their caseload composition and higher volume of melanocytic lesions. LIMITATIONS: Data gathered in a test set situation by using a classification tool not currently in clinical use. CONCLUSION: Diagnoses are more accurate among pathologists with specialty training and those with more experience interpreting melanocytic lesions. These findings support the practice of referring difficult cases to more experienced pathologists to improve diagnostic accuracy, although the impact of these referrals on patient outcomes requires additional research.

Complexities of perceived and actual performance in pathology interpretation: A comparison of cutaneous melanocytic skin and breast interpretations.

BACKGROUND: Little is known about how pathologists process differences between actual and perceived interpretations. OBJECTIVE: To compare perceived and actual diagnostic agreement before and after educational interventions. METHODS: Pathologists interpreted test sets of skin and/or breast specimens that included benign, atypical, in situ and invasive lesions. Interventions involved self-directed learning, one skin and one breast, that showed pathologists how their interpretations compared to a reference diagnoses. Prior to the educational intervention, participants estimated how their interpretations would compare to the reference diagnoses. After the intervention, participants estimated their overall agreement with the reference diagnoses. Perceived and actual agreements were compared. RESULTS: For pathologists interpreting skin, mean actual agreement was 52.4% and overall pre- and postinterventional mean perceived agreement was 72.9% vs 54.2%, an overestimated mean difference of 20.5% (95% confidence interval [CI] 17.2% to 24.0%) and 1.8% (95% CI -0.5% to 4.1%), respectively. For pathologists interpreting breast, mean actual agreement was 75.9% and overall pre- and postinterventional mean perceived agreement was 81.4% vs 76.9%, an overestimation of 5.5% (95% CI 3.0% to 8.0%) and 1.0% (95% CI 0.0% to 2.0%), respectively. CONCLUSIONS: Pathologists interpreting breast tissue had improved comprehension of their performance after the intervention compared to pathologists interpreting skin lesions.

Pathologists' Use of Second Opinions in Interpretation of Melanocytic Cutaneous Lesions: Policies, Practices, and Perceptions.

BACKGROUND: Research examining the role of second opinions in pathology for diagnosis of melanocytic lesions is limited. OBJECTIVE: To assess current laboratory policies, clinical use of second opinions, and pathologists' perceptions of second opinions for melanocytic lesions. MATERIALS AND METHODS: Cross-sectional data collected from 207 pathologists in 10 US states who diagnose melanocytic lesions. The web-based survey ascertained pathologists' professional information, laboratory second opinion policy, use of second opinions, and perceptions of second opinion value for melanocytic lesions. RESULTS: Laboratory policies required second opinions for 31% of pathologists and most commonly required for melanoma in situ (26%) and invasive melanoma (30%). In practice, most pathologists reported requesting second opinions for melanocytic tumors of uncertain malignant potential (85%) and atypical Spitzoid lesions (88%). Most pathologists perceived that second opinions increased interpretive accuracy (78%) and protected them from malpractice lawsuits (62%). CONCLUSION: Use of second opinions in clinical practice is greater than that required by laboratory policies, especially for melanocytic tumors of uncertain malignant potential and atypical Spitzoid lesions. Quality of care in surgical interventions for atypical melanocytic proliferations critically depends on the accuracy of diagnosis in pathology reporting. Future research should examine the extent to which second opinions improve accuracy of melanocytic lesion diagnosis.

Assessing learning in the adaptive curriculum.

BACKGROUND: Medical education is a dynamic process that will continuously evolve to respond to changes in the foundations of medicine, the clinical practice of medicine and in health systems science. PURPOSE: In this paper, we review how assessing learning in such a dynamic environment requires comprehensive flexible and adaptable methodological approaches designed to assess knowledge attainment and transfer, clinical skills/competency development, and ethical/professional behavior. Adaptive assessments should measure the learner's ability to observe where changes in health care delivery are needed and how to implement them. Balancing formative and summative assessments will promote reflective learning so that each student will reach her/his highest potential. From the programmatic perspective, measuring the design and delivery of instruction in relation to students? efforts to achieve competency will improve learning and foster continuous professional development of faculty and advance the science of learning. APPROACH: We describe how two medical schools are approaching adaptive assessment, including using portfolio systems that encompass teaching and learning experiences while offering real-time longitudinal tracking of digital data toward improving learning and provide curricula continuous improvement cycles. Using latest technologies, portfolios produce actionable data displays with precise guidance for learning and program development.

Attitudes toward cost-conscious care among U.S. physicians and medical students: analysis of national cross-sectional survey data by age and stage of training.

BACKGROUND: The success of initiatives intended to increase the value of health care depends, in part, on the degree to which cost-conscious care is endorsed by current and future physicians. This study aimed to first analyze attitudes of U.S. physicians by age and then compare the attitudes of physicians and medical students. METHODS: A paper survey was mailed in mid-2012 to 3897 practicing physicians randomly selected from the American Medical Association Masterfile. An electronic survey was sent in early 2015 to all 5,992 students at 10 U.S. medical schools. Survey items measured attitudes toward cost-conscious care and perceived responsibility for reducing healthcare costs. Physician responses were first compared across age groups (30-40 years, 41-50 years, 51-60 years, and > 60 years) and then compared to student responses using Chi square tests and logistic regression analyses (controlling for sex). RESULTS: A total of 2,556 physicians (65%) and 3395 students (57%) responded. Physician attitudes generally did not differ by age, but differed significantly from those of students. Specifically, students were more likely than physicians to agree that cost to society should be important in treatment decisions (p < 0.001) and that physicians should sometimes deny beneficial but costly services (p < 0.001). Students were less likely to agree that it is unfair to ask physicians to be cost-conscious while prioritizing patient welfare (p < 0.001). Compared to physicians, students assigned more responsibility for reducing healthcare costs to hospitals and health systems (p < 0.001) and less responsibility to lawyers (p < 0.001) and patients (p < 0.001). Nearly all significant differences persisted after controlling for sex and when only the youngest physicians were compared to students. CONCLUSIONS: Physician attitudes toward cost-conscious care are similar across age groups. However, physician attitudes differ significantly from medical students, even among the youngest physicians most proximate to students in age. Medical student responses suggest they are more accepting of cost-conscious care than physicians and attribute more responsibility for reducing costs to organizations and systems rather than individuals. This may be due to the combined effects of generational differences, new medical school curricula, students' relative inexperience providing cost-conscious care within complex healthcare systems, and the rapidly evolving U.S. healthcare system.

Economic models for sustainable interprofessional education.

Limited information exists on funding models for interprofessional education (IPE) course delivery, even though potential savings from IPE could be gained in healthcare delivery efficiencies and patient safety. Unanticipated economic barriers to implementing an IPE curriculum across programs and schools in University settings can stymie or even end movement toward collaboration and sustainable culture change. Clarity among stakeholders, including institutional leadership, faculty, and students, is necessary to avoid confusion about IPE tuition costs and funds flow, given that IPE involves multiple schools and programs sharing space, time, faculty, and tuition dollars. In this paper, we consider three funding models for IPE: (a) Centralized (b) Blended, and (c) Decentralized. The strengths and challenges associated with each of these models are discussed. Beginning such a discussion will move us toward understanding the return on investment of IPE.

The influence of tumor regression, solar elastosis, and patient age on pathologists' interpretation of melanocytic skin lesions.

It is not known whether patient age or tumor characteristics such as tumor regression or solar elastosis influence pathologists' interpretation of melanocytic skin lesions (MSLs). We undertook a study to determine the influence of these factors, and to explore pathologist's characteristics associated with the direction of diagnosis. To meet our objective, we designed a cross-sectional survey study of pathologists' clinical practices and perceptions. Pathologists were recruited from diverse practices in 10 states in the United States. We enrolled 207 pathologist participants whose practice included the interpretation of MSLs. Our findings indicated that the majority of pathologists (54.6%) were influenced toward a less severe diagnosis when patients were <30 years of age. Most pathologists were influenced toward a more severe diagnosis when patients were >70 years of age, or by the presence of tumor regression or solar elastosis (58.5%, 71.0%, and 57.0%, respectively). Generally, pathologists with dermatopathology board certification and/or a high caseload of MSLs were more likely to be influenced, whereas those with more years' experience interpreting MSL were less likely to be influenced. Our findings indicate that the interpretation of MSLs is influenced by patient age, tumor regression, and solar elastosis; such influence is associated with dermatopathology training and higher caseload, consistent with expertise and an appreciation of lesion complexity.

Diagnostic Reproducibility: What Happens When the Same Pathologist Interprets the Same Breast Biopsy Specimen at Two Points in Time?

BACKGROUND: Surgeons may receive a different diagnosis when a breast biopsy is interpreted by a second pathologist. The extent to which diagnostic agreement by the same pathologist varies at two time points is unknown. METHODS: Pathologists from eight U.S. states independently interpreted 60 breast specimens, one glass slide per case, on two occasions separated by ≥9 months. Reproducibility was assessed by comparing interpretations between the two time points; associations between reproducibility (intraobserver agreement rates); and characteristics of pathologists and cases were determined and also compared with interobserver agreement of baseline interpretations. RESULTS: Sixty-five percent of invited, responding pathologists were eligible and consented; 49 interpreted glass slides in both study phases, resulting in 2940 interpretations. Intraobserver agreement rates between the two phases were 92% [95% confidence interval (CI) 88-95] for invasive breast cancer, 84% (95% CI 81-87) for ductal carcinoma-in-situ, 53% (95% CI 47-59) for atypia, and 84% (95% CI 81-86) for benign without atypia. When comparing all study participants' case interpretations at baseline, interobserver agreement rates were 89% (95% CI 84-92) for invasive cancer, 79% (95% CI 76-81) for ductal carcinoma-in-situ, 43% (95% CI 41-45) for atypia, and 77% (95% CI 74-79) for benign without atypia. CONCLUSIONS: Interpretive agreement between two time points by the same individual pathologist was low for atypia and was similar to observed rates of agreement for atypia between different pathologists. Physicians and patients should be aware of the diagnostic challenges associated with a breast biopsy diagnosis of atypia when considering treatment and surveillance decisions.

Variation among pathologists' treatment suggestions for melanocytic lesions: A survey of pathologists.

BACKGROUND: The extent of variability in treatment suggestions for melanocytic lesions made by pathologists is unknown. OBJECTIVE: We investigated how often pathologists rendered suggestions, reasons for providing suggestions, and concordance with national guidelines. METHODS: We conducted a cross-sectional survey of pathologists. Data included physician characteristics, experience, and treatment recommendation practices. RESULTS: Of 301 pathologists, 207 (69%) from 10 states (California, Connecticut, Hawaii, Iowa, Kentucky, Louisiana, New Jersey, New Mexico, Utah, and Washington) enrolled. In all, 15% and 7% reported never and always including suggestions, respectively. Reasons for offering suggestions included improved care (79%), clarification (68%), and legal liability (39%). Reasons for not offering suggestions included referring physician preference (48%), lack of clinical information (44%), and expertise (29%). Training and caseload were associated with offering suggestions (P < .05). Physician suggestions were most consistent for mild/moderate dysplastic nevi and melanoma. For melanoma in situ, 18 (9%) and 32 (15%) pathologists made suggestions that undertreated or overtreated lesions based on National Comprehensive Cancer Network (NCCN) guidelines, respectively. For invasive melanoma, 14 (7%) pathologists made treatment suggestions that undertreated lesions based on NCCN guidelines. LIMITATIONS: Treatment suggestions were self-reported. CONCLUSIONS: Pathologists made recommendations ranging in consistency. These findings may inform efforts to reduce treatment variability and optimize patterns of care delivery for patients.

Variability in Pathologists' Interpretations of Individual Breast Biopsy Slides: A Population Perspective.

BACKGROUND: The effect of physician diagnostic variability on accuracy at a population level depends on the prevalence of diagnoses. OBJECTIVE: To estimate how diagnostic variability affects accuracy from the perspective of a U.S. woman aged 50 to 59 years having a breast biopsy. DESIGN: Applied probability using Bayes' theorem. SETTING: B-Path (Breast Pathology) Study comparing pathologists' interpretations of a single biopsy slide versus a reference consensus interpretation from 3 experts. PARTICIPANTS: 115 practicing pathologists (6900 total interpretations from 240 distinct cases). MEASUREMENTS: A single representative slide from each of the 240 cases was used to estimate the proportion of biopsies with a diagnosis that would be verified if the same slide were interpreted by a reference group of 3 expert pathologists. Probabilities of confirmation (predictive values) were estimated using B-Path Study results and prevalence of biopsy diagnoses for women aged 50 to 59 years in the Breast Cancer Surveillance Consortium. RESULTS: Overall, if 1 representative slide were used per case, 92.3% (95% CI, 91.4% to 93.1%) of breast biopsy diagnoses would be verified by reference consensus diagnoses, with 4.6% (CI, 3.9% to 5.3%) overinterpreted and 3.2% (CI, 2.7% to 3.6%) underinterpreted. Verification of invasive breast cancer and benign without atypia diagnoses is highly probable; estimated predictive values were 97.7% (CI, 96.5% to 98.7%) and 97.1% (CI, 96.7% to 97.4%), respectively. Verification is less probable for atypia (53.6% overinterpreted and 8.6% underinterpreted) and ductal carcinoma in situ (DCIS) (18.5% overinterpreted and 11.8% underinterpreted). LIMITATIONS: Estimates are based on a testing situation with 1 slide used per case and without access to second opinions. Population-adjusted estimates may differ for women from other age groups, unscreened women, or women in different practice settings. CONCLUSION: This analysis, based on interpretation of a single breast biopsy slide per case, predicts a low likelihood that a diagnosis of atypia or DCIS would be verified by a reference consensus diagnosis. This diagnostic grey zone should be considered in clinical management decisions in patients with these diagnoses. PRIMARY FUNDING SOURCE: National Cancer Institute.

Identifying and processing the gap between perceived and actual agreement in breast pathology interpretation.

We examined how pathologists' process their perceptions of how their interpretations on diagnoses for breast pathology cases agree with a reference standard. To accomplish this, we created an individualized self-directed continuing medical education program that showed pathologists interpreting breast specimens how their interpretations on a test set compared with a reference diagnosis developed by a consensus panel of experienced breast pathologists. After interpreting a test set of 60 cases, 92 participating pathologists were asked to estimate how their interpretations compared with the standard for benign without atypia, atypia, ductal carcinoma in situ and invasive cancer. We then asked pathologists their thoughts about learning about differences in their perceptions compared with actual agreement. Overall, participants tended to overestimate their agreement with the reference standard, with a mean difference of 5.5% (75.9% actual agreement; 81.4% estimated agreement), especially for atypia and were least likely to overestimate it for invasive breast cancer. Non-academic affiliated pathologists were more likely to more closely estimate their performance relative to academic affiliated pathologists (77.6 vs 48%; P=0.001), whereas participants affiliated with an academic medical center were more likely to underestimate agreement with their diagnoses compared with non-academic affiliated pathologists (40 vs 6%). Before the continuing medical education program, nearly 55% (54.9%) of participants could not estimate whether they would overinterpret the cases or underinterpret them relative to the reference diagnosis. Nearly 80% (79.8%) reported learning new information from this individualized web-based continuing medical education program, and 23.9% of pathologists identified strategies they would change their practice to improve. In conclusion, when evaluating breast pathology specimens, pathologists do a good job of estimating their diagnostic agreement with a reference standard, but for atypia cases, pathologists tend to overestimate diagnostic agreement. Many participants were able to identify ways to improve.