Robust ROI Detection in Whole Slide Images Guided by Pathologists' Viewing Patterns.

Deep learning techniques offer improvements in computer-aided diagnosis systems. However, acquiring image domain annotations is challenging due to the knowledge and commitment required of expert pathologists. Pathologists often identify regions in whole slide images with diagnostic relevance rather than examining the entire slide, with a positive correlation between the time spent on these critical image regions and diagnostic accuracy. In this paper, a heatmap is generated to represent pathologists' viewing patterns during diagnosis and used to guide a deep learning architecture during training. The proposed system outperforms traditional approaches based on color and texture image characteristics, integrating pathologists' domain expertise to enhance region of interest detection without needing individual case annotations. Evaluating our best model, a U-Net model with a pre-trained ResNet-18 encoder, on a skin biopsy whole slide image dataset for melanoma diagnosis, shows its potential in detecting regions of interest, surpassing conventional methods with an increase of 20%, 11%, 22%, and 12% in precision, recall, F1-score, and Intersection over Union, respectively. In a clinical evaluation, three dermatopathologists agreed on the model's effectiveness in replicating pathologists' diagnostic viewing behavior and accurately identifying critical regions. Finally, our study demonstrates that incorporating heatmaps as supplementary signals can enhance the performance of computer-aided diagnosis systems. Without the availability of eye tracking data, identifying precise focus areas is challenging, but our approach shows promise in assisting pathologists in improving diagnostic accuracy and efficiency, streamlining annotation processes, and aiding the training of new pathologists.

Myalgic Encephalomyelitis/Chronic Fatigue Syndrome After SARS-CoV-2 Infection.

Importance: Chronic symptoms reported following an infection with SARS-CoV-2, such as cognitive problems, overlap with symptoms included in the definition of myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS). Objective: To evaluate the prevalence of ME/CFS-like illness subsequent to acute SARS-CoV-2 infection, changes in ME/CFS symptoms through 12 months of follow-up, and the association of ME/CFS symptoms with SARS-CoV-2 test results at the acute infection-like index illness. Design, Setting, and Participants: This prospective, multisite, longitudinal cohort study (Innovative Support for Patients with SARS-CoV-2 Infections Registry [INSPIRE]) enrolled participants from December 11, 2020, to August 29, 2022. Participants were adults aged 18 to 64 years with acute symptoms suggestive of SARS-CoV-2 infection who received a US Food and Drug Administration-approved SARS-CoV-2 test at the time of illness and did not die or withdraw from the study by 3 months. Follow-up surveys were collected through February 28, 2023. Exposure: COVID-19 status (positive vs negative) at enrollment. Main Outcome and Measures: The main outcome was the weighted proportion of participants with ME/CFS-like illness based on the 2015 Institute of Medicine clinical case definition using self-reported symptoms. Results: A total of 4378 participants were included in the study. Most were female (3226 [68.1%]). Mean (SD) age was 37.8 (11.8) years. The survey completion rates ranged from 38.7% (3613 of 4738 participants) to 76.3% (1835 of 4738) and decreased over time. The weighted proportion of participants identified with ME/CFS-like illness did not change significantly at 3 through 12 months of follow-up and was similar in the COVID-19-positive (range, 2.8%-3.7%) and COVID-19-negative (range, 3.1%-4.5%) groups. Adjusted analyses revealed no significant difference in the odds of ME/CFS-like illness at any time point between COVID-19-positive and COVID-19-negative individuals (marginal odds ratio range, 0.84 [95% CI, 0.42-1.67] to 1.18 [95% CI, 0.55-2.51]). Conclusions and Relevance: In this prospective cohort study, there was no evidence that the proportion of participants with ME/CFS-like illness differed between those infected with SARS-CoV-2 vs those without SARS-CoV-2 infection up to 12 months after infection. A 3% to 4% prevalence of ME/CFS-like illness after an acute infection-like index illness would impose a high societal burden given the millions of persons infected with SARS-CoV-2.

Immunohistochemistry for Diagnosing Melanoma in Older Adults.

Importance: Pathologic assessment to diagnose skin biopsies, especially for cutaneous melanoma, can be challenging, and immunohistochemistry (IHC) staining has the potential to aid decision-making. Currently, the temporal trends regarding the use of IHC for the examination of skin biopsies on a national level have not been described. Objective: To illustrate trends in the use of IHC for the examination of skin biopsies in melanoma diagnoses. Design, Setting, and Participants: A retrospective cross-sectional study was conducted to examine incident cases of melanoma diagnosed between January 2000 and December 2017. The analysis used the SEER-Medicare linked database, incorporating data from 17 population-based registries. The study focused on incident cases of in situ or malignant melanoma of the skin diagnosed in patients 65 years or older. Data were analyzed between August 2022 and November 2023. Main Outcomes and Measures: The main outcomes encompassed the identification of claims for IHC within the month of melanoma diagnoses and extending up to 14 days into the month following diagnosis. The SEER data on patients with melanoma comprised demographic, tumor, and area-level characteristics. Results: The final sample comprised 132 547 melanoma tumors in 116 117 distinct patients. Of the 132 547 melanoma diagnoses meeting inclusion criteria from 2000 to 2017, 43 396 cases had accompanying IHC claims (33%). Among these cases, 28 298 (65%) were diagnosed in male patients, 19 019 (44%) were diagnosed in patients aged 65 years to 74 years, 16 444 (38%) in patients aged 75 years to 84 years, and 7933 (18%) in patients aged 85 years and older. In 2000, 11% of melanoma cases had claims for IHC at or near the time of diagnosis. This proportion increased yearly, with 51% of melanoma cases having associated IHC claims in 2017. Increasing IHC use is observed for all stages of melanoma, including in situ melanoma. Claims for IHC in melanomas increased in all 17 SEER registries but at different rates. In 2017, the use of IHC for melanoma diagnosis ranged from 39% to 68% across registries. Conclusions and Relevance: Considering the dramatically rising and variable use of IHC in diagnosing melanoma by pathologists demonstrated in this retrospective cross-sectional study, further investigation is warranted to understand the clinical utility and discern when IHC most improves diagnostic accuracy or helps patients.

Machine learning classification of diagnostic accuracy in pathologists interpreting breast biopsies.

OBJECTIVE: This study explores the feasibility of using machine learning to predict accurate versus inaccurate diagnoses made by pathologists based on their spatiotemporal viewing behavior when evaluating digital breast biopsy images. MATERIALS AND METHODS: The study gathered data from 140 pathologists of varying experience levels who each reviewed a set of 14 digital whole slide images of breast biopsy tissue. Pathologists' viewing behavior, including zooming and panning actions, was recorded during image evaluation. A total of 30 features were extracted from the viewing behavior data, and 4 machine learning algorithms were used to build classifiers for predicting diagnostic accuracy. RESULTS: The Random Forest classifier demonstrated the best overall performance, achieving a test accuracy of 0.81 and area under the receiver-operator characteristic curve of 0.86. Features related to attention distribution and focus on critical regions of interest were found to be important predictors of diagnostic accuracy. Further including case-level and pathologist-level information incrementally improved classifier performance. DISCUSSION: Results suggest that pathologists' viewing behavior during digital image evaluation can be leveraged to predict diagnostic accuracy, affording automated feedback and decision support systems based on viewing behavior to aid in training and, ultimately, clinical practice. They also carry implications for basic research examining the interplay between perception, thought, and action in diagnostic decision-making. CONCLUSION: The classifiers developed herein have potential applications in training and clinical settings to provide timely feedback and support to pathologists during diagnostic decision-making. Further research could explore the generalizability of these findings to other medical domains and varied levels of expertise.

Artificial Intelligence-Driven Mammography-Based Future Breast Cancer Risk Prediction: A Systematic Review.

PURPOSE: To summarize the literature regarding the performance of mammography-image based artificial intelligence (AI) algorithms, with and without additional clinical data, for future breast cancer risk prediction. MATERIALS AND METHODS: A systematic literature review was performed using six databases (medRixiv, bioRxiv, Embase, Engineer Village, IEEE Xplore, and PubMed) from 2012 through September 30, 2022. Studies were included if they used real-world screening mammography examinations to validate AI algorithms for future risk prediction based on images alone or in combination with clinical risk factors. The quality of studies was assessed, and predictive accuracy was recorded as the area under the receiver operating characteristic curve (AUC). RESULTS: Sixteen studies met inclusion and exclusion criteria, of which 14 studies provided AUC values. The median AUC performance of AI image-only models was 0.72 (range 0.62-0.90) compared with 0.61 for breast density or clinical risk factor-based tools (range 0.54-0.69). Of the seven studies that compared AI image-only performance directly to combined image + clinical risk factor performance, six demonstrated no significant improvement, and one study demonstrated increased improvement. CONCLUSIONS: Early efforts for predicting future breast cancer risk based on mammography images alone demonstrate comparable or better accuracy to traditional risk tools with little or no improvement when adding clinical risk factor data. Transitioning from clinical risk factor-based to AI image-based risk models may lead to more accurate, personalized risk-based screening approaches.

Pathologist Characteristics Associated With Rendering Higher-Grade Diagnoses for Melanocytic Lesions.

Importance: The incidence of melanoma diagnoses has been increasing in recent decades, and controlled studies have indicated high histopathologic discordance across the intermediate range of melanocytic lesions. The respective causes for these phenomena remain incompletely understood. Objective: To identify pathologist characteristics associated with tendencies to diagnose melanocytic lesions as higher grade vs lower grade or to diagnose invasive melanoma vs any less severe diagnosis. Design, Setting, and Participants: This exploratory study used data from 2 nationwide studies (the Melanoma Pathology [M-Path] study, conducted from July 2013 to May 2016, and the Reducing Errors in Melanocytic Interpretations [REMI] study, conducted from August 2018 to March 2021) in which participating pathologists who interpreted melanocytic lesions in their clinical practices interpreted study cases in glass slide format. Each pathologist was randomly assigned to interpret a set of study cases from a repository of skin biopsy samples of melanocytic lesions; each case was independently interpreted by multiple pathologists. Data were analyzed from July 2022 to February 2023. Main Outcomes and Measures: The association of pathologist characteristics with diagnosis of a study case as higher grade (including severely dysplastic and melanoma in situ) vs lower grade (including mild to moderately dysplastic nevi) and diagnosis of invasive melanoma vs any less severe diagnosis was assessed using logistic regression. Characteristics included demographics (age, gender, and geographic region), years of experience, academic affiliation, caseload of melanocytic lesions in their practice, specialty training, and history of malpractice suits. Results: A total of 338 pathologists were included: 113 general pathologists and 74 dermatopathologists from M-Path and 151 dermatopathologists from REMI. The predominant factor associated with rendering more severe diagnoses was specialist training in dermatopathology (board certification and/or fellowship training). Pathologists with this training were more likely to render higher-grade diagnoses (odds ratio [OR], 2.63; 95% CI, 2.10-3.30; P < .001) and to diagnose invasive melanoma (OR, 1.95; 95% CI, 1.53-2.49; P < .001) than pathologists without this training interpreting the same case. Nonmitogenic pT1a diagnoses (stage pT1a melanomas with no mitotic activity) accounted for the observed difference in diagnosis of invasive melanoma; when these lesions, which carry a low risk of metastasis, were grouped with the less severe diagnoses, there was no observed association (OR, 0.95; 95% CI, 0.74-1.23; P = .71). Among dermatopathologists, those with a higher caseload of melanocytic lesions in their practice were more likely to assign higher-grade diagnoses (OR for trend, 1.27; 95% CI, 1.04-1.56; P = .02). Conclusions and Relevance: The findings suggest that specialty training in dermatopathology is associated with a greater tendency to diagnose atypical melanocytic proliferations as pT1a melanomas. These low-risk melanomas constitute a growing proportion of melanomas diagnosed in the US.

Implementing the Melanocytic Pathology Assessment Tool and Hierarchy for Diagnosis: Long-term effect of a simple educational intervention.

Background: A standardized pathology management tool for melanocytic skin lesions may improve patient care by simplifying interpretation and categorization of the diverse terminology currently extant. Objective: To assess an online educational intervention that teaches dermatopathologists to use the Melanocytic Pathology Assessment Tool and Hierarchy for Diagnosis (MPATH-Dx), a schema collapsing multiple diagnostic terms into 5 classes ranging from benign to invasive melanoma. Methods: Practicing dermatopathologists (N = 149) from 40 US states participated in a 2-year educational intervention study (71% response rate). The intervention involved a brief tutorial followed by practice on 28 melanocytic lesions, with the goal of teaching pathologists how to correctly use the MPATH-Dx schema; competence using the MPATH-Dx tool 12-24 months postintervention was assessed. Participants' self-reported confidence using the MPATH-Dx tool was assessed preintervention and postintervention. Results: At preintervention, confidence using the MPATH-Dx tool was already high, despite 68% lacking prior familiarity with it, and confidence increased postintervention (P = .0003). During the intervention, participants used the MPATH-Dx tool correctly for 90% of their interpretations; postintervention, participants used the MPATH-Dx tool correctly for 88% of their interpretations. Limitations: Future research should examine implementing a standardized pathology assessment schema in actual clinical practice. Conclusion: Dermatopathologists can be taught to confidently and competently use the MPATH-Dx schema with a simple educational tutorial followed by practice.

Pathologist pupil dilation reflects experience level and difficulty in diagnosing medical images.

Purpose: Digital whole slide imaging allows pathologists to view slides on a computer screen instead of under a microscope. Digital viewing allows for real-time monitoring of pathologists' search behavior and neurophysiological responses during the diagnostic process. One particular neurophysiological measure, pupil diameter, could provide a basis for evaluating clinical competence during training or developing tools that support the diagnostic process. Prior research shows that pupil diameter is sensitive to cognitive load and arousal, and it switches between exploration and exploitation of a visual image. Different categories of lesions in pathology pose different levels of challenge, as indicated by diagnostic disagreement among pathologists. If pupil diameter is sensitive to the perceived difficulty in diagnosing biopsies, eye-tracking could potentially be used to identify biopsies that may benefit from a second opinion. Approach: We measured case onset baseline-corrected (phasic) and uncorrected (tonic) pupil diameter in 90 pathologists who each viewed and diagnosed 14 digital breast biopsy cases that cover the diagnostic spectrum from benign to invasive breast cancer. Pupil data were extracted from the beginning of viewing and interpreting of each individual case. After removing 122 trials ( < 10 % ) with poor eye-tracking quality, 1138 trials remained. We used multiple linear regression with robust standard error estimates to account for dependent observations within pathologists. Results: We found a positive association between the magnitude of phasic dilation and subject-centered difficulty ratings and between the magnitude of tonic dilation and untransformed difficulty ratings. When controlling for case diagnostic category, only the tonic-difficulty relationship persisted. Conclusions: Results suggest that tonic pupil dilation may indicate overall arousal differences between pathologists as they interpret biopsy cases and could signal a need for additional training, experience, or automated decision aids. Phasic dilation is sensitive to characteristics of biopsies that tend to elicit higher difficulty ratings and could indicate a need for a second opinion.

Zoom behavior during visual search modulates pupil diameter and reflects adaptive control states.

Adaptive gain theory proposes that the dynamic shifts between exploration and exploitation control states are modulated by the locus coeruleus-norepinephrine system and reflected in tonic and phasic pupil diameter. This study tested predictions of this theory in the context of a societally important visual search task: the review and interpretation of digital whole slide images of breast biopsies by physicians (pathologists). As these medical images are searched, pathologists encounter difficult visual features and intermittently zoom in to examine features of interest. We propose that tonic and phasic pupil diameter changes during image review may correspond to perceived difficulty and dynamic shifts between exploration and exploitation control states. To examine this possibility, we monitored visual search behavior and tonic and phasic pupil diameter while pathologists (N = 89) interpreted 14 digital images of breast biopsy tissue (1,246 total images reviewed). After viewing the images, pathologists provided a diagnosis and rated the level of difficulty of the image. Analyses of tonic pupil diameter examined whether pupil dilation was associated with pathologists' difficulty ratings, diagnostic accuracy, and experience level. To examine phasic pupil diameter, we parsed continuous visual search data into discrete zoom-in and zoom-out events, including shifts from low to high magnification (e.g., 1× to 10×) and the reverse. Analyses examined whether zoom-in and zoom-out events were associated with phasic pupil diameter change. Results demonstrated that tonic pupil diameter was associated with image difficulty ratings and zoom level, and phasic pupil diameter showed constriction upon zoom-in events, and dilation immediately preceding a zoom-out event. Results are interpreted in the context of adaptive gain theory, information gain theory, and the monitoring and assessment of physicians' diagnostic interpretive processes.

VSGD-Net: Virtual Staining Guided Melanocyte Detection on Histopathological Images.

Detection of melanocytes serves as a critical prerequisite in assessing melanocytic growth patterns when diagnosing melanoma and its precursor lesions on skin biopsy specimens. However, this detection is challenging due to the visual similarity of melanocytes to other cells in routine Hematoxylin and Eosin (H&E) stained images, leading to the failure of current nuclei detection methods. Stains such as Sox10 can mark melanocytes, but they require an additional step and expense and thus are not regularly used in clinical practice. To address these limitations, we introduce VSGD-Net, a novel detection network that learns melanocyte identification through virtual staining from H&E to Sox10. The method takes only routine H&E images during inference, resulting in a promising approach to support pathologists in the diagnosis of melanoma. To the best of our knowledge, this is the first study that investigates the detection problem using image synthesis features between two distinct pathology stainings. Extensive experimental results show that our proposed model outperforms state-of-the-art nuclei detection methods for melanocyte detection. The source code and pre-trained model are available at: https://github.com/kechunl/VSGD-Net.

From Image to Diagnosis: Characterizing Sources of Error in Histopathologic Interpretation.

An accurate histopathologic diagnosis on surgical biopsy material is necessary for the clinical management of patients and has important implications for research, clinical trial design/enrollment, and public health education. This study used a mixed methods approach to isolate sources of diagnostic error while residents and attending pathologists interpreted digitized breast biopsy slides. Ninety participants, including pathology residents and attending physicians at major United States medical centers reviewed a set of 14 digitized whole-slide images of breast biopsies. Each case had a consensus-defined diagnosis and critical region of interest (cROI) representing the most significant pathology on the slide. Participants were asked to view unmarked digitized slides, draw their participant region of interest (pROI), describe its features, and render a diagnosis. Participants' review behavior was tracked using case viewer software and an eye-tracking device. Diagnostic accuracy was calculated in comparison to the consensus diagnosis. We measured the frequency of errors emerging during 4 interpretive phases: (1) detecting the cROI, (2) recognizing its relevance, (3) using the correct terminology to describe findings in the pROI, and (4) making a diagnostic decision. According to eye-tracking data, trainees and attending pathologists were very likely (∼94% of the time) to find the cROI when inspecting a slide. However, trainees were less likely to consider the cROI relevant to their diagnosis. Pathology trainees (41% of cases) were more likely to use incorrect terminology to describe pROI features than attending pathologists (21% of cases). Failure to accurately describe features was the only factor strongly associated with an incorrect diagnosis. Identifying where errors emerge in the interpretive and/or descriptive process and working on building organ-specific feature recognition and verbal fluency in describing those features are critical steps for achieving competency in diagnostic decision making.

Variability Among Breast Cancer Risk Classification Models When Applied at the Level of the Individual Woman.

BACKGROUND: Breast cancer risk models guide screening and chemoprevention decisions, but the extent and effect of variability among models, particularly at the individual level, is uncertain. OBJECTIVE: To quantify the accuracy and disagreement between commonly used risk models in categorizing individual women as average vs. high risk for developing invasive breast cancer. DESIGN: Comparison of three risk prediction models: Breast Cancer Risk Assessment Tool (BCRAT), Breast Cancer Surveillance Consortium (BCSC) model, and International Breast Intervention Study (IBIS) model. SUBJECTS: Women 40 to 74 years of age presenting for screening mammography at a multisite health system between 2011 and 2015, with 5-year follow-up for cancer outcome. MAIN MEASURES: Comparison of model discrimination and calibration at the population level and inter-model agreement for 5-year breast cancer risk at the individual level using two cutoffs (≥ 1.67% and ≥ 3.0%). KEY RESULTS: A total of 31,115 women were included. When using the ≥ 1.67% threshold, more than 21% of women were classified as high risk for developing breast cancer in the next 5 years by one model, but average risk by another model. When using the ≥ 3.0% threshold, more than 5% of women had disagreements in risk severity between models. Almost half of the women (46.6%) were classified as high risk by at least one of the three models (e.g., if all three models were applied) for the threshold of ≥ 1.67%, and 11.1% were classified as high risk for ≥ 3.0%. All three models had similar accuracy at the population level. CONCLUSIONS: Breast cancer risk estimates for individual women vary substantially, depending on which risk assessment model is used. The choice of cutoff used to define high risk can lead to adverse effects for screening, preventive care, and quality of life for misidentified individuals. Clinicians need to be aware of the high false-positive and false-negative rates and variation between models when talking with patients.

Diagnostic error, uncertainty, and overdiagnosis in melanoma.

Diagnostic error can be defined as deviation from a gold standard diagnosis, typically defined in terms of expert opinion, although sometimes in terms of unexpected events that might occur in follow-up (such as progression and death from disease). Although diagnostic error does exist for melanoma, deviations from gold standard diagnosis, certainly among appropriately trained and experienced practitioners, are likely to be the result of uncertainty and lack of specific criteria, and differences of opinion, rather than lack of diagnostic skills. In this review, the concept of diagnostic error will be considered in relation to diagnostic uncertainty, and the concept of overdiagnosis in melanoma will be presented and discussed.

Revision of the Melanocytic Pathology Assessment Tool and Hierarchy for Diagnosis Classification Schema for Melanocytic Lesions: A Consensus Statement.

Importance: A standardized pathology classification system for melanocytic lesions is needed to aid both pathologists and clinicians in cataloging currently existing diverse terminologies and in the diagnosis and treatment of patients. The Melanocytic Pathology Assessment Tool and Hierarchy for Diagnosis (MPATH-Dx) has been developed for this purpose. Objective: To revise the MPATH-Dx version 1.0 classification tool, using feedback from dermatopathologists participating in the National Institutes of Health-funded Reducing Errors in Melanocytic Interpretations (REMI) Study and from members of the International Melanoma Pathology Study Group (IMPSG). Evidence Review: Practicing dermatopathologists recruited from 40 US states participated in the 2-year REMI study and provided feedback on the MPATH-Dx version 1.0 tool. Independently, member dermatopathologists participating in an IMPSG workshop dedicated to the MPATH-Dx schema provided additional input for refining the MPATH-Dx tool. A reference panel of 3 dermatopathologists, the original authors of the MPATH-Dx version 1.0 tool, integrated all feedback into an updated and refined MPATH-Dx version 2.0. Findings: The new MPATH-Dx version 2.0 schema simplifies the original 5-class hierarchy into 4 classes to improve diagnostic concordance and to provide more explicit guidance in the treatment of patients. This new version also has clearly defined histopathological criteria for classification of classes I and II lesions; has specific provisions for the most frequently encountered low-cumulative sun damage pathway of melanoma progression, as well as other, less common World Health Organization pathways to melanoma; provides guidance for classifying intermediate class II tumors vs melanoma; and recognizes a subset of pT1a melanomas with very low risk and possible eventual reclassification as neoplasms lacking criteria for melanoma. Conclusions and Relevance: The implementation of the newly revised MPATH-Dx version 2.0 schema into clinical practice is anticipated to provide a robust tool and adjunct for standardized diagnostic reporting of melanocytic lesions and management of patients to the benefit of both health care practitioners and patients.

Are Pathologists Self-Aware of Their Diagnostic Accuracy? Metacognition and the Diagnostic Process in Pathology.

BACKGROUND: Metacognition is a cognitive process that involves self-awareness of thinking, understanding, and performance. This study assesses pathologists' metacognition by examining the association between their diagnostic accuracy and self-reported confidence levels while interpreting skin and breast biopsies. DESIGN: We studied 187 pathologists from the Melanoma Pathology Study (M-Path) and 115 pathologists from the Breast Pathology Study (B-Path). We measured pathologists' metacognitive ability by examining the area under the curve (AUC), the area under each pathologist's receiver operating characteristic (ROC) curve summarizing the association between confidence and diagnostic accuracy. We investigated possible relationships between this AUC measure, referred to as metacognitive sensitivity, and pathologist attributes. We also assessed whether higher metacognitive sensitivity affected the association between diagnostic accuracy and a secondary diagnostic action such as requesting a second opinion. RESULTS: We found no significant associations between pathologist clinical attributes and metacognitive AUC. However, we found that pathologists with higher AUC showed a stronger trend to request secondary diagnostic action for inaccurate diagnoses and not for accurate diagnoses compared with pathologists with lower AUC. LIMITATIONS: Pathologists reported confidence in specific diagnostic terms, rather than the broader classes into which the diagnostic terms were later grouped to determine accuracy. In addition, while there is no gold standard for the correct diagnosis to determine the accuracy of pathologists' interpretations, our studies achieved a high-quality reference diagnosis by using the consensus diagnosis of 3 experienced pathologists. CONCLUSIONS: Metacognition can affect clinical decisions. If pathologists have self-awareness that their diagnosis may be inaccurate, they can request additional tests or second opinions, providing the opportunity to correct inaccurate diagnoses. HIGHLIGHTS: Metacognitive sensitivity varied across pathologists, with most showing higher sensitivity than expected by chance.None of the demographic or clinical characteristics we examined was significantly associated with metacognitive sensitivity.Pathologists with higher metacognitive sensitivity were more likely to request additional tests or second opinions for their inaccurate diagnoses.

Prognostic modeling of cutaneous melanoma stage I patients using cancer registry data identifies subsets with very-low melanoma mortality.

BACKGROUND: Evidence exists that escalating melanoma incidence is due in part to overdiagnosis, the diagnosis of lesions that will not lead to symptoms or death. The authors aimed to characterize subsets of melanoma patients with very-low risk of death that may be contributing to overdiagnosis. METHODS: Melanoma patients diagnosed in 2010 and 2011 with stage I lesions ≤1.0 mm thick and negative clinical lymph nodes from the Surveillance, Epidemiology, and End Results database were selected. Classification and regression tree and logistic regression models were developed and validated to identify patients with very-low risk of death from melanoma within 7 years. Logistic models were also used to identify patients at higher risk of death among this group of stage I patients. RESULTS: Compared to an overall 7-year mortality from melanoma of 2.5% in these patients, a subset comprising 25% had a risk below 1%. Younger age at diagnosis and Clark level II were associated with low risk of death in all models. Breslow thickness below 0.4 mm, absence of mitogenicity, absence of ulceration, and female sex were also associated with lower mortality. A small subset of high-risk patients with >20% risk of death was also identified. CONCLUSION: Patients with very-low risk of dying from melanoma within 7 years of diagnosis were identified. Such cases warrant further study and consensus discussion to develop classification criteria, with the potential to be categorized using an alternative term such as "melanocytic neoplasms of low malignant potential." LAY SUMMARY: Although melanoma is the most serious skin cancer, most melanoma patients have high chances of survival. There is evidence that some lesions diagnosed as melanoma would never have caused symptoms or death, a phenomenon known as overdiagnosis. In this study, we used cancer registry data to identify a subset of early-stage melanoma patients with almost no melanoma deaths. Using two statistical approaches, we identified patients with <1% risk of dying from melanoma in 7 years. Such patients tended to be younger with minimal invasion into the skin. We also identified a very small patient subset with higher mortality risk.

External Validation of an Ensemble Model for Automated Mammography Interpretation by Artificial Intelligence.

Importance: With a shortfall in fellowship-trained breast radiologists, mammography screening programs are looking toward artificial intelligence (AI) to increase efficiency and diagnostic accuracy. External validation studies provide an initial assessment of how promising AI algorithms perform in different practice settings. Objective: To externally validate an ensemble deep-learning model using data from a high-volume, distributed screening program of an academic health system with a diverse patient population. Design, Setting, and Participants: In this diagnostic study, an ensemble learning method, which reweights outputs of the 11 highest-performing individual AI models from the Digital Mammography Dialogue on Reverse Engineering Assessment and Methods (DREAM) Mammography Challenge, was used to predict the cancer status of an individual using a standard set of screening mammography images. This study was conducted using retrospective patient data collected between 2010 and 2020 from women aged 40 years and older who underwent a routine breast screening examination and participated in the Athena Breast Health Network at the University of California, Los Angeles (UCLA). Main Outcomes and Measures: Performance of the challenge ensemble method (CEM) and the CEM combined with radiologist assessment (CEM+R) were compared with diagnosed ductal carcinoma in situ and invasive cancers within a year of the screening examination using performance metrics, such as sensitivity, specificity, and area under the receiver operating characteristic curve (AUROC). Results: Evaluated on 37 317 examinations from 26 817 women (mean [SD] age, 58.4 [11.5] years), individual model AUROC estimates ranged from 0.77 (95% CI, 0.75-0.79) to 0.83 (95% CI, 0.81-0.85). The CEM model achieved an AUROC of 0.85 (95% CI, 0.84-0.87) in the UCLA cohort, lower than the performance achieved in the Kaiser Permanente Washington (AUROC, 0.90) and Karolinska Institute (AUROC, 0.92) cohorts. The CEM+R model achieved a sensitivity (0.813 [95% CI, 0.781-0.843] vs 0.826 [95% CI, 0.795-0.856]; P = .20) and specificity (0.925 [95% CI, 0.916-0.934] vs 0.930 [95% CI, 0.929-0.932]; P = .18) similar to the radiologist performance. The CEM+R model had significantly lower sensitivity (0.596 [95% CI, 0.466-0.717] vs 0.850 [95% CI, 0.766-0.923]; P < .001) and specificity (0.803 [95% CI, 0.734-0.861] vs 0.945 [95% CI, 0.936-0.954]; P < .001) than the radiologist in women with a prior history of breast cancer and Hispanic women (0.894 [95% CI, 0.873-0.910] vs 0.926 [95% CI, 0.919-0.933]; P = .004). Conclusions and Relevance: This study found that the high performance of an ensemble deep-learning model for automated screening mammography interpretation did not generalize to a more diverse screening cohort, suggesting that the model experienced underspecification. This study suggests the need for model transparency and fine-tuning of AI models for specific target populations prior to their clinical adoption.

An analysis of pathologists' viewing processes as they diagnose whole slide digital images.

Although pathologists have their own viewing habits while diagnosing, viewing behaviors leading to the most accurate diagnoses are under-investigated. Digital whole slide imaging has enabled investigators to analyze pathologists' visual interpretation of histopathological features using mouse and viewport tracking techniques. In this study, we provide definitions for basic viewing behavior variables and investigate the association of pathologists' characteristics and viewing behaviors, and how they relate to diagnostic accuracy when interpreting whole slide images. We use recordings of 32 pathologists' actions while interpreting a set of 36 digital whole slide skin biopsy images (5 sets of 36 cases; 180 cases total). These viewport tracking data include the coordinates of a viewport scene on pathologists' screens, the magnification level at which that viewport was viewed, as well as a timestamp. We define a set of variables to quantify pathologists' viewing behaviors such as zooming, panning, and interacting with a consensus reference panel's selected region of interest (ROI). We examine the association of these viewing behaviors with pathologists' demographics, clinical characteristics, and diagnostic accuracy using cross-classified multilevel models. Viewing behaviors differ based on clinical experience of the pathologists. Pathologists with a higher caseload of melanocytic skin biopsy cases and pathologists with board certification and/or fellowship training in dermatopathology have lower average zoom and lower variance of zoom levels. Viewing behaviors associated with higher diagnostic accuracy include higher average and variance of zoom levels, a lower magnification percentage (a measure of consecutive zooming behavior), higher total interpretation time, and higher amount of time spent viewing ROIs. Scanning behavior, which refers to panning with a fixed zoom level, has marginally significant positive association with accuracy. Pathologists' training, clinical experience, and their exposure to a range of cases are associated with their viewing behaviors, which may contribute to their diagnostic accuracy. Research in computational pathology integrating digital imaging and clinical informatics opens up new avenues for leveraging viewing behaviors in medical education and training, potentially improving patient care and the effectiveness of clinical workflow.