RESUMEN
Artificial intelligence (AI)-based diagnostic tools can offer numerous benefits to the field of histopathology, including improved diagnostic accuracy, efficiency and productivity. As a result, such tools are likely to have an increasing role in routine practice. However, all AI tools are prone to errors, and these AI-associated errors have been identified as a major risk in the introduction of AI into healthcare. The errors made by AI tools are different, in terms of both cause and nature, to the errors made by human pathologists. As highlighted by the National Institute for Health and Care Excellence, it is imperative that practising pathologists understand the potential limitations of AI tools, including the errors made. Pathologists are in a unique position to be gatekeepers of AI tool use, maximizing patient benefit while minimizing harm. Furthermore, their pathological knowledge is essential to understanding when, and why, errors have occurred and so to developing safer future algorithms. This paper summarises the literature on errors made by AI diagnostic tools in histopathology. These include erroneous errors, data concerns (data bias, hidden stratification, data imbalances, distributional shift, and lack of generalisability), reinforcement of outdated practices, unsafe failure mode, automation bias, and insensitivity to impact. Methods to reduce errors in both tool design and clinical use are discussed, and the practical roles for pathologists in error minimisation are highlighted. This aims to inform and empower pathologists to move safely through this seismic change in practice and help ensure that novel AI tools are adopted safely.
Asunto(s)
Inteligencia Artificial , Patólogos , Humanos , AlgoritmosRESUMEN
AIMS: To conduct a definitive multicentre comparison of digital pathology (DP) with light microscopy (LM) for reporting histopathology slides including breast and bowel cancer screening samples. METHODS: A total of 2024 cases (608 breast, 607 GI, 609 skin, 200 renal) were studied, including 207 breast and 250 bowel cancer screening samples. Cases were examined by four pathologists (16 study pathologists across the four speciality groups), using both LM and DP, with the order randomly assigned and 6 weeks between viewings. Reports were compared for clinical management concordance (CMC), meaning identical diagnoses plus differences which do not affect patient management. Percentage CMCs were computed using logistic regression models with crossed random-effects terms for case and pathologist. The obtained percentage CMCs were referenced to 98.3% calculated from previous studies. RESULTS: For all cases LM versus DP comparisons showed the CMC rates were 99.95% [95% confidence interval (CI) = 99.90-99.97] and 98.96 (95% CI = 98.42-99.32) for cancer screening samples. In speciality groups CMC for LM versus DP showed: breast 99.40% (99.06-99.62) overall and 96.27% (94.63-97.43) for cancer screening samples; [gastrointestinal (GI) = 99.96% (99.89-99.99)] overall and 99.93% (99.68-99.98) for bowel cancer screening samples; skin 99.99% (99.92-100.0); renal 99.99% (99.57-100.0). Analysis of clinically significant differences revealed discrepancies in areas where interobserver variability is known to be high, in reads performed with both modalities and without apparent trends to either. CONCLUSIONS: Comparing LM and DP CMC, overall rates exceed the reference 98.3%, providing compelling evidence that pathologists provide equivalent results for both routine and cancer screening samples irrespective of the modality used.
Asunto(s)
Neoplasias de la Mama , Neoplasias Colorrectales , Patología Clínica , Humanos , Detección Precoz del Cáncer , Interpretación de Imagen Asistida por Computador/métodos , Microscopía/métodos , Patología Clínica/métodos , Femenino , Estudios Multicéntricos como AsuntoRESUMEN
As digital pathology replaces conventional glass slide microscopy as a means of reporting cellular pathology samples, the annotation of digital pathology whole slide images is rapidly becoming part of a pathologist's regular practice. Currently, there is no recognizable organization of these annotations, and as a result, pathologists adopt an arbitrary approach to defining regions of interest, leading to irregularity and inconsistency and limiting the downstream efficient use of this valuable effort. In this study, we propose a Standardized Annotation Reporting Style for digital whole slide images. We formed a list of 167 commonly annotated entities (under 12 specialty subcategories) based on review of Royal College of Pathologists and College of American Pathologists documents, feedback from reporting pathologists in our NHS department, and experience in developing annotation dictionaries for PathLAKE research projects. Each entity was assigned a suitable annotation shape, SNOMED CT (SNOMED International) code, and unique color. Additionally, as an example of how the approach could be expanded to specific tumor types, all lung tumors in the fifth World Health Organization of thoracic tumors 2021 were included. The proposed standardization of annotations increases their utility, making them identifiable at low power and searchable across and between cases. This would aid pathologists reporting and reviewing cases and enable annotations to be used for research. This structured approach could serve as the basis for an industry standard and be easily adopted to ensure maximum functionality and efficiency in the use of annotations made during routine clinical examination of digital slides.
Asunto(s)
Patología Clínica , Patología Quirúrgica , Neoplasias Torácicas , Humanos , Patología Clínica/métodos , Patología Quirúrgica/métodos , Patólogos , Microscopía/métodosRESUMEN
An increasing number of artificial intelligence (AI) tools are moving towards the clinical realm in histopathology and across medicine. The introduction of such tools will bring several benefits to diagnostic specialities, namely increased diagnostic accuracy and efficiency, however, as no AI tool is infallible, their use will inevitably introduce novel errors. These errors made by AI tools are, most fundamentally, misclassifications made by a computational algorithm. Understanding of how these translate into clinical impact on patients is often lacking, meaning true reporting of AI tool safety is incomplete. In this Perspective we consider AI diagnostic tools in histopathology, which are predominantly assessed in terms of technical performance metrics such as sensitivity, specificity and area under the receiver operating characteristic curve. Although these metrics are essential and allow tool comparison, they alone give an incomplete picture of how an AI tool's errors could impact a patient's diagnosis, management and prognosis. We instead suggest assessing and reporting AI tool errors from a pathological and clinical stance, demonstrating how this is done in studies on human pathologist errors, and giving examples where available from pathology and radiology. Although this seems a significant task, we discuss ways to move towards this approach in terms of study design, guidelines and regulation. This Perspective seeks to initiate broader consideration of the assessment of AI tool errors in histopathology and across diagnostic specialities, in an attempt to keep patient safety at the forefront of AI tool development and facilitate safe clinical deployment.
RESUMEN
Histopathology is a challenging interpretive discipline, and the level of confidence a pathologist has in their diagnosis is known to vary, which is conveyed descriptively in pathology reports. There has been little study to accurately quantify pathologists' diagnostic confidence or the factors that influence it. In this study involving sixteen pathologists from six NHS trusts, we assessed diagnostic confidence across multiple variables and four specialties. Each case was reported by four pathologists, with each pathologist reporting each case twice (on light microscopy (LM) and digital pathology (DP)). For each diagnosis, pathologists recorded their confidence on a 7-point Likert scale. This provided 16,187 diagnoses and associated confidence scores for analysis. All variables investigated were found to be significantly predictive of diagnostic confidence, except level of pathologist experience. Confidence was lower for difficult to report cases, cases where there was inter- and intra-pathologist variation in the diagnosis, and cases where the pathologist made an incorrect diagnosis. Confidence was higher, although nominally, for LM diagnoses than DP (rate ratio 1.09 (95% CI 1.01-1.18), p = 0.035), although results indicate pathologists are confident to report on DP. Lowest confidence scores were seen in areas of known diagnostic complexity and cases with quality issues. High confidence in incorrect diagnoses were almost invariably attributed to interpretive diagnostic differences which occurred across both rare and common lesions. The results highlight the value of external quality control schemes and the benefits of selective peer review when reporting.
RESUMEN
The archiving of whole slide images represents a hurdle to digital pathology implementation largely because of the amount of data generated. The retention of glass slides is currently recommended for a minimum of 10 years, but it is for individual departments to determine how digital images are archived and for how long. In a retrospective study, we examined the combination of Systemised Nomenclature of Medicine (SNOMED) codes allocated to cases reported between July 2011 and December 2015 and recalled more than 12 months after diagnosis in comparison to non-recalled cases.Our results show that 0.2% of cases are recalled after 12 months, and SNOMED code combinations can be used to identify which cases are likely to be recalled and which are not. This approach could reduce the number of cases archived by 62% and still ensure all cases likely to be recalled remain in the archive.
Asunto(s)
Systematized Nomenclature of Medicine , Humanos , Estudios RetrospectivosRESUMEN
Immune checkpoint blockade (ICB) drugs are a novel, effective treatment for advanced urothelial carcinoma. Worldwide, several different ICB drugs are approved, each developed and clinically validated with a specific PD-L1 compound diagnostic assay. As a result, PD-L1 testing workflows in routine practice are complex: requiring multiple assays across two platforms, with each assay having a different method of interpretation. Our service tested 1,401 urothelial carcinoma cases for PD-L1 expression, using both the 22C3 PharmDx assay (required prior to Pembrolizumab therapy) and SP142 assay (required prior to Atezolizumab therapy). Of the 1,401 cases tested, 621 cases (44%) were tested with both the 22C3 PharmDx and SP142 assays, 492 cases (35%) with 22C3 PharmDx only, and 288 cases (21%) with SP142 only. Each assay was used and interpreted according to the manufacturer's guidelines. The rate of positivity we observed was 26% with the 22C3 assay and 31% with the SP142 assay, similar to the pre-licensing studies for both drugs. The discrepancy observed between the assays was 11%, which reinforces the requirement for utilisation of the correct assay for each agent, and limits potential cross-utility of assays. This aspect must be considered when setting up a PD-L1 testing strategy in laboratories where both Pembrolizumab and Atezolizumab are available for the treatment of urothelial carcinoma but also has broader implications for testing of other cancers where multiple ICB drugs and their respective assays are approved.