Understanding the errors made by artificial intelligence algorithms in histopathology in terms of patient impact.

An increasing number of artificial intelligence (AI) tools are moving towards the clinical realm in histopathology and across medicine. The introduction of such tools will bring several benefits to diagnostic specialities, namely increased diagnostic accuracy and efficiency, however, as no AI tool is infallible, their use will inevitably introduce novel errors. These errors made by AI tools are, most fundamentally, misclassifications made by a computational algorithm. Understanding of how these translate into clinical impact on patients is often lacking, meaning true reporting of AI tool safety is incomplete. In this Perspective we consider AI diagnostic tools in histopathology, which are predominantly assessed in terms of technical performance metrics such as sensitivity, specificity and area under the receiver operating characteristic curve. Although these metrics are essential and allow tool comparison, they alone give an incomplete picture of how an AI tool's errors could impact a patient's diagnosis, management and prognosis. We instead suggest assessing and reporting AI tool errors from a pathological and clinical stance, demonstrating how this is done in studies on human pathologist errors, and giving examples where available from pathology and radiology. Although this seems a significant task, we discuss ways to move towards this approach in terms of study design, guidelines and regulation. This Perspective seeks to initiate broader consideration of the assessment of AI tool errors in histopathology and across diagnostic specialities, in an attempt to keep patient safety at the forefront of AI tool development and facilitate safe clinical deployment.

Digital pathology for reporting histopathology samples, including cancer screening samples - definitive evidence from a multisite study.

AIMS: To conduct a definitive multicentre comparison of digital pathology (DP) with light microscopy (LM) for reporting histopathology slides including breast and bowel cancer screening samples. METHODS: A total of 2024 cases (608 breast, 607 GI, 609 skin, 200 renal) were studied, including 207 breast and 250 bowel cancer screening samples. Cases were examined by four pathologists (16 study pathologists across the four speciality groups), using both LM and DP, with the order randomly assigned and 6 weeks between viewings. Reports were compared for clinical management concordance (CMC), meaning identical diagnoses plus differences which do not affect patient management. Percentage CMCs were computed using logistic regression models with crossed random-effects terms for case and pathologist. The obtained percentage CMCs were referenced to 98.3% calculated from previous studies. RESULTS: For all cases LM versus DP comparisons showed the CMC rates were 99.95% [95% confidence interval (CI) = 99.90-99.97] and 98.96 (95% CI = 98.42-99.32) for cancer screening samples. In speciality groups CMC for LM versus DP showed: breast 99.40% (99.06-99.62) overall and 96.27% (94.63-97.43) for cancer screening samples; [gastrointestinal (GI) = 99.96% (99.89-99.99)] overall and 99.93% (99.68-99.98) for bowel cancer screening samples; skin 99.99% (99.92-100.0); renal 99.99% (99.57-100.0). Analysis of clinically significant differences revealed discrepancies in areas where interobserver variability is known to be high, in reads performed with both modalities and without apparent trends to either. CONCLUSIONS: Comparing LM and DP CMC, overall rates exceed the reference 98.3%, providing compelling evidence that pathologists provide equivalent results for both routine and cancer screening samples irrespective of the modality used.

Why do errors arise in artificial intelligence diagnostic tools in histopathology and how can we minimize them?

Artificial intelligence (AI)-based diagnostic tools can offer numerous benefits to the field of histopathology, including improved diagnostic accuracy, efficiency and productivity. As a result, such tools are likely to have an increasing role in routine practice. However, all AI tools are prone to errors, and these AI-associated errors have been identified as a major risk in the introduction of AI into healthcare. The errors made by AI tools are different, in terms of both cause and nature, to the errors made by human pathologists. As highlighted by the National Institute for Health and Care Excellence, it is imperative that practising pathologists understand the potential limitations of AI tools, including the errors made. Pathologists are in a unique position to be gatekeepers of AI tool use, maximizing patient benefit while minimizing harm. Furthermore, their pathological knowledge is essential to understanding when, and why, errors have occurred and so to developing safer future algorithms. This paper summarises the literature on errors made by AI diagnostic tools in histopathology. These include erroneous errors, data concerns (data bias, hidden stratification, data imbalances, distributional shift, and lack of generalisability), reinforcement of outdated practices, unsafe failure mode, automation bias, and insensitivity to impact. Methods to reduce errors in both tool design and clinical use are discussed, and the practical roles for pathologists in error minimisation are highlighted. This aims to inform and empower pathologists to move safely through this seismic change in practice and help ensure that novel AI tools are adopted safely.

Standardized Clinical Annotation of Digital Histopathology Slides at the Point of Diagnosis.

As digital pathology replaces conventional glass slide microscopy as a means of reporting cellular pathology samples, the annotation of digital pathology whole slide images is rapidly becoming part of a pathologist's regular practice. Currently, there is no recognizable organization of these annotations, and as a result, pathologists adopt an arbitrary approach to defining regions of interest, leading to irregularity and inconsistency and limiting the downstream efficient use of this valuable effort. In this study, we propose a Standardized Annotation Reporting Style for digital whole slide images. We formed a list of 167 commonly annotated entities (under 12 specialty subcategories) based on review of Royal College of Pathologists and College of American Pathologists documents, feedback from reporting pathologists in our NHS department, and experience in developing annotation dictionaries for PathLAKE research projects. Each entity was assigned a suitable annotation shape, SNOMED CT (SNOMED International) code, and unique color. Additionally, as an example of how the approach could be expanded to specific tumor types, all lung tumors in the fifth World Health Organization of thoracic tumors 2021 were included. The proposed standardization of annotations increases their utility, making them identifiable at low power and searchable across and between cases. This would aid pathologists reporting and reviewing cases and enable annotations to be used for research. This structured approach could serve as the basis for an industry standard and be easily adopted to ensure maximum functionality and efficiency in the use of annotations made during routine clinical examination of digital slides.

Using Systemised Nomenclature of Medicine (SNOMED) codes to select digital pathology whole slide images for long-term archiving.

The archiving of whole slide images represents a hurdle to digital pathology implementation largely because of the amount of data generated. The retention of glass slides is currently recommended for a minimum of 10 years, but it is for individual departments to determine how digital images are archived and for how long. In a retrospective study, we examined the combination of Systemised Nomenclature of Medicine (SNOMED) codes allocated to cases reported between July 2011 and December 2015 and recalled more than 12 months after diagnosis in comparison to non-recalled cases.Our results show that 0.2% of cases are recalled after 12 months, and SNOMED code combinations can be used to identify which cases are likely to be recalled and which are not. This approach could reduce the number of cases archived by 62% and still ensure all cases likely to be recalled remain in the archive.

PD-L1 Testing in Urothelial Carcinoma: Analysis of a Series of 1401 Cases Using Both the 22C3 and SP142 Assays.

Immune checkpoint blockade (ICB) drugs are a novel, effective treatment for advanced urothelial carcinoma. Worldwide, several different ICB drugs are approved, each developed and clinically validated with a specific PD-L1 compound diagnostic assay. As a result, PD-L1 testing workflows in routine practice are complex: requiring multiple assays across two platforms, with each assay having a different method of interpretation. Our service tested 1,401 urothelial carcinoma cases for PD-L1 expression, using both the 22C3 PharmDx assay (required prior to Pembrolizumab therapy) and SP142 assay (required prior to Atezolizumab therapy). Of the 1,401 cases tested, 621 cases (44%) were tested with both the 22C3 PharmDx and SP142 assays, 492 cases (35%) with 22C3 PharmDx only, and 288 cases (21%) with SP142 only. Each assay was used and interpreted according to the manufacturer's guidelines. The rate of positivity we observed was 26% with the 22C3 assay and 31% with the SP142 assay, similar to the pre-licensing studies for both drugs. The discrepancy observed between the assays was 11%, which reinforces the requirement for utilisation of the correct assay for each agent, and limits potential cross-utility of assays. This aspect must be considered when setting up a PD-L1 testing strategy in laboratories where both Pembrolizumab and Atezolizumab are available for the treatment of urothelial carcinoma but also has broader implications for testing of other cancers where multiple ICB drugs and their respective assays are approved.

Prevalence of complex post-traumatic stress disorder in survivors of human trafficking and modern slavery: a systematic review

Background and objectives. The human rights violation of human trafficking and modern slavery could be described as multiple and prolonged traumatisation. This corresponds to the type of trauma identified as most likely to be associated with ‘complex post-traumatic stress disorder’ (CPTSD) as identified in the new 11th edition of the International Classification of Diseases (ICD-11). This review aims to collate the evidence of complex post-traumatic stress disorder in populations that have been trafficked, with the intention to highlight important considerations to be made in terms of managing survivor's health care needs and minimising further traumatisation. Methods Five databases were searched using key terms related to human trafficking, modern slavery, and complex post-traumatic stress disorder. Results Five studies reporting on a total of 342 participants were included in the review. These studies indicated that an average of 41% of survivors of modern slavery and human trafficking had CPTSD. This was higher than the 14% diagnosed with PTSD. Post-trafficking stress, endured whilst living in refugee camps, was higher in individuals with CPTSD than in those living with PTSD. Healthcare was more difficult to access by populations with PTSD and CPTSD compared to those with no diagnosis. Conclusion There is a high prevalence of CPTSD in modern slavery and trafficking survivors therefore a need for identification and specialised treatment. Consideration should be given to consequent biopsychosocial needs, particularly access to healthcare and minimisation of post-trafficking stress.