Image-based multiplex immune profiling of cancer tissues: translational implications. A report of the International Immuno-oncology Biomarker Working Group on Breast Cancer.

Recent advances in the field of immuno-oncology have brought transformative changes in the management of cancer patients. The immune profile of tumours has been found to have key value in predicting disease prognosis and treatment response in various cancers. Multiplex immunohistochemistry and immunofluorescence have emerged as potent tools for the simultaneous detection of multiple protein biomarkers in a single tissue section, thereby expanding opportunities for molecular and immune profiling while preserving tissue samples. By establishing the phenotype of individual tumour cells when distributed within a mixed cell population, the identification of clinically relevant biomarkers with high-throughput multiplex immunophenotyping of tumour samples has great potential to guide appropriate treatment choices. Moreover, the emergence of novel multi-marker imaging approaches can now provide unprecedented insights into the tumour microenvironment, including the potential interplay between various cell types. However, there are significant challenges to widespread integration of these technologies in daily research and clinical practice. This review addresses the challenges and potential solutions within a structured framework of action from a regulatory and clinical trial perspective. New developments within the field of immunophenotyping using multiplexed tissue imaging platforms and associated digital pathology are also described, with a specific focus on translational implications across different subtypes of cancer. © 2024 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

A Pipeline for Evaluation of Machine Learning/Artificial Intelligence Models to Quantify Programmed Death Ligand 1 Immunohistochemistry.

Immunohistochemistry (IHC) is used to guide treatment decisions in multiple cancer types. For treatment with checkpoint inhibitors, programmed death ligand 1 (PD-L1) IHC is used as a companion diagnostic. However, the scoring of PD-L1 is complicated by its expression in cancer and immune cells. Separation of cancer and noncancer regions is needed to calculate tumor proportion scores (TPS) of PD-L1, which is based on the percentage of PD-L1-positive cancer cells. Evaluation of PD-L1 expression requires highly experienced pathologists and is often challenging and time-consuming. Here, we used a multi-institutional cohort of 77 lung cancer cases stained centrally with the PD-L1 22C3 clone. We developed a 4-step pipeline for measuring TPS that includes the coregistration of hematoxylin and eosin, PD-L1, and negative control (NC) digital slides for exclusion of necrosis, segmentation of cancer regions, and quantification of PD-L1+ cells. As cancer segmentation is a challenging step for TPS generation, we trained DeepLab V3 in the Visiopharm software package to outline cancer regions in PD-L1 and NC images and evaluated the model performance by mean intersection over union (mIoU) against manual outlines. Only 14 cases were required to accomplish a mIoU of 0.82 for cancer segmentation in hematoxylin-stained NC cases. For PD-L1-stained slides, a model trained on PD-L1 tiles augmented by registered NC tiles achieved a mIoU of 0.79. In segmented cancer regions from whole slide images, the digital TPS achieved an accuracy of 75% against the manual TPS scores from the pathology report. Major reasons for algorithmic inaccuracies include the inclusion of immune cells in cancer outlines and poor nuclear segmentation of cancer cells. Our transparent and stepwise approach and performance metrics can be applied to any IHC assay to provide pathologists with important insights on when to apply and how to evaluate commercial automated IHC scoring systems.

Advancing Ki67 hotspot detection in breast cancer: a comparative analysis of automated digital image analysis algorithms.

AIM: Manual detection and scoring of Ki67 hotspots is difficult and prone to variability, limiting its clinical utility. Automated hotspot detection and scoring by digital image analysis (DIA) could improve the assessment of the Ki67 hotspot proliferation index (PI). This study compared the clinical performance of Ki67 hotspot detection and scoring DIA algorithms based on virtual dual staining (VDS) and deep learning (DL) with manual Ki67 hotspot PI assessment. METHODS: Tissue sections of 135 consecutive invasive breast carcinomas were immunohistochemically stained for Ki67. Two DIA algorithms, based on VDS and DL, automatically determined the Ki67 hotspot PI. For manual assessment; two independent observers detected hotspots and calculated scores using a validated scoring protocol. RESULTS: Automated hotspot detection and assessment by VDS and DL could be performed in 73% and 100% of the cases, respectively. Automated hotspot detection by VDS and DL led to higher Ki67 hotspot PIs (mean 39.6% and 38.3%, respectively) compared to manual consensus Ki67 PIs (mean 28.8%). Comparing manual consensus Ki67 PIs with VDS Ki67 PIs revealed substantial correlation (r = 0.90), while manual consensus versus DL Ki67 PIs demonstrated high correlation (r = 0.95). CONCLUSION: Automated Ki67 hotspot detection and analysis correlated strongly with manual Ki67 assessment and provided higher PIs compared to manual assessment. The DL-based algorithm outperformed the VDS-based algorithm in clinical applicability, because it did not depend on virtual alignment of slides and correlated stronger with manual scores. Use of a DL-based algorithm may allow clearer Ki67 PI cutoff values, thereby improving the clinical usability of Ki67.

Spatial analyses of immune cell infiltration in cancer: current methods and future directions: A report of the International Immuno-Oncology Biomarker Working Group on Breast Cancer.

Modern histologic imaging platforms coupled with machine learning methods have provided new opportunities to map the spatial distribution of immune cells in the tumor microenvironment. However, there exists no standardized method for describing or analyzing spatial immune cell data, and most reported spatial analyses are rudimentary. In this review, we provide an overview of two approaches for reporting and analyzing spatial data (raster versus vector-based). We then provide a compendium of spatial immune cell metrics that have been reported in the literature, summarizing prognostic associations in the context of a variety of cancers. We conclude by discussing two well-described clinical biomarkers, the breast cancer stromal tumor infiltrating lymphocytes score and the colon cancer Immunoscore, and describe investigative opportunities to improve clinical utility of these spatial biomarkers. © 2023 The Pathological Society of Great Britain and Ireland.

Pitfalls in machine learning-based assessment of tumor-infiltrating lymphocytes in breast cancer: A report of the International Immuno-Oncology Biomarker Working Group on Breast Cancer

The clinical significance of the tumor-immune interaction in breast cancer is now established, and tumor-infiltrating lymphocytes (TILs) have emerged as predictive and prognostic biomarkers for patients with triple-negative (estrogen receptor, progesterone receptor, and HER2-negative) breast cancer and HER2-positive breast cancer. How computational assessments of TILs might complement manual TIL assessment in trial and daily practices is currently debated. Recent efforts to use machine learning (ML) to automatically evaluate TILs have shown promising results. We review state-of-the-art approaches and identify pitfalls and challenges of automated TIL evaluation by studying the root cause of ML discordances in comparison to manual TIL quantification. We categorize our findings into four main topics: (1) technical slide issues, (2) ML and image analysis aspects, (3) data challenges, and (4) validation issues. The main reason for discordant assessments is the inclusion of false-positive areas or cells identified by performance on certain tissue patterns or design choices in the computational implementation. To aid the adoption of ML for TIL assessment, we provide an in-depth discussion of ML and image analysis, including validation issues that need to be considered before reliable computational reporting of TILs can be incorporated into the trial and routine clinical management of patients with triple-negative breast cancer. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Digital image analysis and assisted reading of the HER2 score display reduced concordance: pitfalls in the categorisation of HER2-low breast cancer.

AIMS: Digital image analysis (DIA) is used increasingly as an assisting tool to evaluate biomarkers, including human epidermal growth factor receptor 2 (HER2) in invasive breast cancer (BC). DIA can assist pathologists in HER2 evaluation by presenting quantitative information about the HER2 staining in APP assisted reading (AR). Concurrently, the HER2-low category (HER2-1+/2+ without HER2 gene amplification) has gained prominence due to newly developed antibody-drug conjugates. However, major inter- and intraobserver variability have been observed for the entity. The present quality assurance study investigated the concordance between DIA and AR in clinical use, especially concerning the HER2-low category. METHODS AND RESULTS: HER2 immunohistochemistry (IHC) in 761 tumours from 727 patients was evaluated in tissue microarray (TMA) cores by DIA (Visiopharm HER2-CONNECT) and AR. Overall concordance between HER2-scores were 73% (n = 552, weighted-κ: 0.66), and 88% (n = 669, weighted-κ: 0.70), when combining HER2-0/1+. A total of 205 scores were discordant by one category, while four were discordant by two categories. A heterogeneous HER2 pattern was relatively common in the discordant cases and a pitfall in the categorisation of HER2-low BC. AR more commonly reassigned a lower HER2 score (from HER2-1+ to HER2-0) within the HER2-low subgroup (n = 624) compared with DIA. CONCLUSION: DIA and AR display moderate agreement with heterogeneous and aberrant staining, representing a source of discordance and a pitfall in the evaluation of HER2.

Quantifying intraepithelial lymphocytes and subepithelial collagen band in microscopic colitis, extracting insights into the interrelationship of lymphocytic and collagenous colitis.

Microscopic colitis (MC) is the umbrella term for the conditions termed lymphocytic colitis (LC) and collagenous colitis (CC). LC with thickening of the subepithelial collagen band or CC with increased number of intraepithelial T- lymphocytes (IELs) is often seen in MC and may lead to difficulties in correct histological classification. We investigated the extent of overlapping features of CC and LC in 60 cases of MC by measuring the exact thickness of the subepithelial collagen band in Van Gieson stained slides and quantifying number of IELs in CD3 stained slides by digital image analysis. A thickened collagen band was observed in nine out of 29 cases with LC (31%) and an increased number of IELs in all 23 cases of CC (100%). There was no correlation between the thickness of the collagen band and number of IELs. Due to the increased number of IELs in all cases of CC we consider the lymphocytic inflammatory infiltration of the mucosa to be the essential histopathological feature of MC. However, although LC and CC are related due to the lymphocytic inflammation, the non-linear correlation of number of IELs and thickness of the collagenous band indicate differences in their pathogenesis.

Development of a method for digital assessment of tumor regression grade in patients with rectal cancer following neoadjuvant therapy.

Neoadjuvant chemo-radiotherapy (nCRT) followed by surgical resection is the standard treatment strategy in patients with locally advanced rectal cancer (RC). The pathological effect of nCRT is assessed by determining the tumor regression grade (TRG) of the resected tumor. Various methods exist for assessing TRG and all are performed manually by the pathologist with an accompanying risk of interobserver variation. Automated digital image analysis could be a more objective and reproducible approach to evaluate TRG. This study aimed at developing a digital method to assess TRG in RC following nCRT, and correlate the results to the currently used Mandard method. A deep learning-based semi-automatic Epithelium-Tumor area Percentage (ETP) algorithm enabling quantification of tumor regression by determining the percentage of residual tumor epithelium out of the total tumor area was developed. The ETP was quantified in 50 cases treated with nCRT and 25 cases with no prior nCRT served as controls. Median ETP was 39.25% in untreated compared with 6.64% in patients who received nCRT (P < .001). The ETP of the resected tumors treated with nCRT increased along with increasing Mandard grade (P < .001). As new treatment strategies in RC are emerging, performing an accurate and reproducible evaluation of TRG is important in the assessment of treatment response and prognosis. TRG is often used as an outcome point in clinical trials. The ETP algorithm is capable of performing a precise and objective value of tumor regression.

Automated Quantification of sTIL Density with H&E-Based Digital Image Analysis Has Prognostic Potential in Triple-Negative Breast Cancers.

Triple-negative breast cancer (TNBC) is an aggressive and difficult-to-treat cancer type that represents approximately 15% of all breast cancers. Recently, stromal tumor-infiltrating lymphocytes (sTIL) resurfaced as a strong prognostic biomarker for overall survival (OS) for TNBC patients. Manual assessment has innate limitations that hinder clinical adoption, and the International Immuno-Oncology Biomarker Working Group (TIL-WG) has therefore envisioned that computational assessment of sTIL could overcome these limitations and recommended that any algorithm should follow the manual guidelines where appropriate. However, no existing studies capture all the concepts of the guideline or have shown the same prognostic evidence as manual assessment. In this study, we present a fully automated digital image analysis pipeline and demonstrate that our hematoxylin and eosin (H&E)-based pipeline can provide a quantitative and interpretable score that correlates with the manual pathologist-derived sTIL status, and importantly, can stratify a retrospective cohort into two significant distinct prognostic groups. We found our score to be prognostic for OS (HR: 0.81 CI: 0.72-0.92 p = 0.001) independent of age, tumor size, nodal status, and tumor type in statistical modeling. While prior studies have followed fragments of the TIL-WG guideline, our approach is the first to follow all complex aspects, where appropriate, supporting the TIL-WG vision of computational assessment of sTIL in the future clinical setting.

Histological disease activity in patients with microscopic colitis is not related to clinical disease activity or long-term prognosis.

BACKGROUND: Microscopic colitis (MC) is a common cause of chronic watery diarrhea. Biopsies with characteristic histological features are crucial for establishing the diagnosis. The two main subtypes are collagenous colitis (CC) and lymphocytic colitis (LC) but incomplete forms exist. The disease course remains unpredictable varying from spontaneous remission to a relapsing course. AIM: To identify possible histological predictors of course of disease. METHODS: Sixty patients from the European prospective MC registry (PRO-MC Collaboration) were included. Digitised histological slides stained with CD3 and Van Gieson were available for all patients. Total cell density and proportion of CD3 positive lymphocytes in lamina propria and surface epithelium were estimated by automated image analysis, and measurement of the subepithelial collagenous band was performed. Histopathological features were correlated to the number of daily stools and daily watery stools at time of endoscopy and at baseline as well as the clinical disease course (quiescent, achieved remission after treatment, relapsing or chronic active) at 1-year follow-up. RESULTS: Neither total cell density in lamina propria, proportion of CD3 positive lymphocytes in lamina propria or surface epithelium, or thickness of collagenous band showed significant correlation to the number of daily stools or daily watery stools at any point of time. None of the assessed histological parameters at initial diagnosis were able to predict clinical disease course at 1-year follow-up. CONCLUSIONS: Our data indicate that the evaluated histological parameters were neither markers of disease activity at the time of diagnosis nor predictors of disease course.

Report on computational assessment of Tumor Infiltrating Lymphocytes from the International Immuno-Oncology Biomarker Working Group.

Assessment of tumor-infiltrating lymphocytes (TILs) is increasingly recognized as an integral part of the prognostic workflow in triple-negative (TNBC) and HER2-positive breast cancer, as well as many other solid tumors. This recognition has come about thanks to standardized visual reporting guidelines, which helped to reduce inter-reader variability. Now, there are ripe opportunities to employ computational methods that extract spatio-morphologic predictive features, enabling computer-aided diagnostics. We detail the benefits of computational TILs assessment, the readiness of TILs scoring for computational assessment, and outline considerations for overcoming key barriers to clinical translation in this arena. Specifically, we discuss: 1. ensuring computational workflows closely capture visual guidelines and standards; 2. challenges and thoughts standards for assessment of algorithms including training, preanalytical, analytical, and clinical validation; 3. perspectives on how to realize the potential of machine learning models and to overcome the perceptual and practical limits of visual scoring.

From Detection of Individual Metastases to Classification of Lymph Node Status at the Patient Level: The CAMELYON17 Challenge.

Automated detection of cancer metastases in lymph nodes has the potential to improve the assessment of prognosis for patients. To enable fair comparison between the algorithms for this purpose, we set up the CAMELYON17 challenge in conjunction with the IEEE International Symposium on Biomedical Imaging 2017 Conference in Melbourne. Over 300 participants registered on the challenge website, of which 23 teams submitted a total of 37 algorithms before the initial deadline. Participants were provided with 899 whole-slide images (WSIs) for developing their algorithms. The developed algorithms were evaluated based on the test set encompassing 100 patients and 500 WSIs. The evaluation metric used was a quadratic weighted Cohen's kappa. We discuss the algorithmic details of the 10 best pre-conference and two post-conference submissions. All these participants used convolutional neural networks in combination with pre- and postprocessing steps. Algorithms differed mostly in neural network architecture, training strategy, and pre- and postprocessing methodology. Overall, the kappa metric ranged from 0.89 to -0.13 across all submissions. The best results were obtained with pre-trained architectures such as ResNet. Confusion matrix analysis revealed that all participants struggled with reliably identifying isolated tumor cells, the smallest type of metastasis, with detection rates below 40%. Qualitative inspection of the results of the top participants showed categories of false positives, such as nerves or contamination, which could be targeted for further optimization. Last, we show that simple combinations of the top algorithms result in higher kappa metric values than any algorithm individually, with 0.93 for the best combination.