KDM5B promotes immune evasion by recruiting SETDB1 to silence retroelements.

Tumours use various strategies to evade immune surveillance1,2. Immunotherapies targeting tumour immune evasion such as immune checkpoint blockade have shown considerable efficacy on multiple cancers3,4 but are ineffective for most patients due to primary or acquired resistance5-7. Recent studies showed that some epigenetic regulators suppress anti-tumour immunity2,8-12, suggesting that epigenetic therapies could boost anti-tumour immune responses and overcome resistance to current immunotherapies. Here we show that, in mouse melanoma models, depletion of KDM5B-an H3K4 demethylase that is critical for melanoma maintenance and drug resistance13-15-induces robust adaptive immune responses and enhances responses to immune checkpoint blockade. Mechanistically, KDM5B recruits the H3K9 methyltransferase SETDB1 to repress endogenous retroelements such as MMVL30 in a demethylase-independent manner. Derepression of these retroelements activates cytosolic RNA-sensing and DNA-sensing pathways and the subsequent type-I interferon response, leading to tumour rejection and induction of immune memory. Our results demonstrate that KDM5B suppresses anti-tumour immunity by epigenetic silencing of retroelements. We therefore reveal roles of KDM5B in heterochromatin regulation and immune evasion in melanoma, opening new paths for the development of KDM5B-targeting and SETDB1-targeting therapies to enhance tumour immunogenicity and overcome immunotherapy resistance.

Image-based multiplex immune profiling of cancer tissues: translational implications. A report of the International Immuno-oncology Biomarker Working Group on Breast Cancer.

Recent advances in the field of immuno-oncology have brought transformative changes in the management of cancer patients. The immune profile of tumours has been found to have key value in predicting disease prognosis and treatment response in various cancers. Multiplex immunohistochemistry and immunofluorescence have emerged as potent tools for the simultaneous detection of multiple protein biomarkers in a single tissue section, thereby expanding opportunities for molecular and immune profiling while preserving tissue samples. By establishing the phenotype of individual tumour cells when distributed within a mixed cell population, the identification of clinically relevant biomarkers with high-throughput multiplex immunophenotyping of tumour samples has great potential to guide appropriate treatment choices. Moreover, the emergence of novel multi-marker imaging approaches can now provide unprecedented insights into the tumour microenvironment, including the potential interplay between various cell types. However, there are significant challenges to widespread integration of these technologies in daily research and clinical practice. This review addresses the challenges and potential solutions within a structured framework of action from a regulatory and clinical trial perspective. New developments within the field of immunophenotyping using multiplexed tissue imaging platforms and associated digital pathology are also described, with a specific focus on translational implications across different subtypes of cancer. © 2024 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Reproducible Reporting of the Collection and Evaluation of Annotations for Artificial Intelligence Models.

This work puts forth and demonstrates the utility of a reporting framework for collecting and evaluating annotations of medical images used for training and testing artificial intelligence (AI) models in assisting detection and diagnosis. AI has unique reporting requirements, as shown by the AI extensions to the Consolidated Standards of Reporting Trials (CONSORT) and Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) checklists and the proposed AI extensions to the Standards for Reporting Diagnostic Accuracy (STARD) and Transparent Reporting of a Multivariable Prediction model for Individual Prognosis or Diagnosis (TRIPOD) checklists. AI for detection and/or diagnostic image analysis requires complete, reproducible, and transparent reporting of the annotations and metadata used in training and testing data sets. In an earlier work by other researchers, an annotation workflow and quality checklist for computational pathology annotations were proposed. In this manuscript, we operationalize this workflow into an evaluable quality checklist that applies to any reader-interpreted medical images, and we demonstrate its use for an annotation effort in digital pathology. We refer to this quality framework as the Collection and Evaluation of Annotations for Reproducible Reporting of Artificial Intelligence (CLEARR-AI).

Training pathologists to assess stromal tumour-infiltrating lymphocytes in breast cancer synergises efforts in clinical care and scientific research.

A growing body of research supports stromal tumour-infiltrating lymphocyte (TIL) density in breast cancer to be a robust prognostic and predicive biomarker. The gold standard for stromal TIL density quantitation in breast cancer is pathologist visual assessment using haematoxylin and eosin-stained slides. Artificial intelligence/machine-learning algorithms are in development to automate the stromal TIL scoring process, and must be validated against a reference standard such as pathologist visual assessment. Visual TIL assessment may suffer from significant interobserver variability. To improve interobserver agreement, regulatory science experts at the US Food and Drug Administration partnered with academic pathologists internationally to create a freely available online continuing medical education (CME) course to train pathologists in assessing breast cancer stromal TILs using an interactive format with expert commentary. Here we describe and provide a user guide to this CME course, whose content was designed to improve pathologist accuracy in scoring breast cancer TILs. We also suggest subsequent steps to translate knowledge into clinical practice with proficiency testing.

Initial interactions with the FDA on developing a validation dataset as a medical device development tool.

Quantifying tumor-infiltrating lymphocytes (TILs) in breast cancer tumors is a challenging task for pathologists. With the advent of whole slide imaging that digitizes glass slides, it is possible to apply computational models to quantify TILs for pathologists. Development of computational models requires significant time, expertise, consensus, and investment. To reduce this burden, we are preparing a dataset for developers to validate their models and a proposal to the Medical Device Development Tool (MDDT) program in the Center for Devices and Radiological Health of the U.S. Food and Drug Administration (FDA). If the FDA qualifies the dataset for its submitted context of use, model developers can use it in a regulatory submission within the qualified context of use without additional documentation. Our dataset aims at reducing the regulatory burden placed on developers of models that estimate the density of TILs and will allow head-to-head comparison of multiple computational models on the same data. In this paper, we discuss the MDDT preparation and submission process, including the feedback we received from our initial interactions with the FDA and propose how a qualified MDDT validation dataset could be a mechanism for open, fair, and consistent measures of computational model performance. Our experiences will help the community understand what the FDA considers relevant and appropriate (from the perspective of the submitter), at the early stages of the MDDT submission process, for validating stromal TIL density estimation models and other potential computational models. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland. This article has been contributed to by U.S. Government employees and their work is in the public domain in the USA.

Spatial analyses of immune cell infiltration in cancer: current methods and future directions: A report of the International Immuno-Oncology Biomarker Working Group on Breast Cancer.

Modern histologic imaging platforms coupled with machine learning methods have provided new opportunities to map the spatial distribution of immune cells in the tumor microenvironment. However, there exists no standardized method for describing or analyzing spatial immune cell data, and most reported spatial analyses are rudimentary. In this review, we provide an overview of two approaches for reporting and analyzing spatial data (raster versus vector-based). We then provide a compendium of spatial immune cell metrics that have been reported in the literature, summarizing prognostic associations in the context of a variety of cancers. We conclude by discussing two well-described clinical biomarkers, the breast cancer stromal tumor infiltrating lymphocytes score and the colon cancer Immunoscore, and describe investigative opportunities to improve clinical utility of these spatial biomarkers. © 2023 The Pathological Society of Great Britain and Ireland.

Pitfalls in machine learning-based assessment of tumor-infiltrating lymphocytes in breast cancer: A report of the International Immuno-Oncology Biomarker Working Group on Breast Cancer

The clinical significance of the tumor-immune interaction in breast cancer is now established, and tumor-infiltrating lymphocytes (TILs) have emerged as predictive and prognostic biomarkers for patients with triple-negative (estrogen receptor, progesterone receptor, and HER2-negative) breast cancer and HER2-positive breast cancer. How computational assessments of TILs might complement manual TIL assessment in trial and daily practices is currently debated. Recent efforts to use machine learning (ML) to automatically evaluate TILs have shown promising results. We review state-of-the-art approaches and identify pitfalls and challenges of automated TIL evaluation by studying the root cause of ML discordances in comparison to manual TIL quantification. We categorize our findings into four main topics: (1) technical slide issues, (2) ML and image analysis aspects, (3) data challenges, and (4) validation issues. The main reason for discordant assessments is the inclusion of false-positive areas or cells identified by performance on certain tissue patterns or design choices in the computational implementation. To aid the adoption of ML for TIL assessment, we provide an in-depth discussion of ML and image analysis, including validation issues that need to be considered before reliable computational reporting of TILs can be incorporated into the trial and routine clinical management of patients with triple-negative breast cancer. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

PD-L1 protein expression in relation to recurrence score values in early-stage ER + breast cancer.

PURPOSE: We assessed associations between PD-L1 protein expression, RS, tumor grade, and stromal tumor-infiltrating lymphocyte (TIL) count in early-stage ER + cancers. METHODS: FFPE tissue blocks of 213 patients with RS in 2012-2017 were identified. PD-L1 immunohistochemistry was performed with SP142 assay, cases with ≥ 1% tumor-infiltrating immune cell positivity in the tumor area were considered PD-L1 + . TIL scores were determined following the international TIL counting guidelines. PD-L1 expression positivity rates were compared across RS (< 11, 11-25, > 25) and TIL categories (< 10%, 10-29%, > 30%), and tumor grade using Wilcoxon and Chi-square tests. Multivariate analysis was performed using logistic regression. RESULTS: PD-L1 and TIL results were available for 201 and 203 patients. Overall, 53% of cases were PD-L1 +. PD-L1 expression was higher among cases with RS > 25, versus RS < 11 (p = 0.00019) and RS 11-25 (p = 0.0017). PD-L1 positivity also correlated with TIL score, tumor grade, and tumor size. Among cancers with TIL > 30%, 92% were PD-L1 + versus 44% PD-L1 + among TIL < 10% (p = 2.8 × 10-6). Grade 3 cancers had higher PD-L1 positivity (79% PD-L1 +) versus grade 2 (49% PD-L1 +) or 1 tumors (48% PD-L1 +) (p = 0.00047). T2 and T3 tumors had more frequent PD-L1 positivity (67% and 83%, respectively) versus T1 cancers (46%) (p = 0.008). In multivariate analysis, only TIL and RS remained as independent predictors of PD-L1 positivity. CONCLUSION: PD-L1 expression is significantly more frequent and higher in larger tumors (T2, T3), grade 3 cancers, and in cancers with RS > 25. PD-L1 expression also correlates with TIL score.

Analysis of the genomic landscapes of Barbadian and Nigerian women with triple negative breast cancer.

PURPOSE: Triple negative breast cancer (TNBC) is an aggressive breast cancer subtype that disproportionately affects women of African ancestry (WAA) and is often associated with poor survival. Although there is a high prevalence of TNBC across West Africa and in women of the African diaspora, there has been no comprehensive genomics study to investigate the mutational profile of ancestrally related women across the Caribbean and West Africa. METHODS: This multisite cross-sectional study used 31 formalin-fixed paraffin-embedded (FFPE) samples from Barbadian and Nigerian TNBC participants. High-resolution whole exome sequencing (WES) was performed on the Barbadian and Nigerian TNBC samples to identify their mutational profiles and comparisons were made to African American, European American and Asian American sequencing data obtained from The Cancer Genome Atlas (TCGA). Whole exome sequencing was conducted on tumors with an average of 382 × coverage and 4335 × coverage for pooled germline non-tumor samples. RESULTS: Variants detected at high frequency in our WAA cohorts were found in the following genes NBPF12, PLIN4, TP53 and BRCA1. In the TCGA TNBC cases, these genes had a lower mutation rate, except for TP53 (32% in our cohort; 63% in TCGA-African American; 67% in TCGA-European American; 63% in TCGA-Asian). For all altered genes, there were no differences in frequency of mutations between WAA TNBC groups including the TCGA-African American cohort. For copy number variants, high frequency alterations were observed in PIK3CA, TP53, FGFR2 and HIF1AN genes. CONCLUSION: This study provides novel insights into the underlying genomic alterations in WAA TNBC samples and shines light on the importance of inclusion of under-represented populations in cancer genomics and biomarker studies.

Quantitative assessment of the immune microenvironment in African American Triple Negative Breast Cancer: a case-control study.

PURPOSE: Triple negative breast cancer (TNBC) is more common in African American (AA) than Non-AA (NAA) population. We hypothesize that tumor microenvironment (TME) contributes to this disparity. Here, we use multiplex quantitative immunofluorescence to characterize the expression of immunologic biomarkers in the TME in both populations. PATIENTS AND METHODS: TNBC tumor resection specimen tissues from a 100-patient case: control cohort including 49 AA and 51 NAA were collected. TME markers including CD45, CD14, CD68, CD206, CD4, CD8, CD20, CD3, Ki67, GzB, Thy1, FAP, aSMA, CD34, Col4, VWF and PD-L1 we quantitatively assessed in every field of view. Mean expression levels were compared between cases and controls. RESULTS: Although no significant differences were detected in individual lymphoid and myeloid markers, we found that infiltration with CD45+ immune cells (p = 0.0102) was higher in TNBC in AA population. AA TNBC tumors also had significantly higher level of lymphocytic infiltration defined as CD45+ CD14- cells (p = 0.0081). CD3+ T-cells in AA tumors expressed significantly higher levels of Ki67 (0.0066) compared to NAAs, indicating that a higher percentage of AA tumors contained activated T-cells. All other biomarkers showed no significant differences between the AA and NAA group. CONCLUSIONS: While the TME in TNBC is rich in immune cells in both racial groups, there is a numerical increase in lymphoid infiltration in AA compared to NAA TNBC. Significantly, higher activated T cells seen in AA patients raises the possibility that there may be a subset of AA patients with improved response to immunotherapy.

Data File Standard for Flow Cytometry, Version FCS 3.2.

FCS 3.2 is a revision of the flow cytometry data standard based on a decade of suggested improvements from the community as well as industry needs to capture instrument conditions and measurement features more precisely. The unchanged goal of the standard is to provide a uniform file format that allows files created by one type of acquisition hardware and software to be analyzed by any other type. The standard retains the overall FCS file structure and most features of previous versions, but also contains a few changes that were required to support new types of data and use cases efficiently. These changes are incompatible with existing FCS file readers. Notably, FCS 3.2 supports mixed data types to, for example, allow FCS measurements that are intrinsically integers (e.g., indices or class assignments) or measurements that are commonly captured as integers (e.g., time ticks) to be more represented as integer values, while capturing other measurements as floating-point values in the same FCS data set. In addition, keywords explicitly specifying dyes, detectors, and analytes were added to avoid having to extract those heuristically and unreliably from measurement names. Types of measurements were formalized, several keywords added, others removed, or deprecated, and various aspects of the specification were clarified. A reference implementation of the cyclic redundancy check (CRC) calculation is provided in two programming languages since a correct CRC implementation was problematic for many vendors. © 2020 International Society for Advancement of Cytometry.

ISAC Probe Tag Dictionary: Standardized Nomenclature for Detection and Visualization Labels Used in Cytometry and Microscopy Imaging.

Since the advent of microscopy imaging and flow cytometry, there has been an explosion in the number of probes, consisting of a component binding to an analyte and a detectable tag, to mark areas of interest in or on cells and tissue. Probe tags have been created to detect and/or visualize probes. Over time, these probe tags have increased in number. The expansion has resulted in arbitrarily created synonyms of probe tags used in publications and software. The synonyms are problematic for readability of publications, accuracy of text/data mining, and bridging data from multiple platforms, protocols, and databases for Big Data analysis. Development and implementation of a universal language for probe tags will ensure equivalent quality and level of data being reported or extracted for clinical/scientific evaluation as well as help connect data from many platforms. The International Society for Advancement of Cytometry Data Standards Task Force composed of academic scientists and industry hardware/software/reagent manufactures have developed recommendations for a standardized nomenclature for probe tags used in cytometry and microscopy imaging. These recommendations are shared in this technical note in the form of a Probe Tag Dictionary. © 2020 International Society for Advancement of Cytometry.

Immune Cell and Cell Cluster Phenotyping, Quantitation, and Visualization Using In Silico Multiplexed Images and Tissue Cytometry.

Phenotyping immune cells and cell clusters in situ, including their activation state and function, can aid in interpretation of spatial relationships within the tissue microenvironment. Immune cell phenotypes require multiple biomarkers. However, conventional microscopy setups can only image up to four biomarkers at one time. In this report, we describe and give an example of a workflow to phenotype, quantitate, and visualize greater than four biomarkers in silico utilizing multiplexed fluorescence histology and the TissueFAXS quantitative imaging system with a conventional microscopy setup. Biomarkers were conjugated to Cy3 or Cy5. Multiplexed staining was performed on formalin-fixed paraffin-embedded tissue sections. We imaged the slides, inactivated the dyes, and repeated the process until all biomarkers were stained. Phenotype profiles were built based on in silico combinations of the biomarkers. We used algorithms that aligned all images to create a composite image, isolated each cell in the image, and identified biomarker positive cells in the image. The in silico phenotypes were quantitated and displayed through flow cytometry-like histograms and dot scatterplots in addition to backgating into the tissue images. The advantage of our workflow is that it provides visual verification of cell isolation and identification as well as highlight characteristics of cells and cell clusters. © 2018 International Society for Advancement of Cytometry.

Immune correlates of talactoferrin alfa in biopsied tumor of relapsed/refractory metastatic non-small cell lung cancer patients.

CONTEXT: Talactoferrin alfa (TLF) is a unique recombinant form of human lactoferrin. The hypothesized mechanism of action involves TLF binding to the intestinal endothelium inducing dendritic cell maturation and cytokine release leading to infiltration of tumor with monocytes and T-lymphocytes and inhibition of tumor growth. OBJECTIVE: Based on promising phase II trial results, this correlative study was undertaken to examine immune mechanism of action of TLF in metastatic non-small cell lung cancer (NSCLC) patients. METHODS: Talactoferrin was administered orally at 1.5 g bid weeks 1-12 with 2 weeks off on a 14-week cycle. Enrolled patients had a pathologic diagnosis of NSCLC previously treated with at least two lines of systemic treatment. Patients had core biopsy of tumor before initiation of talactoferrin and at week 7 on TLF. Flow cytometry and quantitative immunohistochemistry for immune correlates were performed on the biopsied specimens. RESULTS: Four patients with metastatic NSCLC were enrolled. The trial was halted pre-maturely in light of negative phase III trial results. For the two patients who had repeat on-treatment tumor biopsies, a consistent increase in monocytes as a percentage of total immune cells was observed. Otherwise, no clear trend of increase or decrease was observed in any other immune cell parameters compared to matched patient pre-treatment biopsies. CONCLUSION: Repeat biopsies for immune correlates by flow cytometry and quantitative immunohistochemistry in NSCLC patients are feasible. In the few patients sampled before trial closure, increased monocytes as a total percentage of the immune cell population within tumor was observed in response to TLF.

Tumor-Associated Macrophages Expand Chemoresistant, Ovarian Cancer Stem-Like Cells.

The persistence of ovarian cancer stem-like cells (OvCSCs) after chemotherapy resistance has been implicated in relapse. However, the ability of these relatively quiescent cells to produce the robust tumor regrowth necessary for relapse remains an enigma. Since normal stem cells exist in a niche, and tumor-associated macrophages (TAMs) are the highest abundance immune cell within ovarian tumors, we hypothesized that TAMs may influence OvCSC proliferation. To test this, we optimized OvCSC enrichment by sphere culture and in vitro polarization of monocytes to a TAM-like M2 phenotype. Using cocultures that permitted the exchange of only soluble factors, we found that M2 macrophages increased the proliferation of sphere cells. Longer-term exposure (5-7 days) to soluble TAM factors led to retention of some stem cell features by OvCSCs but loss of others, suggesting that TAMs may support an intermediate stemness phenotype in OvCSCs. Although TAM coculture decreased the percentage of OvCSCs surviving chemotherapy, it increased the overall number. We therefore sought to determine the influence of this interaction on chemotherapy efficacy in vivo and found that inhibiting macrophages improved chemotherapy response. Comparing the gene expression changes in OvCSCs cocultured with TAMs to publicly available patient data identified 34 genes upregulated in OvCSCs by exposure to soluble TAM factors whose expression correlates with outcome. Overall, these data suggest that TAMs may influence OvCSC proliferation and impact therapeutic response.

A review of the impact of energy balance on triple-negative breast cancer.

Cancer cells cannot proliferate without sufficient energy to generate biomass for rapid cell division, as well as to fuel their functions at baseline. For this reason, many recent observational and interventional studies have focused on increasing energy expenditure and/or reducing energy intake during and after cancer treatment. The impact of variance in diet composition and in exercise on cancer outcomes has been detailed extensively elsewhere and is not the primary focus of this review. Instead, in this translational, narrative review we examine studies of how energy balance impacts anticancer immune activation and outcomes in triple-negative breast cancer (TNBC). We discuss preclinical, clinical observational, and the few clinical interventional studies on energy balance in TNBC. We advocate for the implementation of clinical studies to examine how optimizing energy balance-through changes in diet and/or exercise-may optimize the response to immunotherapy in people with TNBC. It is our conviction that by taking a holistic approach that includes energy balance as a key factor to be considered during and after treatment, cancer care may be optimized, and the detrimental effects of cancer treatment and recovery on overall health may be minimized.

Image analysis-based tumor infiltrating lymphocytes measurement predicts breast cancer pathologic complete response in SWOG S0800 neoadjuvant chemotherapy trial.

We assessed the predictive value of an image analysis-based tumor-infiltrating lymphocytes (TILs) score for pathologic complete response (pCR) and event-free survival in breast cancer (BC). About 113 pretreatment samples were analyzed from patients with stage IIB-IIIC HER-2-negative BC randomized to neoadjuvant chemotherapy ± bevacizumab. TILs quantification was performed on full sections using QuPath open-source software with a convolutional neural network cell classifier (CNN11). We used easTILs% as a digital metric of TILs score defined as [sum of lymphocytes area (mm2)/stromal area(mm2)] × 100. Pathologist-read stromal TILs score (sTILs%) was determined following published guidelines. Mean pretreatment easTILs% was significantly higher in cases with pCR compared to residual disease (median 36.1 vs.14.8%, p < 0.001). We observed a strong positive correlation (r = 0.606, p < 0.0001) between easTILs% and sTILs%. The area under the prediction curve (AUC) was higher for easTILs% than sTILs%, 0.709 and 0.627, respectively. Image analysis-based TILs quantification is predictive of pCR in BC and had better response discrimination than pathologist-read sTILs%.

Strategies to mitigate the toxicity of cancer therapeutics.

Cancer therapeutics are dynamically evolving, and include traditional chemotherapy and hormone therapy, as well as more recently developed treatment modalities, such as tyrosine kinase inhibitors, monoclonal antibodies and the revolutionary approach based on immune checkpoint inhibition. These regimens are unfortunately not free of adverse events, and patients with cancer are a susceptible population experiencing a myriad of disease and treatment toxicities combined. In this review, we present the latest overview of the management of the most common systemic cancer treatment symptoms and the science of symptom management supporting these strategies. We discuss cancer-related cognitive impairment, ocular toxicity, ototoxicity, oral mucosal toxicities, gastrointestinal toxicities, renal toxicity, aromatase inhibitor-induced musculoskeletal symptoms, chemotherapy-induced peripheral neuropathy, and immunotherapy-induced autoimmunity derived from systemic therapies for cancer. In summary, we review the future directions and ideal goals of symptom science research in order to benefit patients utilizing a comprehensive individualized approach.

Leveraging the Dynamic Immune Environment Triad in Patients with Breast Cancer: Tumour, Lymph Node, and Peripheral Blood.

During the anti-tumour response to breast cancer, the primary tumour, the peripheral blood, and the lymph nodes each play unique roles. Immunological features at each site reveal evidence of continuous immune cross-talk between them before, during and after treatment. As such, immune responses to breast cancer are found to be highly dynamic and truly systemic, integrating three distinct immune sites, complex cell-migration highways, as well as the temporal dimension of disease progression and treatment. In this review, we provide a connective summary of the dynamic immune environment triad of breast cancer. It is critical that future studies seek to establish dynamic immune profiles, constituting multiple sites, that capture the systemic immune response to breast cancer and define patient-selection parameters resulting in more significant overall responses and survival rates for breast cancer patients.

Development of Training Materials for Pathologists to Provide Machine Learning Validation Data of Tumor-Infiltrating Lymphocytes in Breast Cancer.

The High Throughput Truthing project aims to develop a dataset for validating artificial intelligence and machine learning models (AI/ML) fit for regulatory purposes. The context of this AI/ML validation dataset is the reporting of stromal tumor-infiltrating lymphocytes (sTILs) density evaluations in hematoxylin and eosin-stained invasive breast cancer biopsy specimens. After completing the pilot study, we found notable variability in the sTILs estimates as well as inconsistencies and gaps in the provided training to pathologists. Using the pilot study data and an expert panel, we created custom training materials to improve pathologist annotation quality for the pivotal study. We categorized regions of interest (ROIs) based on their mean sTILs density and selected ROIs with the highest and lowest sTILs variability. In a series of eight one-hour sessions, the expert panel reviewed each ROI and provided verbal density estimates and comments on features that confounded the sTILs evaluation. We aggregated and shaped the comments to identify pitfalls and instructions to improve our training materials. From these selected ROIs, we created a training set and proficiency test set to improve pathologist training with the goal to improve data collection for the pivotal study. We are not exploring AI/ML performance in this paper. Instead, we are creating materials that will train crowd-sourced pathologists to be the reference standard in a pivotal study to create an AI/ML model validation dataset. The issues discussed here are also important for clinicians to understand about the evaluation of sTILs in clinical practice and can provide insight to developers of AI/ML models.