Training pathologists to assess stromal tumour-infiltrating lymphocytes in breast cancer synergises efforts in clinical care and scientific research.

A growing body of research supports stromal tumour-infiltrating lymphocyte (TIL) density in breast cancer to be a robust prognostic and predicive biomarker. The gold standard for stromal TIL density quantitation in breast cancer is pathologist visual assessment using haematoxylin and eosin-stained slides. Artificial intelligence/machine-learning algorithms are in development to automate the stromal TIL scoring process, and must be validated against a reference standard such as pathologist visual assessment. Visual TIL assessment may suffer from significant interobserver variability. To improve interobserver agreement, regulatory science experts at the US Food and Drug Administration partnered with academic pathologists internationally to create a freely available online continuing medical education (CME) course to train pathologists in assessing breast cancer stromal TILs using an interactive format with expert commentary. Here we describe and provide a user guide to this CME course, whose content was designed to improve pathologist accuracy in scoring breast cancer TILs. We also suggest subsequent steps to translate knowledge into clinical practice with proficiency testing.

Reproducible Reporting of the Collection and Evaluation of Annotations for Artificial Intelligence Models.

This work puts forth and demonstrates the utility of a reporting framework for collecting and evaluating annotations of medical images used for training and testing artificial intelligence (AI) models in assisting detection and diagnosis. AI has unique reporting requirements, as shown by the AI extensions to the Consolidated Standards of Reporting Trials (CONSORT) and Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) checklists and the proposed AI extensions to the Standards for Reporting Diagnostic Accuracy (STARD) and Transparent Reporting of a Multivariable Prediction model for Individual Prognosis or Diagnosis (TRIPOD) checklists. AI for detection and/or diagnostic image analysis requires complete, reproducible, and transparent reporting of the annotations and metadata used in training and testing data sets. In an earlier work by other researchers, an annotation workflow and quality checklist for computational pathology annotations were proposed. In this manuscript, we operationalize this workflow into an evaluable quality checklist that applies to any reader-interpreted medical images, and we demonstrate its use for an annotation effort in digital pathology. We refer to this quality framework as the Collection and Evaluation of Annotations for Reproducible Reporting of Artificial Intelligence (CLEARR-AI).

Initial interactions with the FDA on developing a validation dataset as a medical device development tool.

Quantifying tumor-infiltrating lymphocytes (TILs) in breast cancer tumors is a challenging task for pathologists. With the advent of whole slide imaging that digitizes glass slides, it is possible to apply computational models to quantify TILs for pathologists. Development of computational models requires significant time, expertise, consensus, and investment. To reduce this burden, we are preparing a dataset for developers to validate their models and a proposal to the Medical Device Development Tool (MDDT) program in the Center for Devices and Radiological Health of the U.S. Food and Drug Administration (FDA). If the FDA qualifies the dataset for its submitted context of use, model developers can use it in a regulatory submission within the qualified context of use without additional documentation. Our dataset aims at reducing the regulatory burden placed on developers of models that estimate the density of TILs and will allow head-to-head comparison of multiple computational models on the same data. In this paper, we discuss the MDDT preparation and submission process, including the feedback we received from our initial interactions with the FDA and propose how a qualified MDDT validation dataset could be a mechanism for open, fair, and consistent measures of computational model performance. Our experiences will help the community understand what the FDA considers relevant and appropriate (from the perspective of the submitter), at the early stages of the MDDT submission process, for validating stromal TIL density estimation models and other potential computational models. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland. This article has been contributed to by U.S. Government employees and their work is in the public domain in the USA.

Pilot study to evaluate tools to collect pathologist annotations for validating machine learning algorithms.

Purpose: Validation of artificial intelligence (AI) algorithms in digital pathology with a reference standard is necessary before widespread clinical use, but few examples focus on creating a reference standard based on pathologist annotations. This work assesses the results of a pilot study that collects density estimates of stromal tumor-infiltrating lymphocytes (sTILs) in breast cancer biopsy specimens. This work will inform the creation of a validation dataset for the evaluation of AI algorithms fit for a regulatory purpose. Approach: Collaborators and crowdsourced pathologists contributed glass slides, digital images, and annotations. Here, "annotations" refer to any marks, segmentations, measurements, or labels a pathologist adds to a report, image, region of interest (ROI), or biological feature. Pathologists estimated sTILs density in 640 ROIs from hematoxylin and eosin stained slides of 64 patients via two modalities: an optical light microscope and two digital image viewing platforms. Results: The pilot study generated 7373 sTILs density estimates from 29 pathologists. Analysis of annotations found the variability of density estimates per ROI increases with the mean; the root mean square differences were 4.46, 14.25, and 26.25 as the mean density ranged from 0% to 10%, 11% to 40%, and 41% to 100%, respectively. The pilot study informs three areas of improvement for future work: technical workflows, annotation platforms, and agreement analysis methods. Upgrades to the workflows and platforms will improve operability and increase annotation speed and consistency. Conclusions: Exploratory data analysis demonstrates the need to develop new statistical approaches for agreement. The pilot study dataset and analysis methods are publicly available to allow community feedback. The development and results of the validation dataset will be publicly available to serve as an instructive tool that can be replicated by developers and researchers.

Development of Training Materials for Pathologists to Provide Machine Learning Validation Data of Tumor-Infiltrating Lymphocytes in Breast Cancer.

The High Throughput Truthing project aims to develop a dataset for validating artificial intelligence and machine learning models (AI/ML) fit for regulatory purposes. The context of this AI/ML validation dataset is the reporting of stromal tumor-infiltrating lymphocytes (sTILs) density evaluations in hematoxylin and eosin-stained invasive breast cancer biopsy specimens. After completing the pilot study, we found notable variability in the sTILs estimates as well as inconsistencies and gaps in the provided training to pathologists. Using the pilot study data and an expert panel, we created custom training materials to improve pathologist annotation quality for the pivotal study. We categorized regions of interest (ROIs) based on their mean sTILs density and selected ROIs with the highest and lowest sTILs variability. In a series of eight one-hour sessions, the expert panel reviewed each ROI and provided verbal density estimates and comments on features that confounded the sTILs evaluation. We aggregated and shaped the comments to identify pitfalls and instructions to improve our training materials. From these selected ROIs, we created a training set and proficiency test set to improve pathologist training with the goal to improve data collection for the pivotal study. We are not exploring AI/ML performance in this paper. Instead, we are creating materials that will train crowd-sourced pathologists to be the reference standard in a pivotal study to create an AI/ML model validation dataset. The issues discussed here are also important for clinicians to understand about the evaluation of sTILs in clinical practice and can provide insight to developers of AI/ML models.

DRAQ5 and Eosin ('D&E') as an Analog to Hematoxylin and Eosin for Rapid Fluorescence Histology of Fresh Tissues.

Real-time on-site histopathology review of biopsy tissues at the point-of-procedure has great potential for significant clinical value and improved patient care. For instance, on-site review can aid in rapid screening of diagnostic biopsies to reduce false-negative results, or in quantitative assessment of biospecimen quality to increase the efficacy of downstream laboratory and histopathology analysis. However, the only currently available rapid pathology method, frozen section analysis (FSA), is too time- and labor-intensive for use in screening large quantities of biopsy tissues and is too destructive for maximum tissue conservation in multiple small needle core biopsies. In this work we demonstrate the spectrally-compatible combination of the nuclear stain DRAQ5 and the anionic counterstain eosin as a dual-component fluorescent staining analog to hematoxylin and eosin intended for use on fresh, unsectioned tissues. Combined with optical sectioning fluorescence microscopy and pseudo-coloring algorithms, DRAQ5 and eosin ("D&E") enables very fast, non-destructive psuedohistological imaging of tissues at the point-of-acquisition with minimal tissue handling and processing. D&E was validated against H&E on a one-to-one basis on formalin-fixed paraffin-embedded and frozen section tissues of various human organs using standard epi-fluorescence microscopy, demonstrating high fidelity of the staining mechanism as an H&E analog. The method was then applied to fresh, whole 18G renal needle core biopsies and large needle core prostate biospecimen biopsies using fluorescence structured illumination optical sectioning microscopy. We demonstrate the ability to obtain high-resolution histology-like images of unsectioned, fresh tissues similar to subsequent H&E staining of the tissue. The application of D&E does not interfere with subsequent standard-of-care H&E staining and imaging, preserving the integrity of the tissue for thorough downstream analysis. These results indicate that this dual-stain pseudocoloring method could provide a real-time histology-like image at the time of acquisition and valuable objective tissue analysis for the clinician at the time of service.

Nondestructive Diagnosis of Kidney Cancer on 18-gauge Core Needle Renal Biopsy Using Dual-color Fluorescence Structured Illumination Microscopy.

OBJECTIVE: To present a novel imaging technique used for rapid, nondestructive histological assessment of renal neoplasias using a dual-component fluorescence stain and structured illumination microscopy (SIM). MATERIALS AND METHODS: After Institutional Review Board approval, 65 total biopsies were obtained from 19 patients undergoing partial or radical nephrectomy. Biopsies were stained with a dual-component fluorescent, and optically sectioned SIM images were obtained from the surface of the intact biopsies. Specimens were subsequently fixed and analyzed using hematoxylin and eosin (H&E) histopathologic methods and compared with SIM images. A single, board-certified pathologist blinded to specimens reviewed all SIM images and H&E slides, and determined the presence or absence of neoplasias. Results of blinded diagnosis of SIM were validated against traditional pathology. RESULTS: Of the 19 patients, 15 underwent robotic partial nephrectomies and 4 underwent laparoscopic nephrectomies. Indications included clinical suspicion of renal cell carcinoma. In total, 65 biopsy specimens were available for review. Twenty-one specimens were determined to be neoplastic on H&E, whereas 41 represented benign renal tissue. The final sensitivity and specificity of our study were 79.2% and 95.1%, respectively. CONCLUSION: SIM is a promising technology for rapid, near-patient, ex vivo renal biopsy assessment. By improving the ability to rapidly assess sufficiency of biopsy specimens and enabling immediate diagnostic capability, SIM aids in more effective biopsy performance, tissue triage, and patient counseling regarding management options. Additionally, because tissue is preserved, effective utilization of downstream diagnostic tests and molecular assessments are possible.

Gigapixel surface imaging of radical prostatectomy specimens for comprehensive detection of cancer-positive surgical margins using structured illumination microscopy.

Achieving cancer-free surgical margins in oncologic surgery is critical to reduce the need for additional adjuvant treatments and minimize tumor recurrence; however, there is a delicate balance between completeness of tumor removal and preservation of adjacent tissues critical for normal post-operative function. We sought to establish the feasibility of video-rate structured illumination microscopy (VR-SIM) of the intact removed tumor surface as a practical and non-destructive alternative to intra-operative frozen section pathology, using prostate cancer as an initial target. We present the first images of the intact human prostate surface obtained with pathologically-relevant contrast and subcellular detail, obtained in 24 radical prostatectomy specimens immediately after excision. We demonstrate that it is feasible to routinely image the full prostate circumference, generating gigapixel panorama images of the surface that are readily interpreted by pathologists. VR-SIM confirmed detection of positive surgical margins in 3 out of 4 prostates with pathology-confirmed adenocarcinoma at the circumferential surgical margin, and furthermore detected extensive residual cancer at the circumferential margin in a case post-operatively classified by histopathology as having negative surgical margins. Our results suggest that the increased surface coverage of VR-SIM could also provide added value for detection and characterization of positive surgical margins over traditional histopathology.

High-Resolution Rapid Diagnostic Imaging of Whole Prostate Biopsies Using Video-Rate Fluorescence Structured Illumination Microscopy.

Rapid assessment of prostate core biopsy pathology at the point-of-procedure could provide benefit in a variety of clinical situations. Even with advanced transrectal ultrasound guidance and saturation biopsy protocols, prostate cancer can be missed in up to half of all initial biopsy procedures. In addition, collection of tumor specimens for downstream histologic, molecular, and genetic analysis is hindered by low tumor yield due to inability to identify prostate cancer grossly. However, current point-of-procedure pathology protocols, such as frozen section analysis (FSA), are destructive and too time- and labor-intensive to be practical or economical. Ex vivo microscopy of the excised specimens, stained with fast-acting fluorescent histology dyes, could be an attractive nondestructive alternative to FSA. In this work, we report the first demonstration of video-rate structured illumination microscopy (VR-SIM) for rapid high-resolution diagnostic imaging of prostate biopsies in realistic point-of-procedure timeframes. Large mosaic images of prostate biopsies stained with acridine orange are rendered in seconds and contain excellent contrast and detail, exhibiting close correlation with corresponding hematoxylin and eosin histology. A clinically relevant review of VR-SIM images of 34 unfixed and uncut prostate core biopsies by two independent pathologists resulted in an area under the receiver operative curve (AUC) of 0.82-0.88, with a sensitivity ranging from 63% to 88% and a specificity ranging from 78% to 89%. When biopsies contained more than 5% tumor content, the sensitivity improved to 75% to 92%. The image quality, speed, minimal complexity, and ease of use of VR-SIM could prove to be features in favor of adoption as an alternative to destructive pathology at the point-of-procedure.

Feasibility of quantitative diffuse reflectance spectroscopy for targeted measurement of renal ischemia during laparoscopic partial nephrectomy.

Reduction of warm ischemia time during partial nephrectomy (PN) is critical to minimizing ischemic damage and improving postoperative kidney function, while maintaining tumor resection efficacy. Recently, methods for localizing the effects of warm ischemia to the region of the tumor via selective clamping of higher-order segmental artery branches have been shown to have superior outcomes compared with clamping the main renal artery. However, artery identification can prolong operative time and increase the blood loss and reduce the positive effects of selective ischemia. Quantitative diffuse reflectance spectroscopy (DRS) can provide a convenient, real-time means to aid in artery identification during laparoscopic PN. The feasibility of quantitative DRS for real-time longitudinal measurement of tissue perfusion and vascular oxygenation in laparoscopic nephrectomy was investigated in vivo in six Yorkshire swine kidneys (n=three animals ). DRS allowed for rapid identification of ischemic areas after selective vessel occlusion. In addition, the rates of ischemia induction and recovery were compared for main renal artery versus tertiary segmental artery occlusion, and it was found that the tertiary segmental artery occlusion trends toward faster recovery after ischemia, which suggests a potential benefit of selective ischemia. Quantitative DRS could provide a convenient and fast tool for artery identification and evaluation of the depth, spatial extent, and duration of selective tissue ischemia in laparoscopic PN.

Video-rate structured illumination microscopy for high-throughput imaging of large tissue areas.

We report the development of a structured illumination microscopy instrument specifically designed for the requirements for high-area-throughput, optically-sectioned imaging of large, fluorescently-stained tissue specimens. The system achieves optical sectioning frame-rates of up to 33 Hz (and pixel sampling rates of up to 138.4 MHz), by combining a fast, ferroelectric spatial light modulator for pattern generation with the latest large-format, high frame-rate scientific CMOS camera technology. Using a 10X 0.45 NA objective and a 7 mm/sec scan stage, we demonstrate 4.4 cm(2)/min area-throughput rates in bright tissue-simulating phantoms, and 2 cm(2)/min area-throughput rates in thick, highly-absorbing, fluorescently-stained muscle tissue, with 1.3 µm lateral resolution. We demonstrate high-contrast, high-resolution imaging of a fluorescently-stained 30.4 cm(2) bovine muscle specimen in 15 minutes comprising 7.55 gigapixels, demonstrating the feasibility of the approach for gigapixel imaging of large tissues in short timeframes, such as would be needed for intraoperative imaging of tumor resection specimens.