RESUMO
Deep learning methods are widely applied in digital pathology to address clinical challenges such as prognosis and diagnosis. As one of the most recent applications, deep models have also been used to extract molecular features from whole slide images. Although molecular tests carry rich information, they are often expensive, time-consuming, and require additional tissue to sample. In this paper, we propose tRNAsformer, an attention-based topology that can learn both to predict the bulk RNA-seq from an image and represent the whole slide image of a glass slide simultaneously. The tRNAsformer uses multiple instance learning to solve a weakly supervised problem while the pixel-level annotation is not available for an image. We conducted several experiments and achieved better performance and faster convergence in comparison to the state-of-the-art algorithms. The proposed tRNAsformer can assist as a computational pathology tool to facilitate a new generation of search and classification methods by combining the tissue morphology and the molecular fingerprint of the biopsy samples.
Assuntos
Algoritmos , Sequência de Bases , RNA-SeqRESUMO
Importance: For prostate cancer, Gleason grading of the biopsy specimen plays a pivotal role in determining case management. However, Gleason grading is associated with substantial interobserver variability, resulting in a need for decision support tools to improve the reproducibility of Gleason grading in routine clinical practice. Objective: To evaluate the ability of a deep learning system (DLS) to grade diagnostic prostate biopsy specimens. Design, Setting, and Participants: The DLS was evaluated using 752 deidentified digitized images of formalin-fixed paraffin-embedded prostate needle core biopsy specimens obtained from 3 institutions in the United States, including 1 institution not used for DLS development. To obtain the Gleason grade group (GG), each specimen was first reviewed by 2 expert urologic subspecialists from a multi-institutional panel of 6 individuals (years of experience: mean, 25 years; range, 18-34 years). A third subspecialist reviewed discordant cases to arrive at a majority opinion. To reduce diagnostic uncertainty, all subspecialists had access to an immunohistochemical-stained section and 3 histologic sections for every biopsied specimen. Their review was conducted from December 2018 to June 2019. Main Outcomes and Measures: The frequency of the exact agreement of the DLS with the majority opinion of the subspecialists in categorizing each tumor-containing specimen as 1 of 5 categories: nontumor, GG1, GG2, GG3, or GG4-5. For comparison, the rate of agreement of 19 general pathologists' opinions with the subspecialists' majority opinions was also evaluated. Results: For grading tumor-containing biopsy specimens in the validation set (n = 498), the rate of agreement with subspecialists was significantly higher for the DLS (71.7%; 95% CI, 67.9%-75.3%) than for general pathologists (58.0%; 95% CI, 54.5%-61.4%) (P < .001). In subanalyses of biopsy specimens from an external validation set (n = 322), the Gleason grading performance of the DLS remained similar. For distinguishing nontumor from tumor-containing biopsy specimens (n = 752), the rate of agreement with subspecialists was 94.3% (95% CI, 92.4%-95.9%) for the DLS and similar at 94.7% (95% CI, 92.8%-96.3%) for general pathologists (P = .58). Conclusions and Relevance: In this study, the DLS showed higher proficiency than general pathologists at Gleason grading prostate needle core biopsy specimens and generalized to an independent institution. Future research is necessary to evaluate the potential utility of using the DLS as a decision support tool in clinical workflows and to improve the quality of prostate cancer grading for therapy decisions.
Assuntos
Interpretação de Imagem Assistida por Computador , Gradação de Tumores/normas , Neoplasias da Próstata/diagnóstico , Adolescente , Adulto , Algoritmos , Inteligência Artificial , Biópsia com Agulha de Grande Calibre/métodos , Aprendizado Profundo , Humanos , Masculino , Neoplasias da Próstata/epidemiologia , Neoplasias da Próstata/patologia , Manejo de Espécimes , Estados Unidos/epidemiologia , Adulto JovemRESUMO
[This corrects the article DOI: 10.1038/s41746-019-0112-2.].
RESUMO
The microscopic assessment of tissue samples is instrumental for the diagnosis and staging of cancer, and thus guides therapy. However, these assessments demonstrate considerable variability and many regions of the world lack access to trained pathologists. Though artificial intelligence (AI) promises to improve the access and quality of healthcare, the costs of image digitization in pathology and difficulties in deploying AI solutions remain as barriers to real-world use. Here we propose a cost-effective solution: the augmented reality microscope (ARM). The ARM overlays AI-based information onto the current view of the sample in real time, enabling seamless integration of AI into routine workflows. We demonstrate the utility of ARM in the detection of metastatic breast cancer and the identification of prostate cancer, with latency compatible with real-time use. We anticipate that the ARM will remove barriers towards the use of AI designed to improve the accuracy and efficiency of cancer diagnosis.
Assuntos
Inteligência Artificial , Neoplasias da Mama/diagnóstico , Neoplasias/diagnóstico , Neoplasias da Próstata/diagnóstico , Neoplasias da Mama/patologia , Feminino , Humanos , Masculino , Microscopia/métodos , Estadiamento de Neoplasias , Neoplasias/patologia , Neoplasias da Próstata/patologiaRESUMO
For prostate cancer patients, the Gleason score is one of the most important prognostic factors, potentially determining treatment independent of the stage. However, Gleason scoring is based on subjective microscopic examination of tumor morphology and suffers from poor reproducibility. Here we present a deep learning system (DLS) for Gleason scoring whole-slide images of prostatectomies. Our system was developed using 112 million pathologist-annotated image patches from 1226 slides, and evaluated on an independent validation dataset of 331 slides. Compared to a reference standard provided by genitourinary pathology experts, the mean accuracy among 29 general pathologists was 0.61 on the validation set. The DLS achieved a significantly higher diagnostic accuracy of 0.70 (p = 0.002) and trended towards better patient risk stratification in correlations to clinical follow-up data. Our approach could improve the accuracy of Gleason scoring and subsequent therapy decisions, particularly where specialist expertise is unavailable. The DLS also goes beyond the current Gleason system to more finely characterize and quantitate tumor morphology, providing opportunities for refinement of the Gleason system itself.
RESUMO
The increasing availability of large institutional and public histopathology image datasets is enabling the searching of these datasets for diagnosis, research, and education. Although these datasets typically have associated metadata such as diagnosis or clinical notes, even carefully curated datasets rarely contain annotations of the location of regions of interest on each image. As pathology images are extremely large (up to 100,000 pixels in each dimension), further laborious visual search of each image may be needed to find the feature of interest. In this paper, we introduce a deep-learning-based reverse image search tool for histopathology images: Similar Medical Images Like Yours (SMILY). We assessed SMILY's ability to retrieve search results in two ways: using pathologist-provided annotations, and via prospective studies where pathologists evaluated the quality of SMILY search results. As a negative control in the second evaluation, pathologists were blinded to whether search results were retrieved by SMILY or randomly. In both types of assessments, SMILY was able to retrieve search results with similar histologic features, organ site, and prostate cancer Gleason grade compared with the original query. SMILY may be a useful general-purpose tool in the pathologist's arsenal, to improve the efficiency of searching large archives of histopathology images, without the need to develop and implement specific tools for each application.
RESUMO
CONTEXT.: Nodal metastasis of a primary tumor influences therapy decisions for a variety of cancers. Histologic identification of tumor cells in lymph nodes can be laborious and error-prone, especially for small tumor foci. OBJECTIVE.: To evaluate the application and clinical implementation of a state-of-the-art deep learning-based artificial intelligence algorithm (LYmph Node Assistant or LYNA) for detection of metastatic breast cancer in sentinel lymph node biopsies. DESIGN.: Whole slide images were obtained from hematoxylin-eosin-stained lymph nodes from 399 patients (publicly available Camelyon16 challenge dataset). LYNA was developed by using 270 slides and evaluated on the remaining 129 slides. We compared the findings to those obtained from an independent laboratory (108 slides from 20 patients/86 blocks) using a different scanner to measure reproducibility. RESULTS.: LYNA achieved a slide-level area under the receiver operating characteristic (AUC) of 99% and a tumor-level sensitivity of 91% at 1 false positive per patient on the Camelyon16 evaluation dataset. We also identified 2 "normal" slides that contained micrometastases. When applied to our second dataset, LYNA achieved an AUC of 99.6%. LYNA was not affected by common histology artifacts such as overfixation, poor staining, and air bubbles. CONCLUSIONS.: Artificial intelligence algorithms can exhaustively evaluate every tissue patch on a slide, achieving higher tumor-level sensitivity than, and comparable slide-level performance to, pathologists. These techniques may improve the pathologist's productivity and reduce the number of false negatives associated with morphologic detection of tumor cells. We provide a framework to aid practicing pathologists in assessing such algorithms for adoption into their workflow (akin to how a pathologist assesses immunohistochemistry results).
Assuntos
Neoplasias da Mama/patologia , Aprendizado Profundo , Interpretação de Imagem Assistida por Computador/métodos , Metástase Linfática/diagnóstico , Patologia Clínica/métodos , Feminino , Humanos , Patologistas , Biópsia de Linfonodo SentinelaRESUMO
BACKGROUND: Digital pathology enables remote access or consults and powerful image analysis algorithms. However, the slide digitization process can create artifacts such as out-of-focus (OOF). OOF is often only detected on careful review, potentially causing rescanning, and workflow delays. Although scan time operator screening for whole-slide OOF is feasible, manual screening for OOF affecting only parts of a slide is impractical. METHODS: We developed a convolutional neural network (ConvFocus) to exhaustively localize and quantify the severity of OOF regions on digitized slides. ConvFocus was developed using our refined semi-synthetic OOF data generation process and evaluated using seven slides spanning three different tissue and three different stain types, each of which were digitized using two different whole-slide scanner models ConvFocus's predictions were compared with pathologist-annotated focus quality grades across 514 distinct regions representing 37,700 35 µm × 35 µm image patches, and 21 digitized "z-stack" WSIs that contain known OOF patterns. RESULTS: When compared to pathologist-graded focus quality, ConvFocus achieved Spearman rank coefficients of 0.81 and 0.94 on two scanners and reproduced the expected OOF patterns from z-stack scanning. We also evaluated the impact of OOF on the accuracy of a state-of-the-art metastatic breast cancer detector and saw a consistent decrease in performance with increasing OOF. CONCLUSIONS: Comprehensive whole-slide OOF categorization could enable rescans before pathologist review, potentially reducing the impact of digitization focus issues on the clinical workflow. We show that the algorithm trained on our semi-synthetic OOF data generalizes well to real OOF regions across tissue types, stains, and scanners. Finally, quantitative OOF maps can flag regions that might otherwise be misclassified by image analysis algorithms, preventing OOF-induced errors.
RESUMO
Advances in the quality of whole-slide images have set the stage for the clinical use of digital images in anatomic pathology. Along with advances in computer image analysis, this raises the possibility for computer-assisted diagnostics in pathology to improve histopathologic interpretation and clinical care. To evaluate the potential impact of digital assistance on interpretation of digitized slides, we conducted a multireader multicase study utilizing our deep learning algorithm for the detection of breast cancer metastasis in lymph nodes. Six pathologists reviewed 70 digitized slides from lymph node sections in 2 reader modes, unassisted and assisted, with a wash-out period between sessions. In the assisted mode, the deep learning algorithm was used to identify and outline regions with high likelihood of containing tumor. Algorithm-assisted pathologists demonstrated higher accuracy than either the algorithm or the pathologist alone. In particular, algorithm assistance significantly increased the sensitivity of detection for micrometastases (91% vs. 83%, P=0.02). In addition, average review time per image was significantly shorter with assistance than without assistance for both micrometastases (61 vs. 116 s, P=0.002) and negative images (111 vs. 137 s, P=0.018). Lastly, pathologists were asked to provide a numeric score regarding the difficulty of each image classification. On the basis of this score, pathologists considered the image review of micrometastases to be significantly easier when interpreted with assistance (P=0.0005). Utilizing a proof of concept assistant tool, this study demonstrates the potential of a deep learning algorithm to improve pathologist accuracy and efficiency in a digital pathology workflow.
Assuntos
Neoplasias da Mama/patologia , Aprendizado Profundo , Diagnóstico por Computador/métodos , Interpretação de Imagem Assistida por Computador/métodos , Linfonodos/patologia , Patologia Clínica/métodos , Biópsia , Feminino , Humanos , Metástase Linfática , Micrometástase de Neoplasia , Variações Dependentes do Observador , Reconhecimento Automatizado de Padrão , Valor Preditivo dos Testes , Estudo de Prova de Conceito , Reprodutibilidade dos Testes , Fatores de Tempo , Fluxo de TrabalhoRESUMO
INTRODUCTION: The development and application of new molecular diagnostic assays based on next-generation sequencing and proteomics require improved methodologies for procurement of target cells from histological sections. Laser microdissection can successfully isolate distinct cells from tissue specimens based on visual selection for many research and clinical applications. However, this can be a daunting task when a large number of cells are required for molecular analysis or when a sizeable number of specimens need to be evaluated. MATERIALS AND METHODS: To improve the efficiency of the cellular identification process, we describe a microdissection workflow that leverages recently developed and open source image analysis algorithms referred to as computer-aided laser dissection (CALD). CALD permits a computer algorithm to identify the cells of interest and drive the dissection process. RESULTS: We describe several "use cases" that demonstrate the integration of image analytic tools probabilistic pairwise Markov model, ImageJ, spatially invariant vector quantization (SIVQ), and eSeg onto the ThermoFisher Scientific ArcturusXT and Leica LMD7000 microdissection platforms. CONCLUSIONS: The CALD methodology demonstrates the integration of image analysis tools with the microdissection workflow and shows the potential impact to clinical and life science applications.
RESUMO
Strategies to help improve the efficacy of the immune system against cancer represent an important innovation, with recent attention having focused on anti-programmed death (PD)-1/PD-ligand 1 (L1) monoclonal antibodies. Clinical trials have shown objective clinical activity of these agents (e.g., nivolumab, pembrolizumab) in several malignancies, including melanoma, non-small-cell lung cancer, bladder cancer, squamous head and neck cancer, renal cell cancer, ovarian cancer, microsatellite-unstable colorectal cancer, and Hodgkin's lymphoma. Expression of PD-L1 in the tumor microenvironment appears to be crucial for therapeutic activity, and initial trials suggested positive PD-L1 tumor expression was associated with higher response rates. However, subsequent observations have questioned the prospect of using PD-L1 expression as a biomarker for selecting patients for therapy, especially since many patients considered PD-L1-negative experience a benefit from treatment. Importantly, there is not yet a definitive test for determination of PD-L1 and a cut-off reference for PD-L1-positive status has not been established. Immunohistochemistry with different antibodies and different thresholds has been used to define PD-L1 positivity (1-50 %), with no clear superiority of one threshold over another for identifying which patients respond. Moreover, the type of cells on which PD-L1 expression is most relevant is not yet clear, with immune infiltrate cells and tumor cells both being used. In conclusion, while PD-L1 expression is often a predictive factor for treatment response, it must be complemented by other biomarkers or histopathologic features, such as the composition and amount of inflammatory cells in the tumor microenvironment and their functional status. Multi-parameter quantitative or semi-quantitative algorithms may become useful and reliable tools to guide patient selection.
Assuntos
Anticorpos Monoclonais/uso terapêutico , Antineoplásicos/uso terapêutico , Antígeno B7-H1/metabolismo , Terapia de Alvo Molecular , Neoplasias/tratamento farmacológico , Neoplasias/metabolismo , Receptor de Morte Celular Programada 1/antagonistas & inibidores , Biomarcadores Tumorais/metabolismo , Ensaios Clínicos como Assunto , Humanos , Seleção de Pacientes , Microambiente TumoralRESUMO
Precision medicine promises to enhance patient treatment through the use of emerging molecular technologies, including genomics, transcriptomics, and proteomics. However, current tools in surgical pathology lack the capability to efficiently isolate specific cell populations in complex tissues/tumors, which can confound molecular results. Expression microdissection (xMD) is an immuno-based cell/subcellular isolation tool that procures targets of interest from a cytological or histological specimen. In this study, we demonstrate the accuracy and precision of xMD by rapidly isolating immunostained targets, including cytokeratin AE1/AE3, p53, and estrogen receptor (ER) positive cells and nuclei from tissue sections. Other targets procured included green fluorescent protein (GFP) expressing fibroblasts, in situ hybridization positive Epstein-Barr virus nuclei, and silver stained fungi. In order to assess the effect on molecular data, xMD was utilized to isolate specific targets from a mixed population of cells where the targets constituted only 5% of the sample. Target enrichment from this admixed cell population prior to next-generation sequencing (NGS) produced a minimum 13-fold increase in mutation allele frequency detection. These data suggest a role for xMD in a wide range of molecular pathology studies, as well as in the clinical workflow for samples where tumor cell enrichment is needed, or for those with a relative paucity of target cells.
Assuntos
Sequenciamento de Nucleotídeos em Larga Escala/métodos , Microdissecção/métodos , Animais , Núcleo Celular/metabolismo , Epitélio/metabolismo , Humanos , Camundongos , Células NIH 3T3 , Coloração e RotulagemRESUMO
SIVQ-LCM is a new methodology that automates and streamlines the more traditional, user-dependent laser dissection process. It aims to create an advanced, rapidly customizable laser dissection platform technology. In this report, we describe the integration of the image analysis software Spatially Invariant Vector Quantization (SIVQ) onto the ArcturusXT instrument. The ArcturusXT system contains both an infrared (IR) and ultraviolet (UV) laser, allowing for specific cell or large area dissections. The principal goal is to improve the speed, accuracy, and reproducibility of the laser dissection to increase sample throughput. This novel approach facilitates microdissection of both animal and human tissues in research and clinical workflows.
Assuntos
Algoritmos , Processamento de Imagem Assistida por Computador/métodos , Microdissecção e Captura a Laser/métodos , Animais , Automação/métodos , Humanos , Microdissecção e Captura a Laser/instrumentação , Reconhecimento Automatizado de PadrãoRESUMO
Isolation of well-preserved pure cell populations is a prerequisite for sound studies of the molecular basis of any tissue-based biological phenomenon. This updated chapter reviews current methods for obtaining anatomically specific signals from molecules isolated from tissues, a basic requirement for productive linking of phenotype and genotype. The quality of samples isolated from tissue and used for molecular analysis is often glossed over or omitted from publications, making interpretation and replication of data difficult or impossible. Fortunately, recently developed techniques allow life scientists to better document and control the quality of samples used for a given assay, creating a foundation for improvement in this area. Tissue processing for molecular studies usually involves some or all of the following steps: tissue collection, gross dissection/identification, fixation, processing/embedding, storage/archiving, sectioning, staining, microdissection/annotation, and pure analyte labeling/identification and quantification. We provide a detailed comparison of some current tissue microdissection technologies and provide detailed example protocols for tissue component handling upstream and downstream from microdissection. We also discuss some of the physical and chemical issues related to optimal tissue processing and include methods specific to cytology specimens. We encourage each laboratory to use these as a starting point for optimization of their overall process of moving from collected tissue to high-quality, appropriately anatomically tagged scientific results. Improvement in this area will significantly increase life science quality and productivity. The chapter is divided into introduction, materials, protocols, and notes subheadings. Because many protocols are covered in each of these sections, information relating to a single protocol is not contiguous. To get the greatest benefit from this chapter, readers are advised to read through the entire chapter first, identify protocols appropriate to their laboratory for each step in their workflow, and then reread entries in each section pertaining to each of these single protocols.
Assuntos
Microdissecção/métodos , Separação Celular/métodos , Microdissecção/instrumentação , Ácidos Nucleicos/isolamento & purificação , Preservação Biológica/métodos , Coloração e RotulagemRESUMO
BACKGROUND: Last year, our pathology informatics fellowship added informatics-based interactive case studies to its existing educational platform of operational and research rotations, clinical conferences, a common core curriculum with an accompanying didactic course, and national meetings. METHODS: The structure of the informatics case studies was based on the traditional business school case study format. Three different formats were used, varying in length from short, 15-minute scenarios to more formal multiple hour-long case studies. Case studies were presented over the course of three retreats (Fall 2011, Winter 2012, and Spring 2012) and involved both local and visiting faculty and fellows. RESULTS: Both faculty and fellows found the case studies and the retreats educational, valuable, and enjoyable. From this positive feedback, we plan to incorporate the retreats in future academic years as an educational component of our fellowship program. CONCLUSIONS: Interactive case studies appear to be valuable in teaching several aspects of pathology informatics that are difficult to teach in more traditional venues (rotations and didactic class sessions). Case studies have become an important component of our fellowship's educational platform.
RESUMO
OBJECTIVE: In the past decade molecular diagnostics has changed the clinical management of lung adenocarcinoma patients. Molecular diagnostics, however, is largely dependent on the quantity and quality of the tumor DNA that is retrieved from the tissue or cytology samples. Frequently, patients are diagnosed on cytology specimens where the tumor cells are scattered within the cell block, making selecting for tumor enrichment difficult. In the past we have used laser capture microdissection (LCM) to select for pure populations of tumor cells to increase the sensitivity of molecular assays. This study explores several methods for semiautomated computer-guided LCM. STUDY DESIGN: Hematoxylin and eosin- or TTF-1-immunostained slides from a pleural effusion cell block with metastatic lung adenocarcinoma were used for LCM with either AutoScan or a recently described pattern-matching algorithm, spatially invariant vector quantization (SIVQ), to define morphologic predicates (vectors) to select cells of interest. RESULTS: We retrieved pure populations of tumor cells using both algorithm-guided LCM approaches with slight variations in cellular retrievals. Both methods were semiautomated, requiring minimum technical supervision. CONCLUSION: In this study we demonstrate the first semiautomated, computer-guided LCM of a cytology specimen using SIVQ and AutoScan, a first step towards the long-term goal of integrating LCM into the clinical cytology-molecular workflow.
Assuntos
Adenocarcinoma/diagnóstico , Citodiagnóstico , Microdissecção e Captura a Laser , Neoplasias Pulmonares/diagnóstico , Derrame Pleural Maligno/diagnóstico , Ácido Aspártico Endopeptidases/metabolismo , Automação , Biomarcadores Tumorais/metabolismo , Hematoxilina , Humanos , Técnicas Imunoenzimáticas , Proteínas Nucleares/metabolismo , Fator Nuclear 1 de Tireoide , Fatores de Transcrição/metabolismoRESUMO
BACKGROUND: Conventional tissue microarrays (TMAs) consist of cores of tissue inserted into a recipient paraffin block such that a tissue section on a single glass slide can contain numerous patient samples in a spatially structured pattern. Scanning TMAs into digital slides for subsequent analysis by computer-aided diagnostic (CAD) algorithms all offers the possibility of evaluating candidate algorithms against a near-complete repertoire of variable disease morphologies. This parallel interrogation approach simplifies the evaluation, validation, and comparison of such candidate algorithms. A recently developed digital tool, digital core (dCORE), and image microarray maker (iMAM) enables the capture of uniformly sized and resolution-matched images, with these representing key morphologic features and fields of view, aggregated into a single monolithic digital image file in an array format, which we define as an image microarray (IMA). We further define the TMA-IMA construct as IMA-based images derived from whole slide images of TMAs themselves. METHODS: Here we describe the first combined use of the previously described dCORE and iMAM tools, toward the goal of generating a higher-order image construct, with multiple TMA cores from multiple distinct conventional TMAs assembled as a single digital image montage. This image construct served as the basis of the carrying out of a massively parallel image analysis exercise, based on the use of the previously described spatially invariant vector quantization (SIVQ) algorithm. RESULTS: Multicase, multifield TMA-IMAs of follicular lymphoma and follicular hyperplasia were separately rendered, using the aforementioned tools. Each of these two IMAs contained a distinct spectrum of morphologic heterogeneity with respect to both tingible body macrophage (TBM) appearance and apoptotic body morphology. SIVQ-based pattern matching, with ring vectors selected to screen for either tingible body macrophages or apoptotic bodies, was subsequently carried out on the differing TMA-IMAs, with attainment of excellent discriminant classification between the two diagnostic classes. CONCLUSION: The TMA-IMA construct enables and accelerates high-throughput multicase, multifield based image feature discovery and classification, thus simplifying the development, validation, and comparison of CAD algorithms in settings where the heterogeneity of diagnostic feature morphologic is a significant factor.