A Preliminary Investigation into Search and Matching for Tumor Discrimination in World Health Organization Breast Taxonomy Using Deep Networks.

Breast cancer is one of the most common cancers affecting women worldwide. It includes a group of malignant neoplasms with a variety of biological, clinical, and histopathologic characteristics. There are more than 35 different histologic forms of breast lesions that can be classified and diagnosed histologically according to cell morphology, growth, and architecture patterns. Recently, deep learning, in the field of artificial intelligence, has drawn a lot of attention for the computerized representation of medical images. Searchable digital atlases can provide pathologists with patch-matching tools, allowing them to search among evidently diagnosed and treated archival cases, a technology that may be regarded as computational second opinion. In this study, we indexed and analyzed the World Health Organization breast taxonomy (Classification of Tumors fifth ed.) spanning 35 tumor types. We visualized all tumor types using deep features extracted from a state-of-the-art deep-learning model, pretrained on millions of diagnostic histopathology images from the Cancer Genome Atlas repository. Furthermore, we tested the concept of a digital "atlas" as a reference for search and matching with rare test cases. The patch similarity search within the World Health Organization breast taxonomy data reached >88% accuracy when validating through "majority vote" and >91% accuracy when validating using top n tumor types. These results show for the first time that complex relationships among common and rare breast lesions can be investigated using an indexed digital archive.

Immunohistochemistry Biomarkers-Guided Image Search for Histopathology.

Medical practitioners use a number of diagnostic tests to make a reliable diagnosis. Traditionally, Haematoxylin and Eosin (H&E) stained glass slides have been used for cancer diagnosis and tumor detection. However, recently a variety of immunohistochemistry (IHC) stained slides can be requested by pathologists to examine and confirm diagnoses for determining the subtype of a tumor when this is difficult using H&E slides only. Deep learning (DL) has received a lot of interest recently for image search engines to extract features from tissue regions, which may or may not be the target region for diagnosis. This approach generally fails to capture high-level patterns corresponding to the malignant or abnormal content of histopathology images. In this work, we are proposing a targeted image search approach, inspired by the pathologists' workflow, which may use information from multiple IHC biomarker images when available. These IHC images could be aligned, filtered, and merged together to generate a composite biomarker image (CBI) that could eventually be used to generate an attention map to guide the search engine for localized search. In our experiments, we observed that an IHC-guided image search engine can retrieve relevant data more accurately than a conventional (i.e., H&E-only) search engine without IHC guidance. Moreover, such engines are also able to accurately conclude the subtypes through majority votes.

Composite Biomarker Image for Advanced Visualization in Histopathology.

Immunohistochemistry (IHC) biomarkers are essential tools for reliable cancer diagnosis and subtyping. It requires cross-staining comparison among Whole Slide Images (WSIs) of IHCs and hematoxylin and eosin (H&E) slides. Currently, pathologists examine the visually co-localized areas across IHC and H&E glass slides for a final diagnosis, which is a tedious and challenging task. Moreover, visually inspecting different IHC slides back and forth to analyze local co-expressions is inherently subjective and prone to error, even when carried out by experienced pathologists. Relying on digital pathology, we propose "Composite Biomarker Image" (CBI) in this work. CBI is a single image that can be composed using different filtered IHC biomarker images for better visualization. We present a CBI image produced in two steps by the proposed solution for better visualization and hence more efficient clinical workflow. In the first step, IHC biomarker images are aligned with the H&E images using one coordinate system and orientation. In the second step, the positive or negative IHC regions from each biomarker image (based on the pathologists' recommendation) are filtered and combined into one image using a fuzzy inference system. For evaluation, the resulting CBI images, from the proposed system, were evaluated qualitatively by the expert pathologists. The CBI concept helps the pathologists to identify the suspected target tissues more easily, which could be further assessed by examining the actual WSIs at the same suspected regions.

Class-imbalanced Unsupervised and Semi-Supervised Domain Adaptation for Histopathology Images.

In dealing with the lack of sufficient annotated data and in contrast to supervised learning, unsupervised, self-supervised, and semi-supervised domain adaptation methods are promising approaches, enabling us to transfer knowledge from rich labeled source domains to different (but related) unlabeled target domains, reducing distribution discrepancy between the source and target domains. However, most existing domain adaptation methods do not consider the imbalanced nature of the real-world data, affecting their performance in practice. We propose to overcome this limitation by proposing a novel domain adaptation approach that includes two modifications to the existing models. Firstly, we leverage the focal loss function in response to class-imbalanced labeled data in the source domain. Secondly, we introduce a novel co-training approach to involve pseudo-labeled target data points in the training process. Experiments show that the proposed model can be effective in transferring knowledge from source to target domain. As an example, we use the classification of prostate cancer images into low-cancerous and high-cancerous regions.

Automatic Multi-Stain Registration of Whole Slide Images in Histopathology.

Joint analysis of multiple biomarker images and tissue morphology is important for disease diagnosis, treatment planning and drug development. It requires cross-staining comparison among Whole Slide Images (WSIs) of immune-histochemical and hematoxylin and eosin (H&E) microscopic slides. However, automatic, and fast cross-staining alignment of enormous gigapixel WSIs at single-cell precision is challenging. In addition to morphological deformations introduced during slide preparation, there are large variations in cell appearance and tissue morphology across different staining. In this paper, we propose a two-step automatic feature-based cross-staining WSI alignment to assist localization of even tiny metastatic foci in the assessment of lymph node. Image pairs were aligned allowing for translation, rotation, and scaling. The registration was performed automatically by first detecting landmarks in both images, using the scale-invariant image transform (SIFT), followed by the fast sample consensus (FSC) protocol for finding point correspondences and finally aligned the images. The Registration results were evaluated using both visual and quantitative criteria using the Jaccard index. The average Jaccard similarity index of the results produced by the proposed system is 0.942 when compared with the manual registration.

FPGA implementation of real-time SENSE reconstruction using pre-scan and Emaps sensitivities.

Sensitivity Encoding (SENSE) is a widely used technique in Parallel Magnetic Resonance Imaging (MRI) to reduce scan time. Reconfigurable hardware based architecture for SENSE can potentially provide image reconstruction with much less computation time. Application specific hardware platform for SENSE may dramatically increase the power efficiency of the system and can decrease the execution time to obtain MR images. A new implementation of SENSE on Field Programmable Gate Array (FPGA) is presented in this study, which provides real-time SENSE reconstruction right on the receiver coil data acquisition system with no need to transfer the raw data to the MRI server, thereby minimizing the transmission noise and memory usage. The proposed SENSE architecture can reconstruct MR images using receiver coil sensitivity maps obtained using pre-scan and eigenvector (E-maps) methods. The results show that the proposed system consumes remarkably less computation time for SENSE reconstruction, i.e., 0.164ms @ 200MHz, while maintaining the quality of the reconstructed images with good mean SNR (29+ dB), less RMSE (<5×10-2) and comparable artefact power (<9×10-4) to conventional SENSE reconstruction. A comparison of the center line profiles of the reconstructed and reference images also indicates a good quality of the reconstructed images. Furthermore, the results indicate that the proposed architectural design can prove to be a significant tool for SENSE reconstruction in modern MRI scanners and its low power consumption feature can be remarkable for portable MRI scanners.