A study of criteria for grading follicular lymphoma using a cell type classifier from pathology images based on complementary-label learning.

We propose a criterion for grading follicular lymphoma that is consistent with the intuitive evaluation, which is conducted by experienced pathologists. A criterion for grading follicular lymphoma is defined by the World Health Organization (WHO) based on the number of centroblasts and centrocytes within the field of view. However, the WHO criterion is not often used in clinical practice because it is impractical for pathologists to visually identify the cell type of each cell and count the number of centroblasts and centrocytes. Hence, based on the widespread use of digital pathology, we make it practical to identify and count the cell type by using image processing and then construct a criterion for grading based on the number of cells. Here, the problem is that labeling the cell type is not easy even for experienced pathologists. To alleviate this problem, we build a new dataset for cell type classification, which contains the pathologists' confusion records during labeling, and we construct the cell type classifier using complementary-label learning from this dataset. Then we propose a criterion based on the composition ratio of cell types that is consistent with the pathologists' grading. Our experiments demonstrate that the classifier can accurately identify cell types and the proposed criterion is more consistent with the pathologists' grading than the current WHO criterion.

Multimodal Gated Mixture of Experts Using Whole Slide Image and Flow Cytometry for Multiple Instance Learning Classification of Lymphoma.

In this study, we present a deep-learning-based multimodal classification method for lymphoma diagnosis in digital pathology, which utilizes a whole slide image (WSI) as the primary image data and flow cytometry (FCM) data as auxiliary information. In pathological diagnosis of malignant lymphoma, FCM serves as valuable auxiliary information during the diagnosis process, offering useful insights into predicting the major class (superclass) of subtypes. By incorporating both images and FCM data into the classification process, we can develop a method that mimics the diagnostic process of pathologists, enhancing the explainability. In order to incorporate the hierarchical structure between superclasses and their subclasses, the proposed method utilizes a network structure that effectively combines the mixture of experts (MoE) and multiple instance learning (MIL) techniques, where MIL is widely recognized for its effectiveness in handling WSIs in digital pathology. The MoE network in the proposed method consists of a gating network for superclass classification and multiple expert networks for (sub)class classification, specialized for each superclass. To evaluate the effectiveness of our method, we conducted experiments involving a six-class classification task using 600 lymphoma cases. The proposed method achieved a classification accuracy of 72.3%, surpassing the 69.5% obtained through the straightforward combination of FCM and images, as well as the 70.2% achieved by the method using only images. Moreover, the combination of multiple weights in the MoE and MIL allows for the visualization of specific cellular and tumor regions, resulting in a highly explanatory model that cannot be attained with conventional methods. It is anticipated that by targeting a larger number of classes and increasing the number of expert networks, the proposed method could be effectively applied to the real problem of lymphoma diagnosis.

The Detection of Neoplastic Cells Using Objective Cytomorphologic Parameters in Malignant Lymphoma.

Pathologic evaluation is the most crucial method for diagnosing malignant lymphomas. However, there are no established diagnostic criteria for evaluating pathologic morphology. We manually circled cell nuclei in the lesions of 10 patients with diffuse large B-cell lymphoma (DLBCL), follicular lymphoma, and reactive lymphadenitis. Seventeen parameters related to nuclear shape, color, and other characteristics were measured. We attempted to compare the statistical differences between these subtypes and extract distinctive disease-specific populations on the basis of these parameters. Statistically significant differences were observed between the different types of lymphoma for many of the 17 parameters. Through t-distributed stochastic neighbor embedding analysis, we extracted a cluster of cells that showed distinctive features of DLBCL and were not found in follicular lymphoma or reactive lymphadenitis. We created a decision tree to identify the characteristics of the cells within that cluster. Based on a 5-fold cross-validation study, the average sensitivity, specificity, and accuracy obtained were 84.1%, 98.4%, and 97.3%, respectively. A similar result was achieved using a validation experiment. Important parameters that indicate the features of DLBCL include Area, ConcaveCount, MaxGray, and ModeGray. By quantifying pathologic morphology, it was possible to objectively represent the cell morphology specific to each lymphoma subtype using quantitative indicators. The quantified morphologic information has the potential to serve as a reproducible and flexible diagnostic tool.

Case-based similar image retrieval for weakly annotated large histopathological images of malignant lymphoma using deep metric learning.

In the present study, we propose a novel case-based similar image retrieval (SIR) method for hematoxylin and eosin (H&E) stained histopathological images of malignant lymphoma. When a whole slide image (WSI) is used as an input query, it is desirable to be able to retrieve similar cases by focusing on image patches in pathologically important regions such as tumor cells. To address this problem, we employ attention-based multiple instance learning, which enables us to focus on tumor-specific regions when the similarity between cases is computed. Moreover, we employ contrastive distance metric learning to incorporate immunohistochemical (IHC) staining patterns as useful supervised information for defining appropriate similarity between heterogeneous malignant lymphoma cases. In the experiment with 249 malignant lymphoma patients, we confirmed that the proposed method exhibited higher evaluation measures than the baseline case-based SIR methods. Furthermore, the subjective evaluation by pathologists revealed that our similarity measure using IHC staining patterns is appropriate for representing the similarity of H&E stained tissue images for malignant lymphoma.

Transformer-based personalized attention mechanism for medical images with clinical records.

In medical image diagnosis, identifying the attention region, i.e., the region of interest for which the diagnosis is made, is an important task. Various methods have been developed to automatically identify target regions from given medical images. However, in actual medical practice, the diagnosis is made based on both the images and various clinical records. Consequently, pathologists examine medical images with prior knowledge of the patients and the attention regions may change depending on the clinical records. In this study, we propose a method, called the Personalized Attention Mechanism (PersAM) method, by which the attention regions in medical images according to the clinical records. The primary idea underlying the PersAM method is the encoding of the relationships between medical images and clinical records using a variant of the Transformer architecture. To demonstrate the effectiveness of the PersAM method, we applied it to a large-scale digital pathology problem involving identifying the subtypes of 842 malignant lymphoma patients based on their gigapixel whole-slide images and clinical records.

Subtype classification of malignant lymphoma using immunohistochemical staining pattern.

PURPOSE: For the image classification problem, the construction of appropriate training data is important for improving the generalization ability of the classifier in particular when the size of the training data is small. We propose a method that quantitatively evaluates the typicality of a hematoxylin-and-eosin (H&E)-stained tissue slide from a set of immunohistochemical (IHC) stains and applies the typicality to instance selection for the construction of classifiers that predict the subtype of malignant lymphoma to improve the generalization ability. METHODS: We define the typicality of the H&E-stained tissue slides by the ratio of the probability density of the IHC staining patterns on low-dimensional embedded space. Employing a multiple-instance-learning-based convolutional neural network for the construction of the subtype classifier without the annotations indicating cancerous regions in whole slide images, we select the training data by referring to the evaluated typicality to improve the generalization ability. We demonstrate the effectiveness of the instance selection based on the proposed typicality in a three-class subtype classification of 262 malignant lymphoma cases. RESULTS: In the experiment, we confirmed that the subtypes of typical instances could be predicted more accurately than those of atypical instances. Furthermore, it was confirmed that instance selection for the training data based on the proposed typicality improved the generalization ability of the classifier, wherein the classification accuracy was improved from 0.664 to 0.683 compared with the baseline method when the training data was constructed focusing on typical instances. CONCLUSION: The experimental results showed that the typicality of the H&E-stained tissue slides computed from IHC staining patterns is useful as a criterion for instance selection to enhance the generalization ability, and this typicality could be employed for instance selection under some practical limitations.

Robust 3D image reconstruction of pancreatic cancer tumors from histopathological images with different stains and its quantitative performance evaluation.

PURPOSE: Histopathological imaging is widely used for the analysis and diagnosis of multiple diseases. Several methods have been proposed for the 3D reconstruction of pathological images, captured from thin sections of a given specimen, which get nonlinearly deformed due to the preparation process. The majority of the available methods for registering such images use the degree of matching of adjacent images as the criteria for registration, which can result in unnatural deformations of the anatomical structures. Moreover, most methods assume that the same staining is used for all images, when in fact multiple staining is usually applied in order to enhance different structures in the images. METHODS: This paper proposes a non-rigid 3D reconstruction method based on the assumption that internal structures on the original tissue must be smooth and continuous. Landmarks are detected along anatomical structures using template matching based on normalized cross-correlation (NCC), forming jagged shape trajectories that traverse several slices. The registration process smooths out these trajectories and deforms the images accordingly. Artifacts are automatically handled by using the confidence of the NCC in order to reject unreliable landmarks. RESULTS: The proposed method was applied to a large series of histological sections from the pancreas of a KPC mouse. Some portions were dyed primarily with HE stain, while others were dyed alternately with HE, CK19, MT and Ki67 stains. A new evaluation method is proposed to quantitatively evaluate the smoothness and isotropy of the obtained reconstructions, both for single and multiple staining. CONCLUSIONS: The experimental results show that the proposed method produces smooth and nearly isotropic 3D reconstructions of pathological images with either single or multiple stains. From these reconstructions, microanatomical structures enhanced by different stains can be simultaneously observed.