Colorectal cancer histopathology image analysis: A comparative study of prognostic values of automatically extracted morphometric nuclear features in multispectral and red-blue-green imagery.

OBJECTIVES: Multispectral imaging (MSI) has been utilized to predict the prognosis of colorectal cancer (CRC) patients, however, our understanding of the prognostic value of nuclear morphological parameters of bright-field MSI in CRC is still limited. This study was designed to compare the efficiency of MSI and standard red-green-blue (RGB) images in predicting the prognosis of CRC. METHODS: We compared the efficiency of MS and conventional RGB images on the quantitative assessment of hematoxylin-eosin (HE) stained histopathology images. A pipeline was developed using a pixel-wise support vector machine (SVM) classifier for gland-stroma segmentation, and a marker-controlled watershed algorithm was used for nuclei segmentation. The correlation between extracted morphological parameters and the five-year disease-free survival (5-DFS) was analyzed. RESULTS: Forty-seven nuclear morphological parameters were extracted in total. Based on Kaplan-Meier analysis, eight features derived from MS images and seven featured derived from RGB images were significantly associated with 5-DFS, respectively. Compared with RGB images, MSI showed higher accuracy, precision, and Dice index in nuclei segmentation. Multivariate analysis indicated that both integrated parameters 1 (factors negatively correlated with CRC prognosis including nuclear number, circularity, eccentricity, major axis length) and 2 (factors positively correlated with CRC prognosis including nuclear average area, area perimeter, total area/total perimeter ratio, average area/perimeter ratio) in MS images were independent prognostic factors of 5-DFS, in contrast with only integrated parameter 1 (P<0.001) in RGB images. More importantly, the quantification of HE-stained MS images displayed higher accuracy in predicting 5-DFS compared with RGB images (76.9% vs 70.9%). CONCLUSIONS: Quantitative evaluation of HE-stained MS images could yield more information and better predictive performance for CRC prognosis than conventional RGB images, thereby contributing to precision oncology.

Dual encoder network with transformer-CNN for multi-organ segmentation.

Medical image segmentation is a critical step in many imaging applications. Automatic segmentation has gained extensive concern using a convolutional neural network (CNN). However, the traditional CNN-based methods fail to extract global and long-range contextual information due to local convolution operation. Transformer overcomes the limitation of CNN-based models. Inspired by the success of transformers in computer vision (CV), many researchers focus on designing the transformer-based U-shaped method in medical image segmentation. The transformer-based approach cannot effectively capture the fine-grained details. This paper proposes a dual encoder network with transformer-CNN for multi-organ segmentation. The new segmentation framework takes full advantage of CNN and transformer to enhance the segmentation accuracy. The Swin-transformer encoder extracts global information, and the CNN encoder captures local information. We introduce fusion modules to fuse convolutional features and the sequence of features from the transformer. Feature fusion is concatenated through the skip connection to smooth the decision boundary effectively. We extensively evaluate our method on the synapse multi-organ CT dataset and the automated cardiac diagnosis challenge (ACDC) dataset. The results demonstrate that the proposed method achieves Dice similarity coefficient (DSC) metrics of 80.68% and 91.12% on the synapse multi-organ CT and ACDC datasets, respectively. We perform the ablation studies on the ACDC dataset, demonstrating the effectiveness of critical components of our method. Our results match the ground-truth boundary more consistently than the existing models. Our approach gains more accurate results on challenging 2D images for multi-organ segmentation. Compared with the state-of-the-art methods, our proposed method achieves superior performance in multi-organ segmentation tasks. Graphical Abstract The key process in medical image segmentation.

Contour-aware semantic segmentation network with spatial attention mechanism for medical image.

Medical image segmentation is a critical and important step for developing computer-aided system in clinical situations. It remains a complicated and challenging task due to the large variety of imaging modalities and different cases. Recently, Unet has become one of the most popular deep learning frameworks because of its accurate performance in biomedical image segmentation. In this paper, we propose a contour-aware semantic segmentation network, which is an extension of Unet, for medical image segmentation. The proposed method includes a semantic branch and a detail branch. The semantic branch focuses on extracting the semantic features from shallow and deep layers; the detail branch is used to enhance the contour information implied in the shallow layers. In order to improve the representation capability of the network, a MulBlock module is designed to extract semantic information with different receptive fields. Spatial attention module (CAM) is used to adaptively suppress the redundant features. In comparison with the state-of-the-art methods, our method achieves a remarkable performance on several public medical image segmentation challenges.

Quantum dots-based double imaging combined with organic dye imaging to establish an automatic computerized method for cancer Ki67 measurement.

As a widely used proliferative marker, Ki67 has important impacts on cancer prognosis, especially for breast cancer (BC). However, variations in analytical practice make it difficult for pathologists to manually measure Ki67 index. This study is to establish quantum dots (QDs)-based double imaging of nuclear Ki67 as red signal by QDs-655, cytoplasmic cytokeratin (CK) as yellow signal by QDs-585, and organic dye imaging of cell nucleus as blue signal by 4',6-diamidino-2-phenylindole (DAPI), and to develop a computer-aided automatic method for Ki67 index measurement. The newly developed automatic computerized Ki67 measurement could efficiently recognize and count Ki67-positive cancer cell nuclei with red signals and cancer cell nuclei with blue signals within cancer cell cytoplasmic with yellow signals. Comparisons of computerized Ki67 index, visual Ki67 index, and marked Ki67 index for 30 patients of 90 images with Ki67 ≤ 10% (low grade), 10% < Ki67 < 50% (moderate grade), and Ki67 ≥ 50% (high grade) showed computerized Ki67 counting is better than visual Ki67 counting, especially for Ki67 low and moderate grades. Based on QDs-based double imaging and organic dye imaging on BC tissues, this study successfully developed an automatic computerized Ki67 counting method to measure Ki67 index.

Application of multispectral imaging in quantitative immunohistochemistry study of breast cancer: a comparative study.

Multispectral imaging (MSI) based on imaging and spectroscopy, as relatively novel to the field of histopathology, has been used in biomedical multidisciplinary researches. We analyzed and compared the utility of multispectral (MS) versus conventional red-green-blue (RGB) images for immunohistochemistry (IHC) staining to explore the advantages of MSI in clinical-pathological diagnosis. The MS images acquired of IHC-stained membranous marker human epidermal growth factor receptor 2 (HER2), cytoplasmic marker cytokeratin5/6 (CK5/6), and nuclear marker estrogen receptor (ER) have higher resolution, stronger contrast, and more accurate segmentation than the RGB images. The total signal optical density (OD) values for each biomarker were higher in MS images than in RGB images (all P < 0.05). Moreover, receiver operator characteristic (ROC) analysis revealed that a greater area under the curve (AUC), higher sensitivity, and specificity in evaluation of HER2 gene were achieved by MS images (AUC = 0.91, 89.1 %, 83.2 %) than RGB images (AUC = 0.87, 84.5, and 81.8 %). There was no significant difference between quantitative results of RGB images and clinico-pathological characteristics (P > 0.05). However, by quantifying MS images, the total signal OD values of HER2 positive expression were correlated with lymph node status and histological grades (P = 0.02 and 0.04). Additionally, the consistency test results indicated the inter-observer agreement was more robust in MS images for HER2 (inter-class correlation coefficient (ICC) = 0.95, r s = 0.94), CK5/6 (ICC = 0.90, r s = 0.88), and ER (ICC = 0.94, r s = 0.94) (all P < 0.001) than that in RGB images for HER2 (ICC = 0.91, r s = 0.89), CK5/6 (ICC = 0.85, r s = 0.84), and ER (ICC = 0.90, r s = 0.89) (all P < 0.001). Our results suggest that the application of MS images in quantitative IHC analysis could obtain higher accuracy, reliability, and more information of protein expression in relation to clinico-pathological characteristics versus conventional RGB images. It may become an optimal IHC digital imaging system used in quantitative pathology.

[Computer-aided Prognosis for Breast Cancer Based on Hematoxylin & Eosin Histopathology Image].

Quantitatively analyzing hematoxylin &eosin(H&E)histopathology images is an emerging field attracting increasing attentions in recent years.This paper reviews the application of computer-aided image analysis in breast cancer prognosis.The traditional prognosis based on H&E histopathology image for breast cancer is firstly sketched,followed by a detailed description of the workflow of computer-aided prognosis including image acquisition,image preprocessing,regions of interest detection and object segmentation,feature extraction,and computer-aided prognosis.In the end,major technical challenges and future directions in this field are summarized.

New breast cancer prognostic factors identified by computer-aided image analysis of HE stained histopathology images.

Computer-aided image analysis (CAI) can help objectively quantify morphologic features of hematoxylin-eosin (HE) histopathology images and provide potentially useful prognostic information on breast cancer. We performed a CAI workflow on 1,150 HE images from 230 patients with invasive ductal carcinoma (IDC) of the breast. We used a pixel-wise support vector machine classifier for tumor nests (TNs)-stroma segmentation, and a marker-controlled watershed algorithm for nuclei segmentation. 730 morphologic parameters were extracted after segmentation, and 12 parameters identified by Kaplan-Meier analysis were significantly associated with 8-year disease free survival (P < 0.05 for all). Moreover, four image features including TNs feature (HR 1.327, 95%CI [1.001-1.759], P = 0.049), TNs cell nuclei feature (HR 0.729, 95%CI [0.537-0.989], P = 0.042), TNs cell density (HR 1.625, 95%CI [1.177-2.244], P = 0.003), and stromal cell structure feature (HR 1.596, 95%CI [1.142-2.229], P = 0.006) were identified by multivariate Cox proportional hazards model to be new independent prognostic factors. The results indicated that CAI can assist the pathologist in extracting prognostic information from HE histopathology images for IDC. The TNs feature, TNs cell nuclei feature, TNs cell density, and stromal cell structure feature could be new prognostic factors.

Morphological study and comprehensive cellular constituents of milky spots in the human omentum.

OBJECTIVE: To analyze morphological features of omental milky spots (MS). METHOD: Hematoxylin-eosin staining and immunohistochemistry technique were used to study the omental MS of gastric cancer (GC) patients and rectal cancer (RC) patients. We focused on morphological features of MS and conducted quantitative analysis on the cells number and cellular constituents. Differences in MS parameters between GC and RC were also analyzed. RESULTS: Various shapes of MS were mainly round, oval, irregular form in the adipose and perivascular annular. The median MS perimeter was 2752 (range 817~7753) computer-based pixels. The median value of immune cells in one MS was 141 (43~650), comprising T lymphocytes (46.1%), B lymphocytes (28.4%), macrophages (12.4%) and other immune cells (13.1%). Relatively high density of vessels in MS could be calculated by micro-vessel density (MVD) as 4 (0~13). The median value of mesothelial cells loosely arranged in the surface layer was 5 (0~51). There were no significant differences in MS perimeter, MVD, the number of mesothelial cells, total immune cells, T lymphocytes and macrophages between GC and RC (P>0.05), while the number of MS B lymphocytes in RC was significantly higher than that in GC (P<0.001). CONCLUSION: MS are primary immune tissues in the omentum and structural bases for development and progression of peritoneal dissemination of GC and RC. Analyzing the morphology and cellular constituents could help understanding the mechanism of peritoneal metastasis.

Computer-based image studies on tumor nests mathematical features of breast cancer and their clinical prognostic value.

BACKGROUND: The expending and invasive features of tumor nests could reflect the malignant biological behaviors of breast invasive ductal carcinoma. Useful information on cancer invasiveness hidden within tumor nests could be extracted and analyzed by computer image processing and big data analysis. METHODS: Tissue microarrays from invasive ductal carcinoma (n = 202) were first stained with cytokeratin by immunohistochemical method to clearly demarcate the tumor nests. Then an expert-aided computer analysis system was developed to study the mathematical and geometrical features of the tumor nests. Computer recognition system and imaging analysis software extracted tumor nests information, and mathematical features of tumor nests were calculated. The relationship between tumor nests mathematical parameters and patients' 5-year disease free survival was studied. RESULTS: There were 8 mathematical parameters extracted by expert-aided computer analysis system. Three mathematical parameters (number, circularity and total perimeter) with area under curve >0.5 and 4 mathematical parameters (average area, average perimeter, total area/total perimeter, average (area/perimeter)) with area under curve <0.5 in ROC analysis were combined into integrated parameter 1 and integrated parameter 2, respectively. Multivariate analysis showed that integrated parameter 1 (P = 0.040) was independent prognostic factor of patients' 5-year disease free survival. The hazard risk ratio of integrated parameter 1 was 1.454 (HR 95% CI [1.017-2.078]), higher than that of N stage (HR 1.396, 95% CI [1.125-1.733]) and hormone receptor status (HR 0.575, 95% CI [0.353-0.936]), but lower than that of histological grading (HR 3.370, 95% CI [1.125-5.364]) and T stage (HR 1.610, 95% CI [1.026 -2.527]). CONCLUSIONS: This study indicated integrated parameter 1 of mathematical features (number, circularity and total perimeter) of tumor nests could be a useful parameter to predict the prognosis of early stage breast invasive ductal carcinoma.