Improving the diagnosis of ductal carcinoma in situ with microinvasion without immunohistochemistry: An innovative method with H&E-stained and multiphoton microscopy images.

Ductal carcinoma in situ with microinvasion (DCISM) is a challenging subtype of breast cancer with controversial invasiveness and prognosis. Accurate diagnosis of DCISM from ductal carcinoma in situ (DCIS) is crucial for optimal treatment and improved clinical outcomes. However, there are often some suspicious small cancer nests in DCIS, and it is difficult to diagnose the presence of intact myoepithelium by conventional hematoxylin and eosin (H&E) stained images. Although a variety of biomarkers are available for immunohistochemical (IHC) staining of myoepithelial cells, no single biomarker is consistently sensitive to all tumor lesions. Here, we introduced a new diagnostic method that provides rapid and accurate diagnosis of DCISM using multiphoton microscopy (MPM). Suspicious foci in H&E-stained images were labeled as regions of interest (ROIs), and the nuclei within these ROIs were segmented using a deep learning model. MPM was used to capture images of the ROIs in H&E-stained sections. The intensity of two-photon excitation fluorescence (TPEF) in the myoepithelium was significantly different from that in tumor parenchyma and tumor stroma. Through the use of MPM, the myoepithelium and basement membrane can be easily observed via TPEF and second-harmonic generation (SHG), respectively. By fusing the nuclei in H&E-stained images with MPM images, DCISM can be differentiated from suspicious small cancer clusters in DCIS. The proposed method demonstrated good consistency with the cytokeratin 5/6 (CK5/6) myoepithelial staining method (kappa coefficient = 0.818).

Exploration of the relationship between tumor-infiltrating lymphocyte score and histological grade in breast cancer.

BACKGROUND: The histological grade is an important factor in the prognosis of invasive breast cancer and is vital to accurately identify the histological grade and reclassify of Grade2 status in breast cancer patients. METHODS: In this study, data were collected from 556 invasive breast cancer patients, and then randomly divided into training cohort (n = 335) and validation cohort (n = 221). All patients were divided into actual low risk group (Grade1) and high risk group (Grade2/3) based on traditional histological grade, and tumor-infiltrating lymphocyte score (TILs-score) obtained from multiphoton images, and the TILs assessment method proposed by International Immuno-Oncology Biomarker Working Group (TILs-WG) were also used to differentiate between high risk group and low risk group of histological grade in patients with invasive breast cancer. Furthermore, TILs-score was used to reclassify Grade2 (G2) into G2 /Low risk and G2/High risk. The coefficients for each TILs in the training cohort were retrieved using ridge regression and TILs-score was created based on the coefficients of the three kinds of TILs. RESULTS: Statistical analysis shows that TILs-score is significantly correlated with histological grade, and is an independent predictor of histological grade (odds ratio [OR], 2.548; 95%CI, 1.648-3.941; P < 0.0001), but TILs-WG is not an independent predictive factor for grade (P > 0.05 in the univariate analysis). Moreover, the risk of G2/High risk group is higher than that of G2/Low risk group, and the survival rate of patients with G2/Low risk is similar to that of Grade1, while the survival rate of patients with G2/High risk is even worse than that of patients with G3. CONCLUSION: Our results suggest that TILs-score can be used to predict the histological grade of breast cancer and potentially to guide the therapeutic management of breast cancer patients.

Quantitative Assessment of Hepatic Steatosis Using Label-Free Multiphoton Imaging and Customized Image Processing Program.

Nonalcoholic fatty liver disease is rapidly becoming one of the most common causes of chronic liver disease worldwide and is the leading cause of liver-related morbidity and mortality. A quantitative assessment of the degree of steatosis would be more advantageous for diagnostic evaluation and exploring the patterns of disease progression. Here, multiphoton microscopy, based on the second harmonic generation and 2-photon excited fluorescence, was used to label-free image the samples of nonalcoholic fatty liver. Imaging results confirm that multiphoton microscopy is capable of directly visualizing important pathologic features such as normal hepatocytes, hepatic steatosis, Mallory bodies, necrosis, inflammation, collagen deposition, microvessel, and so on and is a reliable auxiliary tool for the diagnosis of nonalcoholic fatty liver disease. Furthermore, we developed an image segmentation algorithm to simultaneously assess hepatic steatosis and fibrotic changes, and quantitative results reveal that there is a correlation between the degree of steatosis and collagen content. We also developed a feature extraction program to precisely display the spatial distribution of hepatocyte steatosis in tissues. These studies may be beneficial for a better clinical understanding of the process of steatosis as well as for exploring possible therapeutic targets.

Prognostic value of tumor necrosis based on the evaluation of frequency in invasive breast cancer.

BACKGROUND: Tumor necrosis (TN) was associated with poor prognosis. However, the traditional classification of TN ignored spatial intratumor heterogeneity, which may be associated with important prognosis. The purpose of this study was to propose a new method to reveal the hidden prognostic value of spatial heterogeneity of TN in invasive breast cancer (IBC). METHODS: Multiphoton microscopy (MPM) was used to obtain multiphoton images from 471 patients. According to the relative spatial positions of TN, tumor cells, collagen fibers and myoepithelium, four spatial heterogeneities of TN (TN1-4) were defined. Based on the frequency of individual TN, TN-score was obtained to investigate the prognostic value of TN. RESULTS: Patients with high-risk TN had worse 5-year disease-free survival (DFS) than patients with no necrosis (32.5% vs. 64.7%; P < 0.0001 in training set; 45.8% vs. 70.8%; P = 0.017 in validation set), while patients with low-risk TN had a 5-year DFS comparable to patients with no necrosis (60.0% vs. 64.7%; P = 0.497 in training set; 59.8% vs. 70.8%; P = 0.121 in validation set). Furthermore, high-risk TN "up-staged" the patients with IBC. Patients with high-risk TN and stage I tumors had a 5-year DFS comparable to patients with stage II tumors (55.6% vs. 62.0%; P = 0.565 in training set; 62.5% vs. 66.3%; P = 0.856 in validation set), as well as patients with high-risk TN and stage II tumors had a 5-year DFS comparable to patients with stage III tumors (33.3% vs. 24.6%; P = 0.271 in training set; 44.4% vs. 39.3%; P = 0.519 in validation set). CONCLUSIONS: TN-score was an independent prognostic factor for 5-year DFS. Only high-risk TN was associated with poor prognosis. High-risk TN "up-staged" the patients with IBC. Incorporating TN-score into staging category could improve its performance to stratify patients.

[White blood segmentation based on dual path and atrous spatial pyramid pooling].

The count and recognition of white blood cells in blood smear images play an important role in the diagnosis of blood diseases including leukemia. Traditional manual test results are easily disturbed by many factors. It is necessary to develop an automatic leukocyte analysis system to provide doctors with auxiliary diagnosis, and blood leukocyte segmentation is the basis of automatic analysis. In this paper, we improved the U-Net model and proposed a segmentation algorithm of leukocyte image based on dual path and atrous spatial pyramid pooling. Firstly, the dual path network was introduced into the feature encoder to extract multi-scale leukocyte features, and the atrous spatial pyramid pooling was used to enhance the feature extraction ability of the network. Then the feature decoder composed of convolution and deconvolution was used to restore the segmented target to the original image size to realize the pixel level segmentation of blood leukocytes. Finally, qualitative and quantitative experiments were carried out on three leukocyte data sets to verify the effectiveness of the algorithm. The results showed that compared with other representative algorithms, the proposed blood leukocyte segmentation algorithm had better segmentation results, and the mIoU value could reach more than 0.97. It is hoped that the method could be conducive to the automatic auxiliary diagnosis of blood diseases in the future.

Computer-assisted quantification of tumor-associated collagen signatures to improve the prognosis prediction of breast cancer.

BACKGROUND: Collagen fibers play an important role in tumor initiation, progression, and invasion. Our previous research has already shown that large-scale tumor-associated collagen signatures (TACS) are powerful prognostic biomarkers independent of clinicopathological factors in invasive breast cancer. However, they are observed on a macroscale and are more suitable for identifying high-risk patients. It is necessary to investigate the effect of the corresponding microscopic features of TACS so as to more accurately and comprehensively predict the prognosis of breast cancer patients. METHODS: In this retrospective and multicenter study, we included 942 invasive breast cancer patients in both a training cohort (n = 355) and an internal validation cohort (n = 334) from one clinical center and in an external validation cohort (n = 253) from a different clinical center. TACS corresponding microscopic features (TCMFs) were firstly extracted from multiphoton images for each patient, and then least absolute shrinkage and selection operator (LASSO) regression was applied to select the most robust features to build a TCMF-score. Finally, the Cox proportional hazard regression analysis was used to evaluate the association of TCMF-score with disease-free survival (DFS). RESULTS: TCMF-score is significantly associated with DFS in univariate Cox proportional hazard regression analysis. After adjusting for clinical variables by multivariate Cox regression analysis, the TCMF-score remains an independent prognostic indicator. Remarkably, the TCMF model performs better than the clinical (CLI) model in the three cohorts and is particularly outstanding in the ER-positive and lower-risk subgroups. By contrast, the TACS model is more suitable for the ER-negative and higher-risk subgroups. When the TACS and TCMF are combined, they could complement each other and perform well in all patients. As expected, the full model (CLI+TCMF+TACS) achieves the best performance (AUC 0.905, [0.873-0.938]; 0.896, [0.860-0.931]; 0.882, [0.840-0.925] in the three cohorts). CONCLUSION: These results demonstrate that the TCMF-score is an independent prognostic factor for breast cancer, and the increased prognostic performance (TCMF+TACS-score) may help us develop more appropriate treatment protocols.

Prognostic significance of collagen signatures at breast tumor boundary obtained by combining multiphoton imaging and imaging analysis.

PURPOSE: Collagen features in breast tumor microenvironment is closely associated with the prognosis of patients. We aim to explore the prognostic significance of collagen features at breast tumor border by combining multiphoton imaging and imaging analysis. METHODS: We used multiphoton microscopy (MPM) to label-freely image human breast tumor samples and then constructed an automatic classification model based on deep learning to identify collagen signatures from multiphoton images. We recognized three kinds of collagen signatures at tumor boundary (CSTB I-III) in a small-scale, and furthermore obtained a CSTB score for each patient based on the combined CSTB I-III by using the ridge regression analysis. The prognostic performance of CSTB score is assessed by the area under the receiver operating characteristic curve (AUC), Cox proportional hazard regression analysis, as well as Kaplan-Meier survival analysis. RESULTS: As an independent prognostic factor, statistical results reveal that the prognostic performance of CSTB score is better than that of the clinical model combining three independent prognostic indicators, molecular subtype, tumor size, and lymph nodal metastasis (AUC, Training dataset: 0.773 vs. 0.749; External validation: 0.753 vs. 0.724; HR, Training dataset: 4.18 vs. 3.92; External validation: 4.98 vs. 4.16), and as an auxiliary indicator, it can greatly improve the accuracy of prognostic prediction. And furthermore, a nomogram combining the CSTB score with the clinical model is established for prognosis prediction and clinical decision making. CONCLUSION: This standardized and automated imaging prognosticator may convince pathologists to adopt it as a prognostic factor, thereby customizing more effective treatment plans for patients.

Rapid label-free detection of early-stage lung adenocarcinoma and tumor boundary via multiphoton microscopy.

Lung cancer is the most commonly diagnosed cancer and the leading cause of cancer-related deaths in China. Rapid and precise evaluation of tumor tissue during lung cancer surgery can reduce operative time and improve negative-margin assessment, thus increasing disease-free and overall survival rates. This study aimed to explore the potential of label-free multiphoton microscopy (MPM) for imaging adenocarcinoma tissues, detecting histopathological features, and distinguishing between normal and cancerous lung tissues. We showed that second harmonic generation (SHG) signals exhibit significant specificity for collagen fibers, enabling the quantification of collagen features in lung adenocarcinomas. In addition, we developed a collagen score that could be used to distinguish between normal and tumor areas at the tumor boundary, showing good classification performance. Our findings demonstrate that MPM imaging technology combined with an image-based collagen feature extraction method can rapidly and accurately detect early-stage lung adenocarcinoma tissues.

Label-free detection of invasive micropapillary carcinoma of the breast using multiphoton microscopy.

Invasive micropapillary carcinoma of the breast (IMPC) is a rare form of breast cancer with unique histological features, and is associated with high axillary lymph node metastasis and poor clinical prognosis. Thus, IMPC should be diagnosed in time to improve the treatment and management of patients. In this study, multiphoton microscopy (MPM) is used to label-free visualize the morphological features of IMPC. Our results demonstrate that MPM images are well in agreement with hematoxylin and eosin staining and epithelial membrane antigen staining, indicating MPM is comparable to traditional histological analysis in identifying the tissue structure and cell morphology. Statistical analysis shows significant differences in the circumference and area of the glandular lumen and cancer nest between the different IMPC cell clusters with complete glandular lumen morphology, and also shows difference in collagen length, width, and orientation, indicating the invasive ability of different morphologies of IMPC may be different.

Detection of fibrotic changes in the progression of liver diseases by label-free multiphoton imaging.

Collagen fibers play an important role in the progression of liver diseases. The formation and progression of liver fibrosis is a dynamic pathological process accompanied by morphological changes in collagen fibers. In this study, we used multiphoton microscopy for label-free imaging of liver tissues, allowing direct detection of various components including collagen fibers, tumors, blood vessels, and lymphocytes. Then, we developed a deep learning classification model to automatically identify tumor regions, and the accuracy reaches 0.998. We introduced an automated image processing method to extract eight collagen morphological features from various stages of liver diseases. Statistical analysis showed significant differences between them, indicating the potential use of these quantitative features for monitoring fibrotic changes during the progression of liver diseases. Therefore, multiphoton imaging combined with automatic image processing method would hold a promising future in rapid and label-free diagnosis of liver diseases.

Combining the guidelines and multiphoton imaging methods to improve the prognostic value of tumor-infiltrating lymphocytes in breast cancer.

Multiphoton microscopy (MPM) was introduced to label-freely obtain tumor-infiltrating lymphocytes (TILs) images from a total of 611 patients, and the prognostic value of TILs in breast cancer was assessed by the MPM method (TILs-MPM) and guidelines method proposed by the International Immuno-Oncology Biomarker Working Group (TILs-WG), respectively. Moreover, the clinical (CLI) model, TILs-WG + TILs-MPM model, and full model (CLI + TILs-WG + TILs-MPM) were developed to investigate the prognostic value of TILs. The results show that TILs-WG performs better in estrogen receptor (ER)-negative subgroup, and TILs-MPM is comparable with TILs-WG in the ER-negative subgroup, but has the best performance in the ER-positive subgroup. Furthermore, the TILs-WG + TILs-MPM model can significantly improve the prognostic power compared with the TILs-WG model, and the full model has excellent performance, with high area under the curve (AUC) and hazard ratio (HR) in both ER-positive, ER-negative subgroups, and the complete cohort. Our results suggest that the combination of TILs-WG with TILs-MPM model can greatly improve the prognostic value of TILs.

Intratumor graph neural network recovers hidden prognostic value of multi-biomarker spatial heterogeneity.

Biomarkers are indispensable for precision medicine. However, focused single-biomarker development using human tissue has been complicated by sample spatial heterogeneity. To address this challenge, we tested a representation of primary tumor that synergistically integrated multiple in situ biomarkers of extracellular matrix from multiple sampling regions into an intratumor graph neural network. Surprisingly, the differential prognostic value of this computational model over its conventional non-graph counterpart approximated that of combined routine prognostic biomarkers (tumor size, nodal status, histologic grade, molecular subtype, etc.) for 995 breast cancer patients under a retrospective study. This large prognostic value, originated from implicit but interpretable regional interactions among the graphically integrated in situ biomarkers, would otherwise be lost if they were separately developed into single conventional (spatially homogenized) biomarkers. Our study demonstrates an alternative route to cancer prognosis by taping the regional interactions among existing biomarkers rather than developing novel biomarkers.

Prognostic value of tumour-infiltrating lymphocytes based on the evaluation of frequency in patients with oestrogen receptor-positive breast cancer.

PURPOSE: We investigate the prognostic value of tumour-infiltrating lymphocytes (TILs) based on the evaluation of the present frequency in patients with breast cancer rather than that of the density proposed in previous research. METHODS: Multiphoton microscopy (MPM) was introduced to label-freely obtain TIL images from a total of 564 patients, and then TILs were redefined as TILs-1 to TILs-3 from MPM images according to the relative positions between TILs, tumour cells and collagen fibres. More seminally, a new method, which was based on the present frequency of TILs-1 to TILs-3, was presented for assessing the predictive ability of TILs, and then a tumour-infiltrating lymphocytes score (TILs-score) for each patient was obtained by ridge regression analysis. RESULTS: Data results from Cox proportional hazards regression analysis showed that the TILs-score was an independent prognostic factor for both disease-free survival (DFS) and overall survival (OS) in the complete cohort (n = 564), oestrogen receptor (ER)-positive subgroup (n = 352) and ER-negative subgroup (n = 212), but was more suitable for the ER-positive subgroup. Furthermore, the nomogram model combining the TILs-score with independent clinical factors further improved the predictive ability for the ER-positive subgroup: area under the curve (AUC) at 5-year DFS (OS) and hazard ratio (HR) for DFS (OS) in the training cohort increase from 0.735 (0.785) to 0.814 (0.830) and from 3.156 (5.845) to 4.643 (7.006), respectively, and in the validation cohort from 0.749 (0.748) to 0.804 (0.830) and from 3.104 (3.701) to 3.729 (5.132), respectively. CONCLUSION: The TILs-score is an independent prognostic factor and displays a strong prognostic value for ER-positive breast cancer. To our knowledge, this is the first time to use MPM for studying the prognostic value of TILs in breast cancer.

Nomogram model combining macro and micro tumor-associated collagen signatures obtained from multiphoton images to predict the histologic grade in breast cancer.

The purpose of this study is to develop and validate a new nomogram model combining macro and micro tumor-associated collagen signatures obtained from multiphoton images to differentiate tumor grade in patients with invasive breast cancer. A total of 543 patients were included in this study. We used computer-generated random numbers to assign 328 of these patients to the training cohort and 215 patients to the validation cohort. Macroscopic tumor-associated collagen signatures (TACS1-8) were obtained by multiphoton microscopy at the invasion front and inside of the breast primary tumor. TACS corresponding microscopic features (TCMF) including morphology and texture features were extracted from the segmented regions of interest using Matlab 2016b. Using ridge regression analysis, we obtained a TACS-score for each patient based on the combined TACS1-8, and the least absolute shrinkage and selection operator (LASSO) regression was applied to select the most robust TCMF features to build a TCMF-score. Univariate logistic regression analysis demonstrates that the TACS-score and TCMF-score are significantly associated with histologic grade (odds ratio, 2.994; 95% CI, 2.013-4.452; P < 0.001; 4.245, 2.876-6.264, P < 0.001 in the training cohort). The nomogram (collagen) model combining the TACS-score and TCMF-score could stratify patients into Grade1 and Grade2/3 groups with the AUC of 0.859 and 0.863 in the training and validation cohorts. The predictive performance can be further improved by combining the clinical factors, achieving the AUC of 0.874 in both data cohorts. The nomogram model combining the TACS-score and TCMF-score can be useful in differentiating breast tumor patients with Grade1 and Grade2/3.

Large-scale tumor-associated collagen signatures identify high-risk breast cancer patients.

The notion of personalized medicine demands proper prognostic biomarkers to guide the optimal therapy for an invasive breast cancer patient. However, various risk prediction models based on conventional clinicopathological factors and emergent molecular assays have been frequently limited by either a low strength of prognosis or restricted applicability to specific types of patients. Therefore, there is a critical need to develop a strong and general prognosticator. Methods: We observed five large-scale tumor-associated collagen signatures (TACS4-8) obtained by multiphoton microscopy at the invasion front of the breast primary tumor, which contrasted with the three tumor-associated collagen signatures (TACS1-3) discovered by Keely and coworkers at a smaller scale. Highly concordant TACS1-8 classifications were obtained by three independent observers. Using the ridge regression analysis, we obtained a TACS-score for each patient based on the combined TACS1-8 and established a risk prediction model based on the TACS-score. In a blind fashion, consistent retrospective prognosis was obtained from 995 breast cancer patients in both a training cohort (n= 431) and an internal validation cohort (n = 300) collected from one clinical center, and in an external validation cohort (n = 264) collected from a different clinical center. Results: TACS1-8 model alone competed favorably with all reported models in predicting disease-free survival (AUC: 0.838, [0.800-0.872]; 0.827, [0.779-0.868]; 0.807, [0.754-0.853] in the three cohorts) and stratifying low- and high-risk patients (HR 7.032, [4.869-10.158]; 6.846, [4.370-10.726], 4.423, [2.917-6.708]). The combination of these factors with the TACS-score into a nomogram model further improved the prognosis (AUC: 0.865, [0.829-0.896]; 0.861, [0.816-0.898]; 0.854, [0.805-0.894]; HR 7.882, [5.487-11.323]; 9.176, [5.683-14.816], and 5.548, [3.705-8.307]). The nomogram identified 72 of 357 (~20%) patients with unsuccessful 5-year disease-free survival that might have been undertreated postoperatively. Conclusions: The risk prediction model based on TACS1-8 considerably outperforms the contextual clinical model and may thus convince pathologists to pursue a TACS-based breast cancer prognosis. Our methodology identifies a significant portion of patients susceptible to undertreatment (high-risk patients), in contrast to the multigene assays that often strive to mitigate overtreatment. The compatibility of our methodology with standard histology using traditional (non-tissue-microarray) formalin-fixed paraffin-embedded (FFPE) tissue sections could simplify subsequent clinical translation.

Evaluation of breast carcinoma regression after preoperative chemotherapy by label-free multiphoton imaging and image analysis.

Neoadjuvant chemotherapy is increasingly being used in breast carcinoma as it significantly improves the prognosis and consistently leads to an increased rate of breast preservation. How to accurately assess tumor response after treatment is a crucial factor for developing reasonable therapeutic strategy. In this study, we were in an attempt to monitor tumor response by multimodal multiphoton imaging including two-photon excitation fluorescence and second-harmonic generation imaging. We found that multiphoton imaging can identify different degrees of tumor response such as a slight, significant, or complete response and can detect morphological alteration associated with extracellular matrix during the progression of breast carcinoma following preoperative chemotherapy. Two quantitative optical biomarkers including tumor cellularity and collagen content were extracted based on automatic image analysis to help monitor changes in tumor and its microenvironment. Furthermore, tumor regression grade diagnosis was tried to evaluate by multiphoton microscopy. These results may offer a basic framework for using multiphoton microscopic imaging techniques as a helpful diagnostic tool for assessing breast carcinoma response after presurgical treatment.

Label-free multiphoton imaging to assess neoadjuvant therapy responses in breast carcinoma.

Neoadjuvant chemotherapy has been used increasingly in patients with early-stage or locally advanced breast carcinoma, and has been recommended as a general approach in locally advanced-stage diseases. Assessing therapy response could offer prognostic information to help determine subsequent nursing plan; particularly it is essential to identify responders and non-responders for the sake of helping develop follow-up treatment strategies. However, at present, diagnostic accuracy of preoperative clinical examination are still not satisfactory. Here we presented an alternate approach to monitor tumor and stroma changes associated with neoadjuvant therapy responses in breast carcinoma, with a great potential for becoming a new diagnostic tool-multiphoton microscopy. Imaging results showed that multiphoton imaging techniques have the ability to label-freely visualize tumor response such as tumor necrosis, and stromal response including fibrosis, mucinous response, inflammatory response as well as vascular hyperplasia in situ at cellular and subcellular levels. Moreover, using automated image analysis and a set of scoring methods, we found significant differences in the area of cell nucleus and in the content of collagen fibers between the pre-treatment and post-treatment breast carcinoma tissues. In summary, this study was conducted to pathologically evaluate the response of breast carcinoma to preoperative chemotherapy as well as to assess the efficacy of multiphoton microscopy in detecting these pathological changes, and experimental results demonstrated that this microscope may be a promising tool for label-free, real-time assessment of treatment response without the use of any exogenous contrast agents.

Label-free identification of early gastrointestinal neuroendocrine tumors via biomedical multiphoton microscopy and automatic image analysis.

At present, early diagnosis and treatment is the most effective way to treat early gastrointestinal neuroendocrine tumors. Therefore, we attempted to carry out multiphoton imaging of early neuroendocrine tumors because of its ability to label-free image tissue microstructure at the cellular level. Imaging results show that this imaging technique can quickly identify the histopathological changes in mucosa and submucosa caused by tumor invasion. Furthermore, we performed automatic image analysis on SHG images and extracted two optical diagnostic features-collagen density and average intensity, and also found obvious differences in the density as well as average intensity of collagen fibers in tumor microenvironment using a series of quantitative analysis. These findings may further facilitate the development of multiphoton microscopic imaging technique for clinical use.