Multimodal Co-attention Fusion Network with Online Data Augmentation for Cancer Subtype Classification.

It is an essential task to accurately diagnose cancer subtypes in computational pathology for personalized cancer treatment. Recent studies have indicated that the combination of multimodal data, such as whole slide images (WSIs) and multi-omics data, could achieve more accurate diagnosis. However, robust cancer diagnosis remains challenging due to the heterogeneity among multimodal data, as well as the performance degradation caused by insufficient multimodal patient data. In this work, we propose a novel multimodal co-attention fusion network (MCFN) with online data augmentation (ODA) for cancer subtype classification. Specifically, a multimodal mutual-guided co-attention (MMC) module is proposed to effectively perform dense multimodal interactions. It enables multimodal data to mutually guide and calibrate each other during the integration process to alleviate inter- and intra-modal heterogeneities. Subsequently, a self-normalizing network (SNN)-Mixer is developed to allow information communication among different omics data and alleviate the high-dimensional small-sample size problem in multi-omics data. Most importantly, to compensate for insufficient multimodal samples for model training, we propose an ODA module in MCFN. The ODA module leverages the multimodal knowledge to guide the data augmentations of WSIs and maximize the data diversity during model training. Extensive experiments are conducted on the public TCGA dataset. The experimental results demonstrate that the proposed MCFN outperforms all the compared algorithms, suggesting its effectiveness.

Radiomics model based on MRI to differentiate spinal multiple myeloma from metastases: A two-center study.

Purpose: Spinal multiple myeloma (MM) and metastases are two common cancer types with similar imaging characteristics, for which differential diagnosis is needed to ensure precision therapy. The aim of this study is to establish radiomics models for effective differentiation between them. Methods: Enrolled in this study were 263 patients from two medical institutions, including 127 with spinal MM and 136 with spinal metastases. Of them, 210 patients from institution I were used as the internal training cohort and 53 patients from Institution II were used as the external validation cohort. Contrast-enhanced T1-weighted imaging (CET1) and T2-weighted imaging (T2WI) sequences were collected and reviewed. Based on the 1037 radiomics features extracted from both CET1 and T2WI images, Logistic Regression (LR), AdaBoost (AB), Support Vector Machines (SVM), Random Forest (RF), and multiple kernel learning based SVM (MKL-SVM) were constructed. Hyper-parameters were tuned by five-fold cross-validation. The diagnostic efficiency among different radiomics models was compared by accuracy (ACC), sensitivity (SEN), specificity (SPE), area under the ROC curve (AUC), YI, positive predictive value (PPV), negative predictive value (NPY), and F1-score. Results: Based on single-sequence, the RF model outperformed all other models. All models based on T2WI images performed better than those based on CET1. The efficiency of all models was boosted by incorporating CET1 and T2WI sequences, and the MKL-SVM model achieved the best performance with ACC, AUC, and F1-score of 0.862, 0.870, and 0.874, respectively. Conclusions: The radiomics models constructed based on MRI achieved satisfactory diagnostic performance for differentiation of spinal MM and metastases, demonstrating broad application prospects for individualized diagnosis and treatment.

Pseudo-Data Based Self-Supervised Federated Learning for Classification of Histopathological Images.

Computer-aided diagnosis (CAD) can help pathologists improve diagnostic accuracy together with consistency and repeatability for cancers. However, the CAD models trained with the histopathological images only from a single center (hospital) generally suffer from the generalization problem due to the straining inconsistencies among different centers. In this work, we propose a pseudo-data based self-supervised federated learning (FL) framework, named SSL-FT-BT, to improve both the diagnostic accuracy and generalization of CAD models. Specifically, the pseudo histopathological images are generated from each center, which contain both inherent and specific properties corresponding to the real images in this center, but do not include the privacy information. These pseudo images are then shared in the central server for self-supervised learning (SSL) to pre-train the backbone of global mode. A multi-task SSL is then designed to effectively learn both the center-specific information and common inherent representation according to the data characteristics. Moreover, a novel Barlow Twins based FL (FL-BT) algorithm is proposed to improve the local training for the CAD models in each center by conducting model contrastive learning, which benefits the optimization of the global model in the FL procedure. The experimental results on four public histopathological image datasets indicate the effectiveness of the proposed SSL-FL-BT on both diagnostic accuracy and generalization.

Multi-Scale Efficient Graph-Transformer for Whole Slide Image Classification.

The multi-scale information among the whole slide images (WSIs) is essential for cancer diagnosis. Although the existing multi-scale vision Transformer has shown its effectiveness for learning multi-scale image representation, it still cannot work well on the gigapixel WSIs due to their extremely large image sizes. To this end, we propose a novel Multi-scale Efficient Graph-Transformer (MEGT) framework for WSI classification. The key idea of MEGT is to adopt two independent efficient Graph-based Transformer (EGT) branches to process the low-resolution and high-resolution patch embeddings (i.e., tokens in a Transformer) of WSIs, respectively, and then fuse these tokens via a multi-scale feature fusion module (MFFM). Specifically, we design an EGT to efficiently learn the local-global information of patch tokens, which integrates the graph representation into Transformer to capture spatial-related information of WSIs. Meanwhile, we propose a novel MFFM to alleviate the semantic gap among different resolution patches during feature fusion, which creates a non-patch token for each branch as an agent to exchange information with another branch by cross-attention mechanism. In addition, to expedite network training, a new token pruning module is developed in EGT to reduce the redundant tokens. Extensive experiments on both TCGA-RCC and CAMELYON16 datasets demonstrate the effectiveness of the proposed MEGT.

Deep Learning Model Based on Dual-Modal Ultrasound and Molecular Data for Predicting Response to Neoadjuvant Chemotherapy in Breast Cancer.

RATIONALE AND OBJECTIVES: To carry out radiomics analysis/deep convolutional neural network (CNN) based on B-mode ultrasound (BUS) and shear wave elastography (SWE) to predict response to neoadjuvant chemotherapy (NAC) in breast cancer patients. MATERIALS AND METHODS: In this prospective study, 255 breast cancer patients who received NAC between September 2016 and December 2021 were included. Radiomics models were designed using a support vector machine classifier based on US images obtained before treatment, including BUS and SWE. And CNN models also were developed using ResNet architecture. The final predictive model was developed by combining the dual-modal US and independently associated clinicopathologic characteristics. The predictive performances of the models were assessed with five-fold cross-validation. RESULTS: Pretreatment SWE performed better than BUS in predicting the response to NAC for breast cancer for both the CNN and radiomics models (P < 0.001). The predictive results of the CNN models were significantly better than the radiomics models, with AUCs of 0.72 versus 0.69 for BUS and 0.80 versus 0.77 for SWE, respectively (P = 0.003). The CNN model based on the dual-modal US and molecular data exhibited outstanding performance in predicting NAC response, with an accuracy of 83.60% ± 2.63%, a sensitivity of 87.76% ± 6.44%, and a specificity of 77.45% ± 4.38%. CONCLUSION: The pretreatment CNN model based on the dual-modal US and molecular data achieved excellent performance for predicting the response to chemotherapy in breast cancer. Therefore, this model has the potential to serve as a non-invasive objective biomarker to predict NAC response and aid clinicians with individual treatments.

Effects of Cycloplegic Agents on Ocular Parameters in Children with Myopia and Hyperopia.

Background: To study the effect of cycloplegia on ocular parameters in children with myopia and hyperopia. Methods: Forty-two myopia and forty-four hyperopia eyes in children between 5 and 10 years of age were included. Measurements were taken before and after cycloplegia using 1% atropine sulfate ointment. The ocular parameters included central corneal thickness (CCT), corneal curvature (CC), anterior chamber depth (ACD), pupil diameter (PD), axial length (AL), and central retinal thickness (CRT). Results: There was no significant difference in CCT, CC, and CRT between the two groups without cycloplegia, but the ACD of the myopia (3.64 ± 0.28 mm) group was significantly higher than that of hyperopia (3.40 ± 0.24 mm; t = -4.522; P < 0.0001). The average PD of the myopia (4.85 ± 0.87 mm) group was significantly smaller than that of the hyperopia group (5.47 ± 1.15 mm; t = 2.903; P < 0.0046). The average AL of myopia (24.25 ± 0.77 mm) was significantly higher than that of hyperopia (21.73 ± 1.24 mm; t = 12.084; P < 0.0001). However, it was found that the average PD of myopia (7.68 ± 0.51 mm) was significantly larger than that of hyperopia (7.41 ± 0.57 mm; t = 2.364; P=0.0202) under cycloplegia. As for the changes in refractive factors before and after cycloplegia, deepened ACD and enlarged PD were noted in both the groups after cycloplegia. Conclusions: Cycloplegia not only affects ACD and PD but also leads to the reversal of PD differences between the two groups. Cycloplegia effects enabled us to study changes in all known ocular parameters in a short period.

The CUSUM statistics of change-point models based on dependent sequences.

In this paper, we investigate the mean change-point models based on associated sequences. Under some weak conditions, we obtain a limit distribution of CUSUM statistic which can be used to judge the mean change-mount δ n is satisfied or dissatisfied n 1 / 2 δ n = o ( 1 ) . We also study the consistency of sample covariances and change-point location statistics. Based on Normality and Lognormality data, some simulations such as empirical sizes, empirical powers and convergence are presented to test our results. As an important application, we use CUSUM statistics to do the mean change-point analysis for a financial series.

Deep attention branch networks for skin lesion classification.

BACKGROUND AND OBJECTIVE: The skin lesion usually covers a small region of the dermoscopy image, and the lesions of different categories might own high similarities. Therefore, it is essential to design an elaborate network for accurate skin lesion classification, which can focus on semantically meaningful lesion parts. Although the Class Activation Mapping (CAM) shows good localization capability of highlighting the discriminative parts, it cannot be obtained in the forward propagation process. METHODS: We propose a Deep Attention Branch Network (DABN) model, which introduces the attention branches to expand the conventional Deep Convolutional Neural Networks (DCNN). The attention branch is designed to obtain the CAM in the training stage, which is then utilized as an attention map to make the network focus on discriminative parts of skin lesions. DABN is applicable to multiple DCNN structures and can be trained in an end-to-end manner. Moreover, a novel Entropy-guided Loss Weighting (ELW) strategy is designed to counter class imbalance influence in the skin lesion datasets. RESULTS: The proposed method achieves an Average Precision (AP) of 0.719 on the ISIC-2016 dataset and an average area under the ROC curve (AUC) of 0.922 on the ISIC-2017 dataset. Compared with other state-of-the-art methods, our method obtains better performance without external data and ensemble learning. Moreover, extensive experiments demonstrate that it can be applied to multi-class classification tasks and improves mean sensitivity by more than 2.6% in different DCNN structures. CONCLUSIONS: The proposed method can adaptively focus on the discriminative regions of dermoscopy images and allows for effective training when facing class imbalance, leading to the performance improvement of skin lesion classification, which could also be applied to other clinical applications.

Immobilization of the extracellular recombinant Lucky9 xylanase from Bacillus subtilis enhances activity at high temperature and pH.

In the paper industry, chlorine is often used to treat the pulp for bleaching. After pulping, a large amount of xylan is present in the fiber. Xylanase can be used to degrade xylan in an eco-friendly process called biobleaching, which can help minimize the usage of chlorine in the delignification process. However, a bottleneck in the adoption of biobleaching is the cost of xylanase and the requirement that xylanase be active and stable at extreme conditions. Here, we investigated whether using sodium alginate beads to immobilize an extracellular xylanase from Bacillus subtilis (Lucky9) can reduce the potential cost of enzyme usage. The optimal pH and the activity of the immobilized enzyme were increased at optimal temperature compared with the free enzyme. In addition, immobilized xylanase was shown to be more stable than free xylanase. The results of this study suggest that the immobilized xylanase has potential applications in the biobleaching industry.