SDMT: Spatial Dependence Multi-Task Transformer Network for 3D Knee MRI Segmentation and Landmark Localization.

Knee segmentation and landmark localization from 3D MRI are two significant tasks for diagnosis and treatment of knee diseases. With the development of deep learning, Convolutional Neural Network (CNN) based methods have become the mainstream. However, the existing CNN methods are mostly single-task methods. Due to the complex structure of bone, cartilage and ligament in the knee, it is challenging to complete the segmentation or landmark localization alone. And establishing independent models for all tasks will bring difficulties for surgeon's clinical using. In this paper, a Spatial Dependence Multi-task Transformer (SDMT) network is proposed for 3D knee MRI segmentation and landmark localization. We use a shared encoder for feature extraction, then SDMT utilizes the spatial dependence of segmentation results and landmark position to mutually promote the two tasks. Specifically, SDMT adds spatial encoding to the features, and a task hybrided multi-head attention mechanism is designed, in which the attention heads are divided into the inter-task attention head and the intra-task attention head. The two attention head deal with the spatial dependence between two tasks and correlation within the single task, respectively. Finally, we design a dynamic weight multi-task loss function to balance the training process of two task. The proposed method is validated on our 3D knee MRI multi-task datasets. Dice can reach 83.91% in the segmentation task, and MRE can reach 2.12 mm in the landmark localization task, it is competitive and superior over other state-of-the-art single-task methods.

An Integrated Multitasking Intelligent Bearing Fault Diagnosis Scheme Based on Representation Learning Under Imbalanced Sample Condition.

Accurate bearing fault diagnosis is of great significance of the safety and reliability of rotary mechanical system. In practice, the sample proportion between faulty data and healthy data in rotating mechanical system is imbalanced. Furthermore, there are commonalities between the bearing fault detection, classification, and identification tasks. Based on these observations, this article proposes a novel integrated multitasking intelligent bearing fault diagnosis scheme with the aid of representation learning under imbalanced sample condition, which realizes bearing fault detection, classification, and unknown fault identification. Specifically, in the unsupervised condition, a bearing fault detection approach based on modified denoising autoencoder (DAE) with self-attention mechanism for bottleneck layer (MDAE-SAMB) is proposed in the integrated scheme, which only uses the healthy data for training. The self-attention mechanism is introduced into the neurons in the bottleneck layer, which can assign different weights to the neurons in the bottleneck layer. Moreover, the transfer learning based on representation learning is proposed for few-shot fault classification. Only a few fault samples are used for offline training, and high-accuracy online bearing fault classification is achieved. Finally, according to the known fault data, the unknown bearing faults can be effectively identified. A bearing dataset generated by rotor dynamics experiment rig (RDER) and a public bearing dataset demonstrates the applicability of the proposed integrated fault diagnosis scheme.

MSFR-Net: Multi-modality and single-modality feature recalibration network for brain tumor segmentation.

BACKGROUND: Accurate and automated brain tumor segmentation from multi-modality MR images plays a significant role in tumor treatment. However, the existing approaches mainly focus on the fusion of multi-modality while ignoring the correlation between single-modality and tumor subcomponents. For example, T2-weighted images show good visualization of edema, and T1-contrast images have a good contrast between enhancing tumor core and necrosis. In the actual clinical process, professional physicians also label tumors according to these characteristics. We design a method for brain tumors segmentation that utilizes both multi-modality fusion and single-modality characteristics. METHODS: A multi-modality and single-modality feature recalibration network (MSFR-Net) is proposed for brain tumor segmentation from MR images. Specifically, multi-modality information and single-modality information are assigned to independent pathways. Multi-modality network explicitly learns the relationship between all modalities and all tumor sub-components. Single-modality network learns the relationship between single-modality and its highly correlated tumor subcomponents. Then, a dual recalibration module (DRM) is designed to connect the parallel single-modality network and multi-modality network at multiple stages. The function of the DRM is to unify the two types of features into the same feature space. RESULTS: Experiments on BraTS 2015 dataset and BraTS 2018 dataset show that the proposed method is competitive and superior to other state-of-the-art methods. The proposed method achieved the segmentation results with Dice coefficients of 0.86 and Hausdorff distance of 4.82 on BraTS 2018 dataset, with dice coefficients of 0.80, positive predictive value of 0.76, and sensitivity of 0.78 on BraTS 2015 dataset. CONCLUSIONS: This work combines the manual labeling process of doctors and introduces the correlation between single-modality and the tumor subcomponents into the segmentation network. The method improves the segmentation performance of brain tumors and can be applied in the clinical practice. The code of the proposed method is available at: https://github.com/xiangQAQ/MSFR-Net.

Explainable multi-instance and multi-task learning for COVID-19 diagnosis and lesion segmentation in CT images.

Coronavirus Disease 2019 (COVID-19) still presents a pandemic trend globally. Detecting infected individuals and analyzing their status can provide patients with proper healthcare while protecting the normal population. Chest CT (computed tomography) is an effective tool for screening of COVID-19. It displays detailed pathology-related information. To achieve automated COVID-19 diagnosis and lung CT image segmentation, convolutional neural networks (CNNs) have become mainstream methods. However, most of the previous works consider automated diagnosis and image segmentation as two independent tasks, in which some focus on lung fields segmentation and the others focus on single-lesion segmentation. Moreover, lack of clinical explainability is a common problem for CNN-based methods. In such context, we develop a multi-task learning framework in which the diagnosis of COVID-19 and multi-lesion recognition (segmentation of CT images) are achieved simultaneously. The core of the proposed framework is an explainable multi-instance multi-task network. The network learns task-related features adaptively with learnable weights, and gives explicable diagnosis results by suggesting local CT images with lesions as additional evidence. Then, severity assessment of COVID-19 and lesion quantification are performed to analyze patient status. Extensive experimental results on real-world datasets show that the proposed framework outperforms all the compared approaches for COVID-19 diagnosis and multi-lesion segmentation.

Identifying Factors Associated with the Recurrence of Tic Disorders.

Tic disorders are neurological disorders that are prone to fluctuation and recurrence. It is important to study the factors related to disease recurrence and to subsequently provide suggestions for clinical treatment. A retrospective study was conducted to assess patients with recurrent and non-recurring tic disorders diagnosed in the Pediatric Tic Disorder Clinic of the First Affiliated Hospital of Tianjin University of Traditional Chinese Medicine, China, and to extract various factors-such as fetal status; medication, allergy, and family history; social and psychological factors; blood lead content; electroencephalogram (EEG); disease duration; type of tics; and disease severity-and identify factors associated with recurrence. The recurrence rate of tic disorders was approximately 45.10% in this study. The childbirth conditions, surgery/trauma, respiratory tract infection, allergy, stress, consumption of tiapride, and severity of tic disorders were factors related to and affected disease recurrence.

Lesion-attention pyramid network for diabetic retinopathy grading.

As one of the most common diabetic complications, diabetic retinopathy (DR) can cause retinal damage, vision loss and even blindness. Automated DR grading technology has important clinical significance, which can help ophthalmologists achieve rapid and early diagnosis. With the popularity of deep learning, DR grading based on the convolutional neural networks (CNNs) has become the mainstream method. Unfortunately, although the CNN-based method can achieve satisfactory diagnostic accuracy, it lacks significant clinical information. In this paper, a lesion-attention pyramid network (LAPN) is presented. The pyramid network integrates the subnetworks with different resolutions to get multi-scale features. In order to take the lesion regions in the high-resolution image as the diagnostic evidence, the low-resolution network calculates the lesion activation map (using the weakly-supervised localization method) and guides the high-resolution network to concentrate on the lesion regions. Furthermore, a lesion attention module (LAM) is designed to capture the complementary relationship between the high-resolution features and the low-resolution features, and to fuse the lesion activation map. Experiment results show that the proposed scheme outperforms other existing approaches, and the proposed method can provide lesion activation map with lesion consistency as an additional evidence for clinical diagnosis.