Recognition of facial emotion based on SOAR model.

Introduction: Expressing emotions play a special role in daily communication, and one of the most essential methods in detecting emotions is to detect facial emotional states. Therefore, one of the crucial aspects of the natural human-machine interaction is the recognition of facial expressions and the creation of feedback, according to the perceived emotion. Methods: To implement each part of this model, two main steps have been introduced. The first step is reading the video and converting it to images and preprocessing on them. The next step is to use the combination of 3D convolutional neural network (3DCNN) and learning automata (LA) to classify and detect the rate of facial emotional recognition. The reason for choosing 3DCNN in our model is that no dimension is removed from the images, and considering the temporal information in dynamic images leads to more efficient and better classification. In addition, the training of the 3DCNN network in calculating the backpropagation error is adjusted by LA so that both the efficiency of the proposed model is increased, and the working memory part of the SOAR model can be implemented. Results and discussion: Due to the importance of the topic, this article presents an efficient method for recognizing emotional states from facial images based on a mixed deep learning and cognitive model called SOAR. Among the objectives of the proposed model, it is possible to mention providing a model for learning the time order of frames in the movie and providing a model for better display of visual features, increasing the recognition rate. The accuracy of recognition rate of facial emotional states in the proposed model is 85.3%. To compare the effectiveness of the proposed model with other models, this model has been compared with competing models. By examining the results, we found that the proposed model has a better performance than other models.

Reviewing 3D convolutional neural network approaches for medical image segmentation.

Background: Convolutional neural networks (CNNs) assume pivotal roles in aiding clinicians in diagnosis and treatment decisions. The rapid evolution of imaging technology has established three-dimensional (3D) CNNs as a formidable framework for delineating organs and anomalies in medical images. The prominence of 3D CNN frameworks is steadily growing within medical image segmentation and classification. Thus, our proposition entails a comprehensive review, encapsulating diverse 3D CNN algorithms for the segmentation of medical image anomalies and organs. Methods: This study systematically presents an exhaustive review of recent 3D CNN methodologies. Rigorous screening of abstracts and titles were carried out to establish their relevance. Research papers disseminated across academic repositories were meticulously chosen, analyzed, and appraised against specific criteria. Insights into the realm of anomalies and organ segmentation were derived, encompassing details such as network architecture and achieved accuracies. Results: This paper offers an all-encompassing analysis, unveiling the prevailing trends in 3D CNN segmentation. In-depth elucidations encompass essential insights, constraints, observations, and avenues for future exploration. A discerning examination indicates the preponderance of the encoder-decoder network in segmentation tasks. The encoder-decoder framework affords a coherent methodology for the segmentation of medical images. Conclusion: The findings of this study are poised to find application in clinical diagnosis and therapeutic interventions. Despite inherent limitations, CNN algorithms showcase commendable accuracy levels, solidifying their potential in medical image segmentation and classification endeavors.

Diagnosis of pulmonary tuberculosis with 3D neural network based on multi-scale attention mechanism.

This paper presents a novel multi-scale attention residual network (MAResNet) for diagnosing patients with pulmonary tuberculosis (PTB) by computed tomography (CT) images. First, a three-dimensional (3D) network structure is applied in MAResNet based on the continuity and correlation of nodal features on different slices of CT images. Secondly, MAResNet incorporates the residual module and Convolutional Block Attention Module (CBAM) to reuse the shallow features of CT images and focus on key features to enhance the feature distinguishability of images. In addition, multi-scale inputs can increase the global receptive field of the network, extract the location information of PTB, and capture the local details of nodules. The expression ability of both high-level and low-level semantic information in the network can also be enhanced. The proposed MAResNet shows excellent results, with overall 94% accuracy in PTB classification. MAResNet based on 3D CT images can assist doctors make more accurate diagnosis of PTB and alleviate the burden of manual screening. In the experiment, a called Grad-CAM was employed to enhance the class activation mapping (CAM) technique for analyzing the model's output, which can identify lesions in important parts of the lungs and make transparent decisions.

ELTS-Net: An enhanced liver tumor segmentation network with augmented receptive field and global contextual information.

The liver is one of the organs with the highest incidence rate in the human body, and late-stage liver cancer is basically incurable. Therefore, early diagnosis and lesion location of liver cancer are of important clinical value. This study proposes an enhanced network architecture ELTS-Net based on the 3D U-Net model, to address the limitations of conventional image segmentation methods and the underutilization of image spatial features by the 2D U-Net network structure. ELTS-Net expands upon the original network by incorporating dilated convolutions to increase the receptive field of the convolutional kernel. Additionally, an attention residual module, comprising an attention mechanism and residual connections, replaces the original convolutional module, serving as the primary components of the encoder and decoder. This design enables the network to capture contextual information globally in both channel and spatial dimensions. Furthermore, deep supervision modules are integrated between different levels of the decoder network, providing additional feedback from deeper intermediate layers. This constrains the network weights to the target regions and optimizing segmentation results. Evaluation on the LiTS2017 dataset shows improvements in evaluation metrics for liver and tumor segmentation tasks compared to the baseline 3D U-Net model, achieving 95.2% liver segmentation accuracy and 71.9% tumor segmentation accuracy, with accuracy improvements of 0.9% and 3.1% respectively. The experimental results validate the superior segmentation performance of ELTS-Net compared to other comparison models, offering valuable guidance for clinical diagnosis and treatment.

Automated intracranial hemorrhage detection in traumatic brain injury using 3D CNN.

Objectives: Intracranial hemorrhage (ICH) is a prevalent and potentially fatal consequence of traumatic brain injury (TBI). Timely identification of ICH is crucial to ensure timely intervention and to optimize better patient outcomes. However, the current methods for diagnosing ICH from head computed tomography (CT) scans require skilled personnel (Radiologists and/or Neurosurgeons) who may be unavailable in all centers, especially in rural areas. The aim of this study is to develop a neurotrauma screening tool for identifying ICH from head CT scans of TBI patients. Materials and Methods: We prospectively collected head CT scans from the Department of Neurosurgery, All India Institute of Medical Sciences, New Delhi. Approximately 738 consecutive head CT scans from patients enrolled in the department were collected for this study spanning a duration of 9 months, that is, January 2020 to September 2020. The metadata collected along with the head CT scans consisted of demographic and clinical details and the radiologist's report which was used as the gold standard. A deep learning-based 3D convolutional neural network (CNN) model was trained on the dataset. The pre-processing, hyperparameters, and augmentation were common for training the 3D CNN model whereas the training modules were set differently. The model was trained along with the save best model option and was monitored by validation metrics. The Institute Ethics Committee permission was taken before starting the study. Results: We developed a 3D CNN model for automatically detecting the ICH from head CT scans. The screening tool was tested in 20 cases and trained on 200 head CT scans, with 99 normal head CT and 101 CT scans with some type of ICH. The final model performed with 90% sensitivity, 70% specificity, and 80% accuracy. Conclusion: Our study reveals that the automated screening tool exhibits a commendable level of accuracy and sensitivity in detecting ICH from the head CT scans. The results indicate that the 3D CNN approach has a potential for further exploring the TBI-related pathologies.

Wireless capsule endoscopy multiclass classification using three-dimensional deep convolutional neural network model.

BACKGROUND: Wireless capsule endoscopy (WCE) is a patient-friendly and non-invasive technology that scans the whole of the gastrointestinal tract, including difficult-to-access regions like the small bowel. Major drawback of this technology is that the visual inspection of a large number of video frames produced during each examination makes the physician diagnosis process tedious and prone to error. Several computer-aided diagnosis (CAD) systems, such as deep network models, have been developed for the automatic recognition of abnormalities in WCE frames. Nevertheless, most of these studies have only focused on spatial information within individual WCE frames, missing the crucial temporal data within consecutive frames. METHODS: In this article, an automatic multiclass classification system based on a three-dimensional deep convolutional neural network (3D-CNN) is proposed, which utilizes the spatiotemporal information to facilitate the WCE diagnosis process. The 3D-CNN model fed with a series of sequential WCE frames in contrast to the two-dimensional (2D) model, which exploits frames as independent ones. Moreover, the proposed 3D deep model is compared with some pre-trained networks. The proposed models are trained and evaluated with 29 subject WCE videos (14,691 frames before augmentation). The performance advantages of 3D-CNN over 2D-CNN and pre-trained networks are verified in terms of sensitivity, specificity, and accuracy. RESULTS: 3D-CNN outperforms the 2D technique in all evaluation metrics (sensitivity: 98.92 vs. 98.05, specificity: 99.50 vs. 86.94, accuracy: 99.20 vs. 92.60). In conclusion, a novel 3D-CNN model for lesion detection in WCE frames is proposed in this study. CONCLUSION: The results indicate the performance of 3D-CNN over 2D-CNN and some well-known pre-trained classifier networks. The proposed 3D-CNN model uses the rich temporal information in adjacent frames as well as spatial data to develop an accurate and efficient model.

Weakly supervised video-based cardiac detection for hypertensive cardiomyopathy.

INTRODUCTION: Parameters, such as left ventricular ejection fraction, peak strain dispersion, global longitudinal strain, etc. are influential and clinically interpretable for detection of cardiac disease, while manual detection requires laborious steps and expertise. In this study, we evaluated a video-based deep learning method that merely depends on echocardiographic videos from four apical chamber views of hypertensive cardiomyopathy detection. METHODS: One hundred eighty-five hypertensive cardiomyopathy (HTCM) patients and 112 healthy normal controls (N) were enrolled in this diagnostic study. We collected 297 de-identified subjects' echo videos for training and testing of an end-to-end video-based pipeline of snippet proposal, snippet feature extraction by a three-dimensional (3-D) convolutional neural network (CNN), a weakly-supervised temporally correlated feature ensemble, and a final classification module. The snippet proposal step requires a preliminarily trained end-systole and end-diastole timing detection model to produce snippets that begin at end-diastole, and involve contraction and dilatation for a complete cardiac cycle. A domain adversarial neural network was introduced to systematically address the appearance variability of echo videos in terms of noise, blur, transducer depth, contrast, etc. to improve the generalization of deep learning algorithms. In contrast to previous image-based cardiac disease detection architectures, video-based approaches integrate spatial and temporal information better with a more powerful 3D convolutional operator. RESULTS: Our proposed model achieved accuracy (ACC) of 92%, area under receiver operating characteristic (ROC) curve (AUC) of 0.90, sensitivity(SEN) of 97%, and specificity (SPE) of 84% with respect to subjects for hypertensive cardiomyopathy detection in the test data set, and outperformed the corresponding 3D CNN (vanilla I3D: ACC (0.90), AUC (0.89), SEN (0.94), and SPE (0.84)). On the whole, the video-based methods remarkably appeared superior to the image-based methods, while few evaluation metrics of image-based methods exhibited to be more compelling (sensitivity of 93% and negative predictive value of 100% for the image-based methods (ES/ED and random)). CONCLUSION: The results supported the possibility of using end-to-end video-based deep learning method for the automated diagnosis of hypertensive cardiomyopathy in the field of echocardiography to augment and assist clinicians. TRIAL REGISTRATION: Current Controlled Trials ChiCTR1900025325, Aug, 24, 2019. Retrospectively registered.

Identification of Turtle-Shell Growth Year Using Hyperspectral Imaging Combined with an Enhanced Spatial-Spectral Attention 3DCNN and a Transformer.

Turtle shell (Chinemys reecesii) is a prized traditional Chinese dietary therapy, and the growth year of turtle shell has a significant impact on its quality attributes. In this study, a hyperspectral imaging (HSI) technique combined with a proposed deep learning (DL) network algorithm was investigated for the objective determination of the growth year of turtle shells. The acquisition of hyperspectral images was carried out in the near-infrared range (948.72-2512.97 nm) from samples spanning five different growth years. To fully exploit the spatial and spectral information while reducing redundancy in hyperspectral data simultaneously, three modules were developed. First, the spectral-spatial attention (SSA) module was developed to better protect the spectral correlation among spectral bands and capture fine-grained spatial information of hyperspectral images. Second, the 3D convolutional neural network (CNN), more suitable for the extracted 3D feature map, was employed to facilitate the joint spatial-spectral feature representation. Thirdly, to overcome the constraints of convolution kernels as well as better capture long-range correlation between spectral bands, the transformer encoder (TE) module was further designed. These modules were harmoniously orchestrated, driven by the need to effectively leverage both spatial and spectral information within hyperspectral data. They collectively enhance the model's capacity to extract joint spatial and spectral features to discern growth years accurately. Experimental studies demonstrated that the proposed model (named SSA-3DTE) achieved superior classification accuracy, with 98.94% on average for five-category classification, outperforming traditional machine learning methods using only spectral information and representative deep learning methods. Also, ablation experiments confirmed the effectiveness of each module to improve performance. The encouraging results of this study revealed the potentiality of HSI combined with the DL algorithm as an efficient and non-destructive method for the quality control of turtle shells.

The OCDA-Net: a 3D convolutional neural network-based system for classification and staging of ovarian cancer patients using [18F]FDG PET/CT examinations.

OBJECTIVE: To create the 3D convolutional neural network (CNN)-based system that can use whole-body [18F]FDG PET for recurrence/post-therapy surveillance in ovarian cancer (OC). METHODS: In this study, 1224 image sets from OC patients who underwent whole-body [18F]FDG PET/CT at Kowsar Hospital between April 2019 and May 2022 were investigated. For recurrence/post-therapy surveillance, diagnostic classification as cancerous, and non-cancerous and staging as stage III, and stage IV were determined by pathological diagnosis and specialists' interpretation. New deep neural network algorithms, the OCDAc-Net, and the OCDAs-Net were developed for diagnostic classification and staging of OC patients using [18F]FDG PET/CT images. Examinations were divided into independent training (75%), validation (10%), and testing (15%) subsets. RESULTS: This study included 37 women (mean age 56.3 years; age range 36-83 years). Data augmentation techniques were applied to the images in two phases. There were 1224 image sets for diagnostic classification and staging. For the test set, 170 image sets were considered for diagnostic classification and staging. The OCDAc-Net areas under the receiver operating characteristic curve (AUCs) and overall accuracy for diagnostic classification were 0.990 and 0.92, respectively. The OCDAs-Net achieved areas under the receiver operating characteristic curve (AUCs) of 0.995 and overall accuracy of 0.94 for staging. CONCLUSIONS: The proposed 3D CNN-based models provide potential tools for recurrence/post-therapy surveillance in OC. The OCDAc-Net and the OCDAs-Net model provide a new prognostic analysis method that can utilize PET images without pathological findings for diagnostic classification and staging.

A deep-learning assisted bioluminescence tomography method to enable radiation targeting in rat glioblastoma.

Objective. A novel solution is required for accurate 3D bioluminescence tomography (BLT) based glioblastoma (GBM) targeting. The provided solution should be computationally efficient to support real-time treatment planning, thus reducing the x-ray imaging dose imposed by high-resolution micro cone-beam CT.Approach. A novel deep-learning approach is developed to enable BLT-based tumor targeting and treatment planning for orthotopic rat GBM models. The proposed framework is trained and validated on a set of realistic Monte Carlo simulations. Finally, the trained deep learning model is tested on a limited set of BLI measurements of real rat GBM models.Significance. Bioluminescence imaging (BLI) is a 2D non-invasive optical imaging modality geared toward preclinical cancer research. It can be used to monitor tumor growth in small animal tumor models effectively and without radiation burden. However, the current state-of-the-art does not allow accurate radiation treatment planning using BLI, hence limiting BLI's value in preclinical radiobiology research.Results. The proposed solution can achieve sub-millimeter targeting accuracy on the simulated dataset, with a median dice similarity coefficient (DSC) of 61%. The provided BLT-based planning volume achieves a median encapsulation of more than 97% of the tumor while keeping the median geometrical brain coverage below 4.2%. For the real BLI measurements, the proposed solution provided median geometrical tumor coverage of 95% and a median DSC of 42%. Dose planning using a dedicated small animal treatment planning system indicated good BLT-based treatment planning accuracy compared to ground-truth CT-based planning, where dose-volume metrics for the tumor fall within the limit of agreement for more than 95% of cases.Conclusion. The combination of flexibility, accuracy, and speed of the deep learning solutions make them a viable option for the BLT reconstruction problem and can provide BLT-based tumor targeting for the rat GBM models.

Automatic identification of schizophrenia based on EEG signals using dynamic functional connectivity analysis and 3D convolutional neural network.

Schizophrenia (ScZ) is a devastating mental disorder of the human brain that causes a serious impact of emotional inclinations, quality of personal and social life and healthcare systems. In recent years, deep learning methods with connectivity analysis only very recently focused into fMRI data. To explore this kind of research into electroencephalogram (EEG) signal, this paper investigates the identification of ScZ EEG signals using dynamic functional connectivity analysis and deep learning methods. A time-frequency domain functional connectivity analysis through cross mutual information algorithm is proposed to extract the features in alpha band (8-12 Hz) of each subject. A 3D convolutional neural network technique was applied to classify the ScZ subjects and health control (HC) subjects. The LMSU public ScZ EEG dataset is employed to evaluate the proposed method, and a 97.74 ± 1.15% accuracy, 96.91 ± 2.76% sensitivity and 98.53 ± 1.97% specificity results were achieved in this study. In addition, we also found not only the default mode network region but also the connectivity between temporal lobe and posterior temporal lobe in both right and left side have significant difference between the ScZ and HC subjects.

High precision tracking analysis of cell position and motion fields using 3D U-net network models.

Cells are the basic units of biological organization, and the quantitative analysis of cellular states is an important topic in medicine and is valuable in revealing the complex mechanisms of microscopic world organisms. In order to better understand cell cycle changes as well as drug actions, we need to track cell migration and division. In this paper, we propose a novel engineering model for tracking cells using cell position and motion fields (CPMF). The training sample does not need to be manually annotated, and we modify and edit it against the ground truth using auxiliary tools. The core idea of the project is to combine detection and correlation, and the cell sequence samples are trained by a U-Net network model composed of 3D CNNs, which can track the migration, division, and entry and exit of cells in the field of view with high accuracy in all directions. The average detection accuracy of the cell coordinates is 98.38% and the average tracking accuracy is 98.70%.

MFCNet: A multi-modal fusion and calibration networks for 3D pancreas tumor segmentation on PET-CT images.

In clinical diagnosis, positron emission tomography and computed tomography (PET-CT) images containing complementary information are fused. Tumor segmentation based on multi-modal PET-CT images is an important part of clinical diagnosis and treatment. However, the existing current PET-CT tumor segmentation methods mainly focus on positron emission tomography (PET) and computed tomography (CT) feature fusion, which weakens the specificity of the modality. In addition, the information interaction between different modal images is usually completed by simple addition or concatenation operations, but this has the disadvantage of introducing irrelevant information during the multi-modal semantic feature fusion, so effective features cannot be highlighted. To overcome this problem, this paper propose a novel Multi-modal Fusion and Calibration Networks (MFCNet) for tumor segmentation based on three-dimensional PET-CT images. First, a Multi-modal Fusion Down-sampling Block (MFDB) with a residual structure is developed. The proposed MFDB can fuse complementary features of multi-modal images while retaining the unique features of different modal images. Second, a Multi-modal Mutual Calibration Block (MMCB) based on the inception structure is designed. The MMCB can guide the network to focus on a tumor region by combining different branch decoding features using the attention mechanism and extracting multi-scale pathological features using a convolution kernel of different sizes. The proposed MFCNet is verified on both the public dataset (Head and Neck cancer) and the in-house dataset (pancreas cancer). The experimental results indicate that on the public and in-house datasets, the average Dice values of the proposed multi-modal segmentation network are 74.14% and 76.20%, while the average Hausdorff distances are 6.41 and 6.84, respectively. In addition, the experimental results show that the proposed MFCNet outperforms the state-of-the-art methods on the two datasets.

Automatic Diagnostic Tool for Detection of Regional Wall Motion Abnormality from Echocardiogram.

The echocardiogram is an ultrasound imaging modality, employed to assess cardiac abnormalities. The Regional Wall Motion Abnormality (RWMA) is the occurrence of abnormal or absent contractility of a region of the heart muscle. Conventional assessment of RWMA is based on visual interpretation of endocardial excursion and myocardial thickening from the echocardiogram videos. Wall motion assessment accuracy depends on the experience of the sonographer. Current automated methods highly depend on the preprocessing steps such as segmentation of ventricle part or manually finding systole and diastole frames from an echocardiogram. Additionally, state-of-the-art methods majorly make use of images rather than videos, which specifically lack the usage of temporal information associated with an echocardiogram. The deep learning models used, employ highly complex networks with billions of trainable parameters. Further, the existing models used on video data add to the computational intensity because of the high frame rates of echocardiogram videos. We developed a novel deep learning architecture EC3D-Net (Echo-Cardio 3D Net), which captures the temporal information for identifying regional wall motion abnormality from echocardiogram. We demonstrate that EC3D-Net can extract temporal information from even raw echocardiogram videos, at low frame rates, employing minimal training parameter-based deep architecture. EC3D-Net achieves both an overall F1-Score and an Area Under Curve (AUC) score of 0.82. Further, we were able to reduce time for training and trainable parameters by 50% through minimizing frames per second. We also show the EC3D-Net is an interpretable model, thereby helping physicians understand our model prediction. RWMA detection from echocardiogram videos is a challenging process and our results demonstrate that we could achieve the state-of-the-art results even while using minimal parameters and time by our EC3D-Net. The proposed network outperforms both complex deep networks as well as fusion methods generally used in video classification.

Learning spatiotemporal features of DSA using 3D CNN and BiConvGRU for ischemic moyamoya disease detection.

BACKGROUND: Moyamoya disease (MMD) is a serious intracranial cerebrovascular disease. Cerebral hemorrhage caused by MMD will bring life risk to patients. Therefore, MMD detection is of great significance in the prevention of cerebral hemorrhage. In order to improve the accuracy of digital subtraction angiography (DSA) in the diagnosis of ischemic MMD, in this paper, a deep network architecture combined with 3D convolutional neural network (3D CNN) and bidirectional convolutional gated recurrent unit (BiConvGRU) is proposed to learn the spatiotemporal features for ischemic MMD detection. METHODS: Firstly, 2D convolutional neural network (2D CNN) is utilized to extract spatial features for each frame of DSA. Secondly, the long-term spatiotemporal features of DSA sequence are extracted by BiConvGRU. Thirdly, the short-term spatiotemporal features of DSA are further extracted by 3D convolutional neural network (3D CNN). In addition, different features are extracted when gray images and optical flow images pass through the network, and multiple features are extracted by features fusion. Finally, the fused features are utilized to classify. RESULTS: The proposed method was quantitatively evaluated on a data sets of 630 cases. The experimental results showed a detection accuracy of 0.9788, sensitivity and specificity were 0.9780 and 0.9796, respectively, and area under curve (AUC) was 0.9856. Compared with other methods, we can get the highest accuracy and AUC. CONCLUSIONS: The experimental results show that the proposed method is stable and reliable for ischemic MMD detection, which provides an option for doctors to accurately diagnose ischemic MMD.

A novel method for efficient estimation of brain effective connectivity in EEG.

BACKGROUND AND OBJECTIVE: Brain connectivity plays a pivotal role in understanding the brain's information processing functions by providing various details including magnitude, direction, and temporal dynamics of inter-neuron connections. While the connectivity may be classified as structural, functional and causal, a complete in-vivo directional analysis is guaranteed by the latter and is referred to as Effective Connectivity (EC). Two most widely used EC techniques are Directed Transfer Function (DTF) and Partial Directed Coherence (PDC) which are based on multivariate autoregressive models. The drawbacks of these techniques include poor frequency resolution and the requirement for experimental approach to determine signal normalization and thresholding techniques in identifying significant connectivities between multivariate sources. METHODS: In this study, the drawbacks of DTF and PDC are addressed by proposing a novel technique, termed as Efficient Effective Connectivity (EEC), for the estimation of EC between multivariate sources using AR spectral estimation and Granger causality principle. In EEC, a linear predictive filter with AR coefficients obtained via multivariate EEG is used for signal prediction. This leads to the estimation of full-length signals which are then transformed into frequency domain by using Burg spectral estimation method. Furthermore, the newly proposed normalization method addressed the effect on each source in EEC using the sum of maximum connectivity values over the entire frequency range. Lastly, the proposed dynamic thresholding works by subtracting the first moment of causal effects of all the sources on one source from individual connections present for that source. RESULTS: The proposed method is evaluated using synthetic and real resting-state EEG of 46 healthy controls. A 3D-Convolutional Neural Network is trained and tested using the PDC and EEC samples. The result indicates that compared to PDC, EEC improves the EEG eye-state classification accuracy, sensitivity and specificity by 5.57%, 3.15% and 8.74%, respectively. CONCLUSION: Correct identification of all connections in synthetic data and improved resting-state classification performance using EEC proved that EEC gives better estimation of directed causality and indicates that it can be used for reliable understanding of brain mechanisms. Conclusively, the proposed technique may open up new research dimensions for clinical diagnosis of mental disorders.

Multi-Modal Feature Fusion-Based Multi-Branch Classification Network for Pulmonary Nodule Malignancy Suspiciousness Diagnosis.

Detecting and identifying malignant nodules on chest computed tomography (CT) plays an important role in the early diagnosis and timely treatment of lung cancer, which can greatly reduce the number of deaths worldwide. In view of the existing methods in pulmonary nodule diagnosis, the importance of clinical radiological structured data (laboratory examination, radiological data) is ignored for the accuracy judgment of patients' condition. Hence, a multi-modal fusion multi-branch classification network is constructed to detect and classify pulmonary nodules in this work: (1) Radiological data of pulmonary nodules are used to construct structured features of length 9. (2) A multi-branch fusion-based effective attention mechanism network is designed for 3D CT Patch unstructured data, which uses 3D ECA-ResNet to dynamically adjust the extracted features. In addition, feature maps with different receptive fields from multi-layer are fully fused to obtain representative multi-scale unstructured features. (3) Multi-modal feature fusion of structured data and unstructured data is performed to distinguish benign and malignant nodules. Numerous experimental results show that this advanced network can effectively classify the benign and malignant pulmonary nodules for clinical diagnosis, which achieves the highest accuracy (94.89%), sensitivity (94.91%), and F1-score (94.65%) and lowest false positive rate (5.55%).

Using a generative adversarial network to generate synthetic MRI images for multi-class automatic segmentation of brain tumors.

Challenging tasks such as lesion segmentation, classification, and analysis for the assessment of disease progression can be automatically achieved using deep learning (DL)-based algorithms. DL techniques such as 3D convolutional neural networks are trained using heterogeneous volumetric imaging data such as MRI, CT, and PET, among others. However, DL-based methods are usually only applicable in the presence of the desired number of inputs. In the absence of one of the required inputs, the method cannot be used. By implementing a generative adversarial network (GAN), we aim to apply multi-label automatic segmentation of brain tumors to synthetic images when not all inputs are present. The implemented GAN is based on the Pix2Pix architecture and has been extended to a 3D framework named Pix2PixNIfTI. For this study, 1,251 patients of the BraTS2021 dataset comprising sequences such as T1w, T2w, T1CE, and FLAIR images equipped with respective multi-label segmentation were used. This dataset was used for training the Pix2PixNIfTI model for generating synthetic MRI images of all the image contrasts. The segmentation model, namely DeepMedic, was trained in a five-fold cross-validation manner for brain tumor segmentation and tested using the original inputs as the gold standard. The inference of trained segmentation models was later applied to synthetic images replacing missing input, in combination with other original images to identify the efficacy of generated images in achieving multi-class segmentation. For the multi-class segmentation using synthetic data or lesser inputs, the dice scores were observed to be significantly reduced but remained similar in range for the whole tumor when compared with evaluated original image segmentation (e.g. mean dice of synthetic T2w prediction NC, 0.74 ± 0.30; ED, 0.81 ± 0.15; CET, 0.84 ± 0.21; WT, 0.90 ± 0.08). A standard paired t-tests with multiple comparison correction were performed to assess the difference between all regions (p < 0.05). The study concludes that the use of Pix2PixNIfTI allows us to segment brain tumors when one input image is missing.

A 3D-CNN with temporal-attention block to predict the recurrence of atrial fibrillation based on body-surface potential mapping signals.

Catheter ablation has become an important treatment for atrial fibrillation (AF), but its recurrence rate is still high. The aim of this study was to predict AF recurrence using a three-dimensional (3D) network model based on body-surface potential mapping signals (BSPMs). BSPMs were recorded with a 128-lead vest in 14 persistent AF patients before undergoing catheter ablation (Maze-IV). The torso geometry was acquired and meshed by point cloud technology, and the BSPM was interpolated into the torso geometry by the inverse distance weighted (IDW) method to generate the isopotential map. Experiments show that the isopotential map of BSPMs can reflect the propagation of the electrical wavefronts. The 3D isopotential sequence map was established by combining the spatial-temporal information of the isopotential map; a 3D convolutional neural network (3D-CNN) model with temporal attention was established to predict AF recurrence. Our study proposes a novel attention block that focuses the characteristics of atrial activations to improve sampling accuracy. In our experiment, accuracy (ACC) in the intra-patient evaluation for predicting the recurrence of AF was 99.38%. In the inter-patient evaluation, ACC of 3D-CNN was 81.48%, and the area under the curve (AUC) was 0.88. It can be concluded that the dynamic rendering of multiple isopotential maps can not only comprehensively display the conduction of cardiac electrical activity on the body surface but also successfully predict the recurrence of AF after CA by using 3D isopotential sequence maps.

Application of high resolution computed tomography image assisted classification model of middle ear diseases based on 3D-convolutional neural network. / 基于3Då·ç§¯ç¥žç»ç½‘ç»œçš„ä¸­è€³ç–¾ç— é«˜åˆ†è¾¨çŽ‡CTå›¾åƒè¾ åŠ©åˆ†ç±»è¯Šæ–­æ¨¡åž‹çš„åº”ç”¨.

OBJECTIVES: Chronic suppurative otitis media (CSOM) and middle ear cholesteatoma (MEC) are the 2 most common chronic middle ear diseases. In the process of diagnosis and treatment, the 2 diseases are prone to misdiagnosis and missed diagnosis due to their similar clinical manifestations. High resolution computed tomography (HRCT) can clearly display the fine anatomical structure of the temporal bone, accurately reflect the middle ear lesions and the extent of the lesions, and has advantages in the differential diagnosis of chronic middle ear diseases. This study aims to develop a deep learning model for automatic information extraction and classification diagnosis of chronic middle ear diseases based on temporal bone HRCT image data to improve the classification and diagnosis efficiency of chronic middle ear diseases in clinical practice and reduce the occurrence of missed diagnosis and misdiagnosis. METHODS: The clinical records and temporal bone HRCT imaging data for patients with chronic middle ear diseases hospitalized in the Department of Otorhinolaryngology, Xiangya Hospital from January 2018 to October 2020 were retrospectively collected. The patient's medical records were independently reviewed by 2 experienced otorhinolaryngologist and the final diagnosis was reached a consensus. A total of 499 patients (998 ears) were enrolled in this study. The 998 ears were divided into 3 groups: an MEC group (108 ears), a CSOM group (622 ears), and a normal group (268 ears). The Gaussian noise with different variances was used to amplify the samples of the dataset to offset the imbalance in the number of samples between groups. The sample size of the amplified experimental dataset was 1 806 ears. In the study, 75% (1 355) samples were randomly selected for training, 10% (180) samples for validation, and the remaining 15% (271) samples for testing and evaluating the model performance. The overall design for the model was a serial structure, and the deep learning model with 3 different functions was set up. The first model was the regional recommendation network algorithm, which searched the middle ear image from the whole HRCT image, and then cut and saved the image. The second model was image contrast convolutional neural network (CNN) based on twin network structure, which searched the images matching the key layers of HRCT images from the cut images, and constructed 3D data blocks. The third model was based on 3D-CNN operation, which was used for the final classification and diagnosis of the 3D data block construction, and gave the final prediction probability. RESULTS: The special level search network based on twin network structure showed an average AUC of 0.939 on 10 special levels. The overall accuracy of the classification network based on 3D-CNN was 96.5%, the overall recall rate was 96.4%, and the average AUC under the 3 classifications was 0.983. The recall rates of CSOM cases and MEC cases were 93.7% and 97.4%, respectively. In the subsequent comparison experiments, the average accuracy of some classical CNN was 79.3%, and the average recall rate was 87.6%. The precision rate and the recall rate of the deep learning network constructed in this study were about 17.2% and 8.8% higher than those of the common CNN. CONCLUSIONS: The deep learning network model proposed in this study can automatically extract 3D data blocks containing middle ear features from the HRCT image data of patients' temporal bone, which can reduce the overall size of the data while preserve the relationship between corresponding images, and further use 3D-CNN for classification and diagnosis of CSOM and MEC. The design of this model is well fitting to the continuous characteristics of HRCT data, and the experimental results show high precision and adaptability, which is better than the current common CNN methods.