Multi-step validation of a deep learning-based system with visual explanations for optical diagnosis of polyps with advanced features.

Artificial intelligence (AI) has been found to assist in optical differentiation of hyperplastic and adenomatous colorectal polyps. We investigated whether AI can improve the accuracy of endoscopists' optical diagnosis of polyps with advanced features. We introduced our AI system distinguishing polyps with advanced features with more than 0.870 of accuracy in the internal and external validation datasets. All 19 endoscopists with different levels showed significantly lower diagnostic accuracy (0.410-0.580) than the AI. Prospective randomized controlled study involving 120 endoscopists into optical diagnosis of polyps with advanced features with or without AI demonstration identified that AI improved endoscopists' proportion of polyps with advanced features correctly sent for histological examination (0.960 versus 0.840, p < 0.001), and the proportion of polyps without advanced features resected and discarded (0.490 versus 0.380, p = 0.007). We thus developed an AI technique that significantly increases the accuracy of colorectal polyps with advanced features.

A simple normalization technique using window statistics to improve the out-of-distribution generalization on medical images.

Since data scarcity and data heterogeneity are prevailing for medical images, well-trained Convolutional Neural Networks (CNNs) using previous normalization methods may perform poorly when deployed to a new site. However, a reliable model for real-world clinical applications should generalize well both on in-distribution (IND) and out-of-distribution (OOD) data (e.g., the new site data). In this study, we present a novel normalization technique called window normalization (WIN) to improve the model generalization on heterogeneous medical images, which offers a simple yet effective alternative to existing normalization methods. Specifically, WIN perturbs the normalizing statistics with the local statistics computed within a window. This feature-level augmentation technique regularizes the models well and improves their OOD generalization significantly. Leveraging its advantage, we propose a novel self-distillation method called WIN-WIN. WIN-WIN can be easily implemented with two forward passes and a consistency constraint, serving as a simple extension to existing methods. Extensive experimental results on various tasks (6 tasks) and datasets (24 datasets) demonstrate the generality and effectiveness of our methods.

Automatic craniomaxillofacial landmarks detection in CT images of individuals with dentomaxillofacial deformities by a two-stage deep learning model.

BACKGROUND: Accurate cephalometric analysis plays a vital role in the diagnosis and subsequent surgical planning in orthognathic and orthodontics treatment. However, manual digitization of anatomical landmarks in computed tomography (CT) is subject to limitations such as low accuracy, poor repeatability and excessive time consumption. Furthermore, the detection of landmarks has more difficulties on individuals with dentomaxillofacial deformities than normal individuals. Therefore, this study aims to develop a deep learning model to automatically detect landmarks in CT images of patients with dentomaxillofacial deformities. METHODS: Craniomaxillofacial (CMF) CT data of 80 patients with dentomaxillofacial deformities were collected for model development. 77 anatomical landmarks digitized by experienced CMF surgeons in each CT image were set as the ground truth. 3D UX-Net, the cutting-edge medical image segmentation network, was adopted as the backbone of model architecture. Moreover, a new region division pattern for CMF structures was designed as a training strategy to optimize the utilization of computational resources and image resolution. To evaluate the performance of this model, several experiments were conducted to make comparison between the model and manual digitization approach. RESULTS: The training set and the validation set included 58 and 22 samples respectively. The developed model can accurately detect 77 landmarks on bone, soft tissue and teeth with a mean error of 1.81 ± 0.89 mm. Removal of region division before training significantly increased the error of prediction (2.34 ± 1.01 mm). In terms of manual digitization, the inter-observer and intra-observer variations were 1.27 ± 0.70 mm and 1.01 ± 0.74 mm respectively. In all divided regions except Teeth Region (TR), our model demonstrated equivalent performance to experienced CMF surgeons in landmarks detection (p > 0.05). CONCLUSIONS: The developed model demonstrated excellent performance in detecting craniomaxillofacial landmarks when considering manual digitization work of expertise as benchmark. It is also verified that the region division pattern designed in this study remarkably improved the detection accuracy.

Chromosome Detection in Metaphase Cell Images Using Morphological Priors.

Reliable chromosome detection in metaphase cell (MC) images can greatly alleviate the workload of cytogeneticists for karyotype analysis and the diagnosis of chromosomal disorders. However, it is still an extremely challenging task due to the complicated characteristics of chromosomes, e.g., dense distributions, arbitrary orientations, and various morphologies. In this article, we propose a novel rotated-anchor-based detection framework, named DeepCHM, for fast and accurate chromosome detection in MC images. Our framework has three main innovations: 1) A deep saliency map representing chromosomal morphological features is learned end-to-end with semantic features. This not only enhances the feature representations for anchor classification and regression but also guides the anchor setting to significantly reduce redundant anchors. This accelerates the detection and improves the performance; 2) A hardness-aware loss weights the contribution of positive anchors, which effectively reinforces the model to identify hard chromosomes; 3) A model-driven sampling strategy addresses the anchor imbalance issue by adaptively selecting hard negative anchors for model training. In addition, a large-scale benchmark dataset with a total of 624 images and 27,763 chromosome instances was built for chromosome detection and segmentation. Extensive experimental results demonstrate that our method outperforms most state-of-the-art (SOTA) approaches and successfully handles chromosome detection, with an AP score of 93.53%.

Deep learning-based PET/MR radiomics for the classification of annualized relapse rate in multiple sclerosis.

Background Annualized Relapse Rate (ARR) is one of the most important indicators of disease progression in patients with Multiple Sclerosis (MS). However, imaging markers that can effectively predict ARR are currently unavailable. In this study, we developed a deep learning-based method for the automated extraction of radiomics features from Positron Emission Computed Tomography (PET) and Magnetic Resonance (MR) images to predict ARR in patients with MS. Methods Twenty-five patients with a definite diagnosis of Relapsing-Remitting MS (RRMS) were enrolled in this study. We designed a multi-branch fully convolutional neural network to segment lesions from PET/MR images. After that, radiomics features were extracted from the obtained lesion volume of interest. Three feature selection methods were used to retain features highly correlated with ARR. We combined four classifiers with different feature selection methods to form twelve models for ARR classification. Finally, the model with the best performance was chosen. Results Our network achieved precise automatic lesion segmentation with a Dice Similarity Coefficient (DSC) of 0.81 and a precision of 0.86. Radiomics features from lesions filtered by Recursive Feature Elimination (RFE) achieved the best performance in the Support Vector Machines (SVM) classifier. The classification model performance was best when radiomics from both PET and MR were combined to predict ARR, with high accuracy at 0.88 and Area Under the ROC curves (AUC) at 0.96, which outperformed MR or PET-based model and clinical indicators-based model. Conclusion Our automatic segmentation masks can replace manual ones with excellent performance. Furthermore, the deep learning and PET/MR radiomics-based model in our research is an effective tool in assisting ARR classification of MS patients.

Prediction of orthognathic surgery plan from 3D cephalometric analysis via deep learning.

BACKGROUND: Preoperative planning of orthognathic surgery is indispensable for achieving ideal surgical outcome regarding the occlusion and jaws' position. However, orthognathic surgery planning is sophisticated and highly experience-dependent, which requires comprehensive consideration of facial morphology and occlusal function. This study aimed to investigate a robust and automatic method based on deep learning to predict reposition vectors of jawbones in orthognathic surgery plan. METHODS: A regression neural network named VSP transformer was developed based on Transformer architecture. Firstly, 3D cephalometric analysis was employed to quantify skeletal-facial morphology as input features. Next, input features were weighted using pretrained results to minimize bias resulted from multicollinearity. Through encoder-decoder blocks, ten landmark-based reposition vectors of jawbones were predicted. Permutation importance (PI) method was used to calculate contributions of each feature to final prediction to reveal interpretability of the proposed model. RESULTS: VSP transformer model was developed with 383 samples and clinically tested with 49 prospectively collected samples. Our proposed model outperformed other four classic regression models in prediction accuracy. Mean absolute errors (MAE) of prediction were 1.41 mm in validation set and 1.34 mm in clinical test set. The interpretability results of the model were highly consistent with clinical knowledge and experience. CONCLUSIONS: The developed model can predict reposition vectors of orthognathic surgery plan with high accuracy and good clinically practical-effectiveness. Moreover, the model was proved reliable because of its good interpretability.

Further predictive value of lymphovascular invasion explored via supervised deep learning for lymph node metastases in breast cancer.

Lymphovascular invasion, specifically lymph-blood vessel invasion (LBVI), is a risk factor for metastases in breast invasive ductal carcinoma (IDC) and is routinely screened using hematoxylin-eosin histopathological images. However, routine reports only describe whether LBVI is present and does not provide other potential prognostic information of LBVI. This study aims to evaluate the clinical significance of LBVI in 685 IDC cases and explore the added predictive value of LBVI on lymph node metastases (LNM) via supervised deep learning (DL), an expert-experience embedded knowledge transfer learning (EEKT) model in 40 LBVI-positive cases signed by the routine report. Multivariate logistic regression and propensity score matching analysis demonstrated that LBVI (OR 4.203, 95% CI 2.809-6.290, P < 0.001) was a significant risk factor for LNM. Then, the EEKT model trained on 5780 image patches automatically segmented LBVI with a patch-wise Dice similarity coefficient of 0.930 in the test set and output counts, location, and morphometric features of the LBVIs. Some morphometric features were beneficial for further stratification within the 40 LBVI-positive cases. The results showed that LBVI in cases with LNM had a higher short-to-long side ratio of the minimum rectangle (MR) (0.686 vs. 0.480, P = 0.001), LBVI-to-MR area ratio (0.774 vs. 0.702, P = 0.002), and solidity (0.983 vs. 0.934, P = 0.029) compared to LBVI in cases without LNM. The results highlight the potential of DL to assist pathologists in quantifying LBVI and, more importantly, in exploring added prognostic information from LBVI.

Multimodal deep learning model on interim [18F]FDG PET/CT for predicting primary treatment failure in diffuse large B-cell lymphoma.

OBJECTIVES: The prediction of primary treatment failure (PTF) is necessary for patients with diffuse large B-cell lymphoma (DLBCL) since it serves as a prominent means for improving front-line outcomes. Using interim 18F-fluoro-2-deoxyglucose ([18F]FDG) positron emission tomography/computed tomography (PET/CT) imaging data, we aimed to construct multimodal deep learning (MDL) models to predict possible PTF in low-risk DLBCL. METHODS: Initially, 205 DLBCL patients undergoing interim [18F]FDG PET/CT scans and the front-line standard of care were included in the primary dataset for model development. Then, 44 other patients were included in the external dataset for generalization evaluation. Based on the powerful backbone of the Conv-LSTM network, we incorporated five different multimodal fusion strategies (pixel intermixing, separate channel, separate branch, quantitative weighting, and hybrid learning) to make full use of PET/CT features and built five corresponding MDL models. Moreover, we found the best model, that is, the hybrid learning model, and optimized it by integrating the contrastive training objective to further improve its prediction performance. RESULTS: The final model with contrastive objective optimization, named the contrastive hybrid learning model, performed best, with an accuracy of 91.22% and an area under the receiver operating characteristic curve (AUC) of 0.926, in the primary dataset. In the external dataset, its accuracy and AUC remained at 88.64% and 0.925, respectively, indicating its good generalization ability. CONCLUSIONS: The proposed model achieved good performance, validated the predictive value of interim PET/CT, and holds promise for directing individualized clinical treatment. KEY POINTS: • The proposed multimodal models achieved accurate prediction of primary treatment failure in DLBCL patients. • Using an appropriate feature-level fusion strategy can make the same class close to each other regardless of the modal heterogeneity of the data source domain and positively impact the prediction performance. • Deep learning validated the predictive value of interim PET/CT in a way that exceeded human capabilities.

Size-adaptive mediastinal multilesion detection in chest CT images via deep learning and a benchmark dataset.

PURPOSE: Many deep learning methods have been developed for pulmonary lesion detection in chest computed tomography (CT) images. However, these methods generally target one particular lesion type, that is, pulmonary nodules. In this work, we intend to develop and evaluate a novel deep learning method for a more challenging task, detecting various benign and malignant mediastinal lesions with wide variations in sizes, shapes, intensities, and locations in chest CT images. METHODS: Our method for mediastinal lesion detection contains two main stages: (a) size-adaptive lesion candidate detection followed by (b) false-positive (FP) reduction and benign-malignant classification. For candidate detection, an anchor-free and one-stage detector, namely 3D-CenterNet is designed to locate suspicious regions (i.e., candidates with various sizes) within the mediastinum. Then, a 3D-SEResNet-based classifier is used to differentiate FPs, benign lesions, and malignant lesions from the candidates. RESULTS: We evaluate the proposed method by conducting five-fold cross-validation on a relatively large-scale dataset, which consists of data collected on 1136 patients from a grade A tertiary hospital. The method can achieve sensitivity scores of 84.3% ± 1.9%, 90.2% ± 1.4%, 93.2% ± 0.8%, and 93.9% ± 1.1%, respectively, in finding all benign and malignant lesions at 1/8, 1/4, ½, and 1 FPs per scan, and the accuracy of benign-malignant classification can reach up to 78.7% ± 2.5%. CONCLUSIONS: The proposed method can effectively detect mediastinal lesions with various sizes, shapes, and locations in chest CT images. It can be integrated into most existing pulmonary lesion detection systems to promote their clinical applications. The method can also be readily extended to other similar 3D lesion detection tasks.

Automatic Grading Assessments for Knee MRI Cartilage Defects via Self-ensembling Semi-supervised Learning with Dual-Consistency.

Knee cartilage defects caused by osteoarthritis are major musculoskeletal disorders, leading to joint necrosis or even disability if not intervened at early stage. Deep learning has demonstrated its effectiveness in computer-aided diagnosis, but it is time-consuming to prepare a large set of well-annotated data by experienced radiologists for model training. In this paper, we propose a semi-supervised framework to effectively use unlabeled data for better evaluation of knee cartilage defect grading. Our framework is developed based on the widely-used mean-teacher classification model, by designing a novel dual-consistency strategy to boost the consistency between the teacher and student models. The main contributions are three-fold: (1) We define an attention loss function to make the network focus on the cartilage regions, which can both achieve accurate attention masks and boost classification performance simultaneously; (2) Besides enforcing the consistency of classification results, we further design a novel attention consistency mechanism to ensure the focusing of the student and teacher networks on the same defect regions; (3) We introduce an aggregation approach to ensemble the slice-level classification outcomes for deriving the final subject-level diagnosis. Experimental results show that our proposed method can significantly improve both classification and localization performances of knee cartilage defects. Our code is available on https://github.com/King-HAW/DC-MT.

Improving the generalization of glaucoma detection on fundus images via feature alignment between augmented views.

Convolutional neural networks (CNNs) are commonly used in glaucoma detection. Due to the various data distribution shift, however, a well-behaved model may be plummeting in performance when deployed in a new environment. On the other hand, the most straightforward method, data collection, is costly and even unrealistic in practice. To address these challenges, we propose a new method named data augmentation-based (DA) feature alignment (DAFA) to improve the out-of-distribution (OOD) generalization with a single dataset, which is based on the principle of feature alignment to learn the invariant features and eliminate the effect of data distribution shifts. DAFA creates two views of a sample by data augmentation and performs the feature alignment between that augmented views through latent feature recalibration and semantic representation alignment. Latent feature recalibration is normalizing the middle features to the same distribution by instance normalization (IN) layers. Semantic representation alignment is conducted by minimizing the Topk NT-Xent loss and the maximum mean discrepancy (MMD), which maximize the semantic agreement across augmented views from individual and population levels. Furthermore, a benchmark is established with seven glaucoma detection datasets and a new metric named mean of clean area under curve (mcAUC) for a comprehensive evaluation of the model performance. Experimental results of five-fold cross-validation demonstrate that DAFA can consistently and significantly improve the out-of-distribution generalization (up to +16.3% mcAUC) regardless of the training data, network architectures, and augmentation policies and outperform lots of state-of-the-art methods.

Structure-Aware Multimodal Deep Learning for Drug-Protein Interaction Prediction.

Identifying drug-protein interactions (DPIs) is crucial in drug discovery, and a number of machine learning methods have been developed to predict DPIs. Existing methods usually use unrealistic data sets with hidden bias, which will limit the accuracy of virtual screening methods. Meanwhile, most DPI prediction methods pay more attention to molecular representation but lack effective research on protein representation and high-level associations between different instances. To this end, we present the novel structure-aware multimodal deep DPI prediction model, STAMP-DPI, which was trained on a curated industry-scale benchmark data set. We built a high-quality benchmark data set named GalaxyDB for DPI prediction. This industry-scale data set along with an unbiased training procedure resulted in a more robust benchmark study. For informative protein representation, we constructed a structure-aware graph neural network method from the protein sequence by combining predicted contact maps and graph neural networks. Through further integration of structure-based representation and high-level pretrained embeddings for molecules and proteins, our model effectively captures the feature representation of the interactions between them. As a result, STAMP-DPI outperformed state-of-the-art DPI prediction methods by decreasing 7.00% mean square error (MSE) in the Davis data set and improving 8.89% area under the curve (AUC) in the GalaxyDB data set. Moreover, our model is an interpretable model with the transformer-based interaction mechanism, which can accurately reveal the binding sites between molecules and proteins.

Deep Learning for nasopharyngeal Carcinoma Identification Using Both White Light and Narrow-Band Imaging Endoscopy.

OBJECTIVES/HYPOTHESIS: To develop a deep-learning-based automatic diagnosis system for identifying nasopharyngeal carcinoma (NPC) from noncancer (inflammation and hyperplasia), using both white light imaging (WLI) and narrow-band imaging (NBI) nasopharyngoscopy images. STUDY DESIGN: Retrospective study. METHODS: A total of 4,783 nasopharyngoscopy images (2,898 WLI and 1,885 NBI) of 671 patients were collected and a novel deep convolutional neural network (DCNN) framework was developed named Siamese deep convolutional neural network (S-DCNN), which can simultaneously utilize WLI and NBI images to improve the classification performance. To verify the effectiveness of combining the above-mentioned two modal images for prediction, we compared the proposed S-DCNN with two baseline models, namely DCNN-1 (only considering WLI images) and DCNN-2 (only considering NBI images). RESULTS: In the threefold cross-validation, an overall accuracy and area under the curve of the three DCNNs achieved 94.9% (95% confidence interval [CI] 93.3%-96.5%) and 0.986 (95% CI 0.982-0.992), 87.0% (95% CI 84.2%-89.7%) and 0.930 (95% CI 0.906-0.961), and 92.8% (95% CI 90.4%-95.3%) and 0.971 (95% CI 0.953-0.992), respectively. The accuracy of S-DCNN is significantly improved compared with DCNN-1 (P-value <.001) and DCNN-2 (P-value = .008). CONCLUSION: Using the deep-learning technology to automatically diagnose NPC under nasopharyngoscopy can provide valuable reference for NPC screening. Superior performance can be obtained by simultaneously utilizing the multimodal features of NBI image and WLI image of the same patient. LEVEL OF EVIDENCE: 3 Laryngoscope, 132:999-1007, 2022.

3D Graph-Connectivity Constrained Network for Hepatic Vessel Segmentation.

Segmentation of hepatic vessels from 3D CT images is necessary for accurate diagnosis and preoperative planning for liver cancer. However, due to the low contrast and high noises of CT images, automatic hepatic vessel segmentation is a challenging task. Hepatic vessels are connected branches containing thick and thin blood vessels, showing an important structural characteristic or a prior: the connectivity of blood vessels. However, this is rarely applied in existing methods. In this paper, we segment hepatic vessels from 3D CT images by utilizing the connectivity prior. To this end, a graph neural network (GNN) used to describe the connectivity prior of hepatic vessels is integrated into a general convolutional neural network (CNN). Specifically, a graph attention network (GAT) is first used to model the graphical connectivity information of hepatic vessels, which can be trained with the vascular connectivity graph constructed directly from the ground truths. Second, the GAT is integrated with a lightweight 3D U-Net by an efficient mechanism called the plug-in mode, in which the GAT is incorporated into the U-Net as a multi-task branch and is only used to supervise the training procedure of the U-Net with the connectivity prior. The GAT will not be used in the inference stage, and thus will not increase the hardware and time costs of the inference stage compared with the U-Net. Therefore, hepatic vessel segmentation can be well improved in an efficient mode. Extensive experiments on two public datasets show that the proposed method is superior to related works in accuracy and connectivity of hepatic vessel segmentation.

Porous Inorganic Materials for Bioanalysis and Diagnostic Applications.

Porous inorganic materials play an important role in adsorbing targeted analytes and supporting efficient reactions in analytical science. The detection performance relies on the structural properties of porous materials, considering the tunable pore size, shape, connectivity, etc. Herein, we first clarify the enhancement mechanisms of porous materials for bioanalysis, concerning the detection sensitivity and selectivity. The diagnostic applications of porous material-assisted platforms by coupling with various analytical techniques, including electrochemical sensing, optical spectrometry, and mass spectrometry, etc., are then reviewed. We foresee that advanced porous materials will bring far-reaching implications in bioanalysis toward real-case applications, especially as diagnostic assays in clinical settings.

Deep multi-instance transfer learning for pneumothorax classification in chest X-ray images.

PURPOSE: Pneumothorax is a life-threatening emergency that requires immediate treatment. Frontal-view chest X-ray images are typically used for pneumothorax detection in clinical practice. However, manual review of radiographs is time-consuming, labor-intensive, and highly dependent on the experience of radiologists, which may lead to misdiagnosis. Here, we aim to develop a reliable automatic classification method to assist radiologists in rapidly and accurately diagnosing pneumothorax in frontal chest radiographs. METHODS: A novel residual neural network (ResNet)-based two-stage deep-learning strategy is proposed for pneumothorax identification: local feature learning (LFL) followed by global multi-instance learning (GMIL). Most of the nonlesion regions in the images are removed for learning discriminative features. Two datasets are used for large-scale validation: a private dataset (27 955 frontal-view chest X-ray images) and a public dataset (the National Institutes of Health [NIH] ChestX-ray14; 112 120 frontal-view X-ray images). The model performance of the identification was evaluated using the accuracy, precision, recall, specificity, F1-score, receiver operating characteristic (ROC), and area under ROC curve (AUC). Fivefold cross-validation is conducted on the datasets, and then the mean and standard deviation of the above-mentioned metrics are calculated to assess the overall performance of the model. RESULTS: The experimental results demonstrate that the proposed learning strategy can achieve state-of-the-art performance on the NIH dataset with an accuracy, AUC, precision, recall, specificity, and F1-score of 94.4% ± 0.7%, 97.3% ± 0.5%, 94.2% ± 0.3%, 94.6% ± 1.5%, 94.2% ± 0.4%, and 94.4% ± 0.7%, respectively. CONCLUSIONS: The experimental results demonstrate that our proposed CAD system is an efficient assistive tool in the identification of pneumothorax.

Deep reconstruction-recoding network for unsupervised domain adaptation and multi-center generalization in colonoscopy polyp detection.

BACKGROUND AND OBJECTIVE: Currently, the best performing methods in colonoscopy polyp detection are primarily based on deep neural networks (DNNs), which are usually trained on large amounts of labeled data. However, different hospitals use different endoscope models and set different imaging parameters, which causes the collected endoscopic images and videos to vary greatly in style. There may be variations in the color space, brightness, contrast, and resolution, and there are also differences between white light endoscopy (WLE) and narrow band image endoscopy (NBIE). We call these variations the domain shift. The DNN performance may decrease when the training data and the testing data come from different hospitals or different endoscope models. Additionally, it is quite difficult to collect enough new labeled data and retrain a new DNN model before deploying that DNN to a new hospital or endoscope model. METHODS: To solve this problem, we propose a domain adaptation model called Deep Reconstruction-Recoding Network (DRRN), which jointly learns a shared encoding representation for two tasks: i) a supervised object detection network for labeled source data, and ii) an unsupervised reconstruction-recoding network for unlabeled target data. Through the DRRN, the object detection network's encoder not only learns the features from the labeled source domain, but also encodes useful information from the unlabeled target domain. Therefore, the distribution difference of the two domains' feature spaces can be reduced. RESULTS: We evaluate the performance of the DRRN on a series of cross-domain datasets. Compared with training the polyp detection network using only source data, the performance of the DRRN on the target domain is improved. Through feature statistics and visualization, it is demonstrated that the DRRN can learn the common distribution and feature invariance of the two domains. The distribution difference between the feature spaces of the two domains can be reduced. CONCLUSION: The DRRN can improve cross-domain polyp detection. With the DRRN, the generalization performance of the DNN-based polyp detection model can be improved without additional labeled data. This improvement allows the polyp detection model to be easily transferred to datasets from different hospitals or different endoscope models.

C3D-UNET: A Comprehensive 3D Unet for Covid-19 Segmentation with Intact Encoding and Local Attention.

For COVID-19 prevention and treatment, it is essential to screen the pneumonia lesions in the lung region and analyze them in a qualitative and quantitative manner. Three-dimensional (3D) computed tomography (CT) volumes can provide sufficient information; however, extra boundaries of the lesions are also needed. The major challenge of automatic 3D segmentation of COVID-19 from CT volumes lies in the inadequacy of datasets and the wide variations of pneumonia lesions in their appearance, shape, and location. In this paper, we introduce a novel network called Comprehensive 3D UNet (C3D-UNet). Compared to 3D-UNet, an intact encoding (IE) strategy designed as residual dilated convolutional blocks with increased dilation rates is proposed to extract features from wider receptive fields. Moreover, a local attention (LA) mechanism is applied in skip connections for more robust and effective information fusion. We conduct five-fold cross-validation on a private dataset and independent offline evaluation on a public dataset. Experimental results demonstrate that our method outperforms other compared methods.

Enhanced Rotated Mask R-CNN for Chromosome Segmentation.

Karyotyping is an important process for finding chromosome abnormalities that could cause genetic disorders. This process first requires cytogeneticists to arrange each chromosome from the metaphase image to generate the karyogram. In this process, chromosome segmentation plays an important role and it is directly related to whether the karyotyping can be achieved. The key to achieving accurate chromosome segmentation is to effectively segment the multiple touching and overlapping chromosomes at the same time identify the isolated chromosomes. This paper proposes a method named Enhanced Rotated Mask R-CNN for automatic chromosome segmentation and classification. The Enhanced Rotated Mask R-CNN method can not only accurately segment and classify the isolated chromosomes in metaphase images but also effectively alleviate the problem of inaccurate segmentation for touching and overlapping chromosomes. Experiments show that the proposed approach achieves competitive performances with 49.52 AP on multi-class evaluation and 69.96 AP on binary-class evaluation for chromosome segmentation.

IMAL-Net: Interpretable multi-task attention learning network for invasive lung adenocarcinoma screening in CT images.

PURPOSE: Feature maps created from deep convolutional neural networks (DCNNs) have been widely used for visual explanation of DCNN-based classification tasks. However, many clinical applications such as benign-malignant classification of lung nodules normally require quantitative and objective interpretability, rather than just visualization. In this paper, we propose a novel interpretable multi-task attention learning network named IMAL-Net for early invasive adenocarcinoma screening in chest computed tomography images, which takes advantage of segmentation prior to assist interpretable classification. METHODS: Two sub-ResNets are firstly integrated together via a prior-attention mechanism for simultaneous nodule segmentation and invasiveness classification. Then, numerous radiomic features from the segmentation results are concatenated with high-level semantic features from the classification subnetwork by FC layers to achieve superior performance. Meanwhile, an end-to-end feature selection mechanism (named FSM) is designed to quantify crucial radiomic features greatly affecting the prediction of each sample, and thus it can provide clinically applicable interpretability to the prediction result. RESULTS: Nodule samples from a total of 1626 patients were collected from two grade-A hospitals for large-scale verification. Five-fold cross validation demonstrated that the proposed IMAL-Net can achieve an AUC score of 93.8% ± 1.1% and a recall score of 93.8% ± 2.8% for identification of invasive lung adenocarcinoma. CONCLUSIONS: It can be concluded that fusing semantic features and radiomic features can achieve obvious improvements in the invasiveness classification task. Moreover, by learning more fine-grained semantic features and highlighting the most important radiomics features, the proposed attention and FSM mechanisms not only can further improve the performance but also can be used for both visual explanations and objective analysis of the classification results.