Semi-Supervised Learning for FGVC With Out-of-Category Data.

Despite great strides made on fine-grained visual classification (FGVC), current methods are still heavily reliant on fully-supervised paradigms where ample expert labels are called for. Semi-supervised learning (SSL) techniques, acquiring knowledge from unlabeled data, provide a considerable means forward and have shown great promise for coarse-grained problems. However, exiting SSL paradigms mostly assume in-category (i.e., category-aligned) unlabeled data, which hinders their effectiveness when re-proposed on FGVC. In this paper, we put forward a novel design specifically aimed at making out-of-category data work for semi-supervised FGVC. We work off an important assumption that all fine-grained categories naturally follow a hierarchical structure (e.g., the phylogenetic tree of "Aves" that covers all bird species). It follows that, instead of operating on individual samples, we can instead predict sample relations within this tree structure as the optimization goal of SSL. Beyond this, we further introduced two strategies uniquely brought by these tree structures to achieve inter-sample consistency regularization and reliable pseudo-relation. Our experimental results reveal that (i) the proposed method yields good robustness against out-of-category data, and (ii) it can be equipped with prior arts, boosting their performance thus yielding state-of-the-art results.

Difference of mean Hounsfield units (dHU) between follow-up and initial noncontrast CT scan predicts 90-day poor outcome in spontaneous supratentorial acute intracerebral hemorrhage with deep convolutional neural networks.

OBJECTIVES: This study aimed to investigate the usefulness of a new non-contrast CT scan (NCCT) sign called the dHU, which represented the difference in mean Hounsfield unit values between follow-up and the initial NCCT for predicting 90-day poor functional outcomes in acute supratentorial spontaneous intracerebral hemorrhage(sICH) using deep convolutional neural networks. METHODS: A total of 377 consecutive patients with sICH from center 1 and 91 patients from center 2 (external validation set) were included. A receiver operating characteristic (ROC) analysis was performed to determine the critical value of dHU for predicting poor outcome at 90 days. Modified Rankin score (mRS) >3 or >2 was defined as the primary and secondary poor outcome, respectively. Two multivariate models were developed to test whether dHU was an independent predictor of the two unfavorable functional outcomes. RESULTS: The ROC analysis showed that a dHU >2.5 was a critical value to predict the poor outcomes (mRS >3) in sICH. The sensitivity, specificity, and accuracy of dHU >2.5 for poor outcome prediction were 37.5%, 86.0%, and 70.6%, respectively. In multivariate models developed after adjusting for all elements of the ICH score and hematoma expansion, dHU >2.5 was an independent predictor of both primary and secondary poor outcomes (OR = 2.61, 95% CI [1.32,5.13], P = 0.006; OR = 2.63, 95% CI [1.36,5.10], P = 0.004, respectively). After adjustment for all possible significant predictors (p < 0.05) by univariate analysis, dHU >2.5 had a positive association with primary and secondary poor outcomes (OR = 3.25, 95% CI [1.52,6.98], P = 0.002; OR = 3.42, 95% CI [1.64,7.15], P = 0.001). CONCLUSIONS: The dHU of hematoma based on serial CT scans is independently associated with poor outcomes after acute sICH, which may help predict clinical evolution and guide therapy for sICH patients.

Graph Convolution Based Cross-Network Multiscale Feature Fusion for Deep Vessel Segmentation.

Vessel segmentation is widely used to help with vascular disease diagnosis. Vessels reconstructed using existing methods are often not sufficiently accurate to meet clinical use standards. This is because 3D vessel structures are highly complicated and exhibit unique characteristics, including sparsity and anisotropy. In this paper, we propose a novel hybrid deep neural network for vessel segmentation. Our network consists of two cascaded subnetworks performing initial and refined segmentation respectively. The second subnetwork further has two tightly coupled components, a traditional CNN-based U-Net and a graph U-Net. Cross-network multi-scale feature fusion is performed between these two U-shaped networks to effectively support high-quality vessel segmentation. The entire cascaded network can be trained from end to end. The graph in the second subnetwork is constructed according to a vessel probability map as well as appearance and semantic similarities in the original CT volume. To tackle the challenges caused by the sparsity and anisotropy of vessels, a higher percentage of graph nodes are distributed in areas that potentially contain vessels while a higher percentage of edges follow the orientation of potential nearby vessels. Extensive experiments demonstrate our deep network achieves state-of-the-art 3D vessel segmentation performance on multiple public and in-house datasets.

Radiomics for predicting revised hematoma expansion with the inclusion of intraventricular hemorrhage growth in patients with supratentorial spontaneous intraparenchymal hematomas.

BACKGROUND: Previous radiomics analyses of hematoma expansion have been based on the traditional definition, which only focused on changes in intraparenchymal volume. However, the ability of radiomics-related models to predict revised hematoma expansion (RHE) with the inclusion of intraventricular hemorrhage expansion remains unclear. To develop and validate a noncontrast computed tomography (NCCT)-based clinical- semantic-radiomics nomogram to identify supratentorial spontaneous intracerebral hemorrhage (sICH) patients with RHE on admission. METHODS: In this double-center retrospective study, data from 376 patients with sICH (training set: n=299; test set: n=77; external validation cohort: n=91) were reviewed. A radiomics model, a clinical-semantic model, and a combined model were then constructed based on the logistic regression machine learning approach. Radiomics features were extracted and selected by least absolute shrinkage and selection operator (LASSO) with 5-fold cross validation. Furthermore, the classical BRAIN scoring system was also constructed to predict RHE. Discriminative performance of the models was evaluated on the training and test set with area under the curve (AUC) and decision curve analysis (DCA). RESULTS: The addition of radiomics to clinical-semantic factors significantly improved the prediction performance of RHE compared with the clinical-semantic model alone in the training (AUC, 0.94 vs. 0.81, P<0.05) and test (AUC, 0.84 vs. 0.71, P<0.05) sets, with similar results in the validation set (AUC, 0.83 vs. 0.69, P<0.05). Moreover, the discrimination efficacy of the BRAIN score was significantly lower than the other 3 models (AUC of 0.71 in the training set, P<0.05). CONCLUSIONS: The clinical-semantic-radiomics combined model had the greatest potential for discriminating RHE, and significantly outperformed the classical BRAIN scoring system.

Automated detection of 3D midline shift in spontaneous supratentorial intracerebral haemorrhage with non-contrast computed tomography using deep convolutional neural networks.

Deep learning (DL)-based convolutional neural networks facilitate more accurate detection and rapid analysis of MLS. Our objective was to assess the feasibility of applying a DL-based convolutional neural network to non-contrast computed tomography (CT) for automated 2D/3D brain midline shift measurement and outcome prediction after spontaneous intracerebral haemorrhage. In this retrospective study, 140 consecutive patients were referred for CT assessment of sICH from January 2014 to April 2019. The level of consciousness of patients was evaluated using the Glasgow Coma Scale (GCS) score, and the Glasgow Outcome Scale (GOS) score was calculated to classify the outcome. The distance of midline shift (MLS-D) and volume of midline shift (MLS-V) were automatically measured via DL methods. Patients were divided into three groups based on GCS scores: mild degree (GCS score: 13-15), moderate degree (GCS score: 9-12), and severe degree (GCS score: 3-8). Spearman's correlation analysis revealed statistically significant (P<0.01) positive correlation between GCS and MLS-D (r=0.709) and MLS-V (r=0.754). The AUC of MLS-V was slightly larger than that of MLS-D (0.831 vs 0.799, P=0.318) in the midline shifting group. The AUC of MLS-V was significantly larger than that of MLS-D (0.854 vs 0.736, P=0.03) in patients with severe degree GCS scores. The DL-based measurements of both MLS-D and MLS-V enable the assessment of consciousness and the prediction of the outcome of sICH. Compared to MLS-D, MLS-V measurement can better indicate mass effect and predict outcomes, particularly in severe cases.

Advances in Deep Learning-Based Medical Image Analysis.

Importance. With the booming growth of artificial intelligence (AI), especially the recent advancements of deep learning, utilizing advanced deep learning-based methods for medical image analysis has become an active research area both in medical industry and academia. This paper reviewed the recent progress of deep learning research in medical image analysis and clinical applications. It also discussed the existing problems in the field and provided possible solutions and future directions.Highlights. This paper reviewed the advancement of convolutional neural network-based techniques in clinical applications. More specifically, state-of-the-art clinical applications include four major human body systems: the nervous system, the cardiovascular system, the digestive system, and the skeletal system. Overall, according to the best available evidence, deep learning models performed well in medical image analysis, but what cannot be ignored are the algorithms derived from small-scale medical datasets impeding the clinical applicability. Future direction could include federated learning, benchmark dataset collection, and utilizing domain subject knowledge as priors.Conclusion. Recent advanced deep learning technologies have achieved great success in medical image analysis with high accuracy, efficiency, stability, and scalability. Technological advancements that can alleviate the high demands on high-quality large-scale datasets could be one of the future developments in this area.

A clinically applicable deep-learning model for detecting intracranial aneurysm in computed tomography angiography images.

Intracranial aneurysm is a common life-threatening disease. Computed tomography angiography is recommended as the standard diagnosis tool; yet, interpretation can be time-consuming and challenging. We present a specific deep-learning-based model trained on 1,177 digital subtraction angiography verified bone-removal computed tomography angiography cases. The model has good tolerance to image quality and is tested with different manufacturers. Simulated real-world studies are conducted in consecutive internal and external cohorts, in which it achieves an improved patient-level sensitivity and lesion-level sensitivity compared to that of radiologists and expert neurosurgeons. A specific cohort of suspected acute ischemic stroke is employed and it is found that 99.0% predicted-negative cases can be trusted with high confidence, leading to a potential reduction in human workload. A prospective study is warranted to determine whether the algorithm could improve patients' care in comparison to clinicians' assessment.

Unifying Visual Attribute Learning with Object Recognition in a Multiplicative Framework.

Attributes are mid-level semantic properties of objects. Recent research has shown that visual attributes can benefit many typical learning problems in computer vision community. However, attribute learning is still a challenging problem as the attributes may not always be predictable directly from input images and the variation of visual attributes is sometimes large across categories. In this paper, we propose a unified multiplicative framework for attribute learning, which tackles the key problems. Specifically, images and category information are jointly projected into a shared feature space, where the latent factors are disentangled and multiplied to fulfil attribute prediction. The resulting attribute classifier is category-specific instead of being shared by all categories. Moreover, our model can leverage auxiliary data to enhance the predictive ability of attribute classifiers, which can reduce the effort of instance-level attribute annotation to some extent. By integrated into an existing deep learning framework, our model can both accurately predict attributes and learn efficient image representations. Experimental results show that our method achieves superior performance on both instance-level and category-level attribute prediction. For zero-shot learning based on visual attributes and human-object interaction recognition, our method can improve the state-of-the-art performance on several widely used datasets.