An Efficient Semi-Supervised Framework with Multi-Task and Curriculum Learning for Medical Image Segmentation.

A practical problem in supervised deep learning for medical image segmentation is the lack of labeled data which is expensive and time-consuming to acquire. In contrast, there is a considerable amount of unlabeled data available in the clinic. To make better use of the unlabeled data and improve the generalization on limited labeled data, in this paper, a novel semi-supervised segmentation method via multi-task curriculum learning is presented. Here, curriculum learning means that when training the network, simpler knowledge is preferentially learned to assist the learning of more difficult knowledge. Concretely, our framework consists of a main segmentation task and two auxiliary tasks, i.e. the feature regression task and target detection task. The two auxiliary tasks predict some relatively simpler image-level attributes and bounding boxes as the pseudo labels for the main segmentation task, enforcing the pixel-level segmentation result to match the distribution of these pseudo labels. In addition, to solve the problem of class imbalance in the images, a bounding-box-based attention (BBA) module is embedded, enabling the segmentation network to concern more about the target region rather than the background. Furthermore, to alleviate the adverse effects caused by the possible deviation of pseudo labels, error tolerance mechanisms are also adopted in the auxiliary tasks, including inequality constraint and bounding-box amplification. Our method is validated on ACDC2017 and PROMISE12 datasets. Experimental results demonstrate that compared with the full supervision method and state-of-the-art semi-supervised methods, our method yields a much better segmentation performance on a small labeled dataset. Code is available at https://github.com/DeepMedLab/MTCL.

An enhanced approach to the robust discriminant analysis and class sparsity based embedding.

In recent times, feature extraction attracted much attention in machine learning and pattern recognition fields. This paper extends and improves a scheme for linear feature extraction that can be used in supervised multi-class classification problems. Inspired by recent frameworks for robust sparse LDA and Inter-class sparsity, we propose a unifying criterion able to retain the advantages of these two powerful linear discriminant methods. We introduce an iterative alternating minimization scheme in order to estimate the linear transformation and the orthogonal matrix. The linear transformation is efficiently updated via the steepest descent gradient technique. The proposed framework is generic in the sense that it allows the combination and tuning of other linear discriminant embedding methods. We used our proposed method to fine tune the linear solutions delivered by two recent linear methods: RSLDA and RDA_FSIS. Experiments have been conducted on public image datasets of different types including objects, faces, and digits. The proposed framework compared favorably with several competing methods.

A Bilevel Learning Model and Algorithm for Self-Organizing Feed-Forward Neural Networks for Pattern Classification.

Conventional artificial neural network (ANN) learning algorithms for classification tasks, either derivative-based optimization algorithms or derivative-free optimization algorithms work by training ANN first (or training and validating ANN) and then testing ANN, which are a two-stage and one-pass learning mechanism. Thus, this learning mechanism may not guarantee the generalization ability of a trained ANN. In this article, a novel bilevel learning model is constructed for self-organizing feed-forward neural network (FFNN), in which the training and testing processes are integrated into a unified framework. In this bilevel model, the upper level optimization problem is built for testing error on testing data set and network architecture based on network complexity, whereas the lower level optimization problem is constructed for network weights based on training error on training data set. For the bilevel framework, an interactive learning algorithm is proposed to optimize the architecture and weights of an FFNN with consideration of both training error and testing error. In this interactive learning algorithm, a hybrid binary particle swarm optimization (BPSO) taken as an upper level optimizer is used to self-organize network architecture, whereas the Levenberg-Marquardt (LM) algorithm as a lower level optimizer is utilized to optimize the connection weights of an FFNN. The bilevel learning model and algorithm have been tested on 20 benchmark classification problems. Experimental results demonstrate that the bilevel learning algorithm can significantly produce more compact FFNNs with more excellent generalization ability when compared with conventional learning algorithms.

A classification-based approach to semi-supervised clustering with pairwise constraints.

In this paper, we introduce a neural network framework for semi-supervised clustering with pairwise (must-link or cannot-link) constraints. In contrast to existing approaches, we decompose semi-supervised clustering into two simpler classification tasks: the first stage uses a pair of Siamese neural networks to label the unlabeled pairs of points as must-link or cannot-link; the second stage uses the fully pairwise-labeled dataset produced by the first stage in a supervised neural-network-based clustering method. The proposed approach is motivated by the observation that binary classification (such as assigning pairwise relations) is usually easier than multi-class clustering with partial supervision. On the other hand, being classification-based, our method solves only well-defined classification problems, rather than less well specified clustering tasks. Extensive experiments on various datasets demonstrate the high performance of the proposed method.

Using Supervised Learning Methods to Develop a List of Prescription Medications of Greatest Concern during Pregnancy.

INTRODUCTION: Women and healthcare providers lack adequate information on medication safety during pregnancy. While resources describing fetal risk are available, information is provided in multiple locations, often with subjective assessments of available data. We developed a list of medications of greatest concern during pregnancy to help healthcare providers counsel reproductive-aged and pregnant women. METHODS: Prescription drug labels submitted to the U.S. Food and Drug Administration with information in the Teratogen Information System (TERIS) and/or Drugs in Pregnancy and Lactation by Briggs & Freeman were included (N = 1,186 medications; 766 from three data sources, 420 from two). We used two supervised learning methods ('support vector machine' and 'sentiment analysis') to create prediction models based on narrative descriptions of fetal risk. Two models were created per data source. Our final list included medications categorized as 'high' risk in at least four of six models (if three data sources) or three of four models (if two data sources). RESULTS: We classified 80 prescription medications as being of greatest concern during pregnancy; over half were antineoplastic agents (n = 24), angiotensin converting enzyme inhibitors (n = 10), angiotensin II receptor antagonists (n = 8), and anticonvulsants (n = 7). DISCUSSION: This evidence-based list could be a useful tool for healthcare providers counseling reproductive-aged and pregnant women about medication use during pregnancy. However, providers and patients may find it helpful to weigh the risks and benefits of any pharmacologic treatment for both pregnant women and the fetus when managing medical conditions before and during pregnancy.

Graph Convolution Networks with manifold regularization for semi-supervised learning.

In recent times, Graph Convolution Networks (GCN) have been proposed as a powerful tool for graph-based semi-supervised learning. In this paper, we introduce a model that enhances label propagation of Graph Convolution Networks (GCN). More precisely, we propose GCNs with Manifold Regularization (GCNMR). The objective function of the proposed GCNMR is composed by a supervised term and an unsupervised term. The supervised term enforces the fitting term between the predicted labels and the known labels. The unsupervised term imposes the smoothness of the predicted labels of the whole data samples. By learning a Graph Convolution Network with the proposed objective function, we are able to derive a more powerful semi-supervised learning. The proposed model retains the advantages of the classic GCN, yet it can improve it with no increase in time complexity. Experiments on three public image datasets show that the proposed model is superior to the GCN and several competing existing graph-based semi-supervised learning methods.

Vulnerability of classifiers to evolutionary generated adversarial examples.

This paper deals with the vulnerability of machine learning models to adversarial examples and its implication for robustness and generalization properties. We propose an evolutionary algorithm that can generate adversarial examples for any machine learning model in the black-box attack scenario. This way, we can find adversarial examples without access to model's parameters, only by querying the model at hand. We have tested a range of machine learning models including deep and shallow neural networks. Our experiments have shown that the vulnerability to adversarial examples is not only the problem of deep networks, but it spreads through various machine learning architectures. Rather, it depends on the type of computational units. Local units, such as Gaussian kernels, are less vulnerable to adversarial examples.

Opening the black box of artificial intelligence for clinical decision support: A study predicting stroke outcome.

State-of-the-art machine learning (ML) artificial intelligence methods are increasingly leveraged in clinical predictive modeling to provide clinical decision support systems to physicians. Modern ML approaches such as artificial neural networks (ANNs) and tree boosting often perform better than more traditional methods like logistic regression. On the other hand, these modern methods yield a limited understanding of the resulting predictions. However, in the medical domain, understanding of applied models is essential, in particular, when informing clinical decision support. Thus, in recent years, interpretability methods for modern ML methods have emerged to potentially allow explainable predictions paired with high performance. To our knowledge, we present in this work the first explainability comparison of two modern ML methods, tree boosting and multilayer perceptrons (MLPs), to traditional logistic regression methods using a stroke outcome prediction paradigm. Here, we used clinical features to predict a dichotomized 90 days post-stroke modified Rankin Scale (mRS) score. For interpretability, we evaluated clinical features' importance with regard to predictions using deep Taylor decomposition for MLP, Shapley values for tree boosting and model coefficients for logistic regression. With regard to performance as measured by Area under the Curve (AUC) values on the test dataset, all models performed comparably: Logistic regression AUCs were 0.83, 0.83, 0.81 for three different regularization schemes; tree boosting AUC was 0.81; MLP AUC was 0.83. Importantly, the interpretability analysis demonstrated consistent results across models by rating age and stroke severity consecutively amongst the most important predictive features. For less important features, some differences were observed between the methods. Our analysis suggests that modern machine learning methods can provide explainability which is compatible with domain knowledge interpretation and traditional method rankings. Future work should focus on replication of these findings in other datasets and further testing of different explainability methods.

Artificial intelligence in the diagnosis of cardiovascular disease

SUMMARY Artificial intelligence (AI) is a field of computer science that aims to mimic human thought processes. AI techniques have been applied in cardiovascular medicine to explore novel genotypes and phenotypes in existing diseases, improve the quality of patient care, enabling cost-effectiveness, and reducing readmission and mortality rates. The potential of AI in cardiovascular medicine is tremendous; however, ignorance of the challenges may overshadow its potential clinical impact. This paper gives a glimpse of AI's application in cardiovascular clinical care and discusses its potential role in facilitating precision cardiovascular medicine.

RESUMO A inteligência artificial (IA) é um campo da ciência da computação que tem como objetivo imitar os processos de pensamento humano. Técnicas de IA têm sido aplicadas na medicina cardiovascular para explorar novos genótipos e fenótipos em doenças existentes, melhorar a qualidade do atendimento ao paciente, possibilitar custo-efetividade e reduzir taxas de readmissão e mortalidade. Existe um grande potencial da IA na medicina cardiovascular; no entanto, a ignorância dos desafios pode ofuscar seu impacto clínico. Esse artigo fornece a aplicação da IA no atendimento clínico cardiovascular e discute seu papel potencial na facilitação da medicina cardiovascular de precisão.

Looking to the future: Learning from experience, averting catastrophe.

As humans go through life sifting vast quantities of complex information, we extract knowledge from settings that are more ambiguous than our early homes and classrooms. Learning from experience in an individual's unique context generally improves expert performance, despite the risks inherent in brain dynamics that can transform previously reliable expectations. Designers of twenty-first century technologies face the challenges and responsibilities posed by fielded systems that continue to learn on their own. The neural model Self-supervised ART, which can acquire significantly new knowledge in unpredictable contexts, is an example of one such system.

Recommendations and future directions for supervised machine learning in psychiatry.

Machine learning methods hold promise for personalized care in psychiatry, demonstrating the potential to tailor treatment decisions and stratify patients into clinically meaningful taxonomies. Subsequently, publication counts applying machine learning methods have risen, with different data modalities, mathematically distinct models, and samples of varying size being used to train and test models with the promise of clinical translation. Consequently, and in part due to the preliminary nature of such works, many studies have reported largely varying degrees of accuracy, raising concerns over systematic overestimation and methodological inconsistencies. Furthermore, a lack of procedural evaluation guidelines for non-expert medical professionals and funding bodies leaves many in the field with no means to systematically evaluate the claims, maturity, and clinical readiness of a project. Given the potential of machine learning methods to transform patient care, albeit, contingent on the rigor of employed methods and their dissemination, we deem it necessary to provide a review of current methods, recommendations, and future directions for applied machine learning in psychiatry. In this review we will cover issues of best practice for model training and evaluation, sources of systematic error and overestimation, model explainability vs. trust, the clinical implementation of AI systems, and finally, future directions for our field.

Looking to the future: Learning from experience, averting catastrophe.

As humans go through life sifting vast quantities of complex information, we extract knowledge from settings that are more ambiguous than our early homes and classrooms. Learning from experience in an individual's unique context generally improves expert performance, despite the risks inherent in brain dynamics that can transform previously reliable expectations. Designers of twenty-first century technologies face the challenges and responsibilities posed by fielded systems that continue to learn on their own. The neural model Self-supervised ART, which can acquire significantly new knowledge in unpredictable contexts, is an example of one such system.

Learning a discriminant graph-based embedding with feature selection for image categorization.

Graph-based embedding methods are very useful for reducing the dimension of high-dimensional data and for extracting their relevant features. In this paper, we introduce a novel nonlinear method called Flexible Discriminant graph-based Embedding with feature selection (FDEFS). The proposed algorithm aims to classify image sample data in supervised learning and semi-supervised learning settings. Specifically, our method incorporates the Manifold Smoothness, Margin Discriminant Embedding and the Sparse Regression for feature selection. The weights add ℓ2,1-norm regularization for local linear approximation. The sparse regression implicitly performs feature selection on the original features of data matrix and of the linear transform. We also provide an effective solution method to optimize the objective function. We apply the algorithm on six public image datasets including scene, face and object datasets. These experiments demonstrate the effectiveness of the proposed embedding method. They also show that proposed the method compares favorably with many competing embedding methods.

Differentiating between bipolar and unipolar depression in functional and structural MRI studies.

Distinguishing depression in bipolar disorder (BD) from unipolar depression (UD) solely based on clinical clues is difficult, which has led to the exploration of promising neural markers in neuroimaging measures for discriminating between BD depression and UD. In this article, we review structural and functional magnetic resonance imaging (MRI) studies that directly compare UD and BD depression based on neuroimaging modalities including functional MRI studies on regional brain activation or functional connectivity, structural MRI on gray or white matter morphology, and pattern classification analyses using a machine learning approach. Numerous studies have reported distinct functional and structural alterations in emotion- or reward-processing neural circuits between BD depression and UD. Different activation patterns in neural networks including the amygdala, anterior cingulate cortex (ACC), prefrontal cortex (PFC), and striatum during emotion-, reward-, or cognition-related tasks have been reported between BD and UD. A stronger functional connectivity pattern in BD was pronounced in default mode and in frontoparietal networks and brain regions including the PFC, ACC, parietal and temporal regions, and thalamus compared to UD. Gray matter volume differences in the ACC, hippocampus, amygdala, and dorsolateral prefrontal cortex (DLPFC) have been reported between BD and UD, along with a thinner DLPFC in BD compared to UD. BD showed reduced integrity in the anterior part of the corpus callosum and posterior cingulum compared to UD. Several studies performed pattern classification analysis using structural and functional MRI data to distinguish between UD and BD depression using a supervised machine learning approach, which yielded a moderate level of accuracy in classification.

Artificial intelligence in the diagnosis of cardiovascular disease.

Artificial intelligence (AI) is a field of computer science that aims to mimic human thought processes. AI techniques have been applied in cardiovascular medicine to explore novel genotypes and phenotypes in existing diseases, improve the quality of patient care, enabling cost-effectiveness, and reducing readmission and mortality rates. The potential of AI in cardiovascular medicine is tremendous; however, ignorance of the challenges may overshadow its potential clinical impact. This paper gives a glimpse of AI's application in cardiovascular clinical care and discusses its potential role in facilitating precision cardiovascular medicine.

Adaptive non-negative projective semi-supervised learning for inductive classification.

We discuss the inductive classification problem by proposing a joint framework termed Adaptive Non-negative Projective Semi-Supervised Learning (ANP-SSL). Specifically, ANP-SSL integrates the adaptive inductive label propagation, adaptive reconstruction weights learning and the neighborhood preserving projective nonnegative matrix factorization (PNMF) explicitly. To make the label prediction results more accurate, ANP-SSL incorporates the semi-supervised data representation and classification errors into regular PNMF for minimization, which can enable our ANP-SSL to perform the adaptive weights learning and label propagation over the spatially local and part-based data representations, which differs from most existing work that usually assign weights and predict labels based on the original data that often has noise and corruptions. Moreover, existing methods usually pre-assign weights before the process of label estimation, but such operation cannot ensure the learnt weights by independent step to be optimal for the subsequent classification. The combined representation error can also make the learnt reduced part-based representations of neighborhood preserving PNMF, which can potentially enhance the prediction results. By minimizing the classification error jointly over the neighborhood preserving nonnegative representation can make the embedding based classification efficient. Extensive results on several public image databases verified the effectiveness of our ANP-SSL, compared with other state-of-the-art methods.

Spiking neural networks for handwritten digit recognition-Supervised learning and network optimization.

We demonstrate supervised learning in Spiking Neural Networks (SNNs) for the problem of handwritten digit recognition using the spike triggered Normalized Approximate Descent (NormAD) algorithm. Our network that employs neurons operating at sparse biological spike rates below 300Hz achieves a classification accuracy of 98.17% on the MNIST test database with four times fewer parameters compared to the state-of-the-art. We present several insights from extensive numerical experiments regarding optimization of learning parameters and network configuration to improve its accuracy. We also describe a number of strategies to optimize the SNN for implementation in memory and energy constrained hardware, including approximations in computing the neuronal dynamics and reduced precision in storing the synaptic weights. Experiments reveal that even with 3-bit synaptic weights, the classification accuracy of the designed SNN does not degrade beyond 1% as compared to the floating-point baseline. Further, the proposed SNN, which is trained based on the precise spike timing information outperforms an equivalent non-spiking artificial neural network (ANN) trained using back propagation, especially at low bit precision. Thus, our study shows the potential for realizing efficient neuromorphic systems that use spike based information encoding and learning for real-world applications.

Merging weighted SVMs for parallel incremental learning.

Parallel incremental learning is an effective approach for rapidly processing large scale data streams, where parallel and incremental learning are often treated as two separate problems and solved one after another. Incremental learning can be implemented by merging knowledge from incoming data and parallel learning can be performed by merging knowledge from simultaneous learners. We propose to simultaneously solve the two learning problems with a single process of knowledge merging, and we propose parallel incremental wESVM (weighted Extreme Support Vector Machine) to do so. Here, wESVM is reformulated such that knowledge from subsets of training data can be merged via simple matrix addition. As such, the proposed algorithm is able to conduct parallel incremental learning by merging knowledge over data slices arriving at each incremental stage. Both theoretical and experimental studies show the equivalence of the proposed algorithm to batch wESVM in terms of learning effectiveness. In particular, the algorithm demonstrates desired scalability and clear speed advantages to batch retraining.