Improved performance on high-dimensional survival data by application of Survival-SVM.

MOTIVATION: New application areas of survival analysis as for example based on micro-array expression data call for novel tools able to handle high-dimensional data. While classical (semi-) parametric techniques as based on likelihood or partial likelihood functions are omnipresent in clinical studies, they are often inadequate for modelling in case when there are less observations than features in the data. Support vector machines (svms) and extensions are in general found particularly useful for such cases, both conceptually (non-parametric approach), computationally (boiling down to a convex program which can be solved efficiently), theoretically (for its intrinsic relation with learning theory) as well as empirically. This article discusses such an extension of svms which is tuned towards survival data. A particularly useful feature is that this method can incorporate such additional structure as additive models, positivity constraints of the parameters or regression constraints. RESULTS: Besides discussion of the proposed methods, an empirical case study is conducted on both clinical as well as micro-array gene expression data in the context of cancer studies. Results are expressed based on the logrank statistic, concordance index and the hazard ratio. The reported performances indicate that the present method yields better models for high-dimensional data, while it gives results which are comparable to what classical techniques based on a proportional hazard model give for clinical data.

Additive survival least-squares support vector machines.

This work studies a new survival modeling technique based on least-squares support vector machines. We propose the use of a least-squares support vector machine combining ranking and regression. The advantage of this kernel-based model is threefold: (i) the problem formulation is convex and can be solved conveniently by a linear system; (ii) non-linearity is introduced by using kernels, componentwise kernels in particular are useful to obtain interpretable results; and (iii) introduction of ranking constraints makes it possible to handle censored data. In an experimental setup, the model is used as a preprocessing step for the standard Cox proportional hazard regression by estimating the functional forms of the covariates. The proposed model was compared with different survival models from the literature on the clinical German Breast Cancer Study Group data and on the high-dimensional Norway/Stanford Breast Cancer Data set.

Influence of passive elements on the dynamics of oscillatory ensembles of cardiac cells.

In this paper we focus on the influence of passive elements on the collective dynamics of oscillatory ensembles. Two major effects considered are (i) the influence of passive elements on the synchronization properties of ensembles of coupled nonidentical oscillators and (ii) the influence of passive elements on the wave dynamics of such systems. For the first effect, it is demonstrated that the introduction of passive elements may lead to both an increase or decrease in the global synchronization threshold. For the second effect, it is also demonstrated that the steady state of the passive element is a key parameter which defines how this passive element affects the wave dynamics of the oscillatory ensemble. It was shown that for different values of this parameter, one can observe increase or decrease in wave propagation velocity and increase or decrease in synchronization frequency in oscillatory ensembles with the growth of influence of passive elements. The results are obtained for the models of cardiac cells dynamics as well as for the Bonhoeffer-Van der Pol model and are compared with data of real biological experiments.

Numerical studies of slow rhythms emergence in neural microcircuits: bifurcations and stability.

There is a growing body of evidence that slow brain rhythms are generated by simple inhibitory neural networks. Sequential switching of tonic spiking activity is a widespread phenomenon underlying such rhythms. A realistic generative model explaining such reproducible switching is a dynamical system that employs a closed stable heteroclinic channel (SHC) in its phase space. Despite strong evidence on the existence of SHC, the conditions on its emergence in a spiking network are unclear. In this paper, we analyze a minimal, reciprocally connected circuit of three spiking units and explore all possible dynamical regimes and transitions between them. We show that the SHC arises due to a Neimark-Sacker bifurcation of an unstable cycle.

Variety of synchronous regimes in neuronal ensembles.

We consider a Hodgkin-Huxley-type model of oscillatory activity in neurons of the snail Helix pomatia. This model has a distinctive feature: It demonstrates multistability in oscillatory and silent modes that is typical for the thalamocortical neurons. A single neuron cell can demonstrate a variety of oscillatory activity: Regular and chaotic spiking and bursting behavior. We study collective phenomena in small and large arrays of nonidentical cells coupled by models of electrical and chemical synapses. Two single elements coupled by electrical coupling show different types of synchronous behavior, in particular in-phase and antiphase synchronous regimes. In an ensemble of three inhibitory synaptically coupled elements, the phenomenon of sequential synchronous dynamics is observed. We study the synchronization phenomena in the chain of nonidentical neurons at different oscillatory behavior coupled with electrical and chemical synapses. Various regimes of phase synchronization are observed: (i) Synchronous regular and chaotic spiking; (ii) synchronous regular and chaotic bursting; and (iii) synchronous regular and chaotic bursting with different numbers of spikes inside the bursts. We detect and study the effect of collective synchronous burst generation due to the cluster formation and the oscillatory death.

A convex approach to validation-based learning of the regularization constant.

This letter investigates a tight convex relaxation to the problem of tuning the regularization constant with respect to a validation based criterion. A number of algorithms is covered including ridge regression, regularization networks, smoothing splines, and least squares support vector machines (LS-SVMs) for regression. This convex approach allows the application of reliable and efficient tools, thereby improving computational cost and automatization of the learning method. It is shown that all solutions of the relaxation allow an interpretation in terms of a solution to a weighted LS-SVM.

Preoperative diagnosis of ovarian tumors using Bayesian kernel-based methods.

OBJECTIVES: To develop flexible classifiers that predict malignancy in adnexal masses using a large database from nine centers. METHODS: The database consisted of 1066 patients with at least one persistent adnexal mass for which a large amount of clinical and ultrasound data were recorded. The outcome of interest was the histological classification of the adnexal mass as benign or malignant. The outcome was predicted using Bayesian least squares support vector machines in comparison with relevance vector machines. The models were developed on a training set (n=754) and tested on a test set (n=312). RESULTS: Twenty-five percent of the patients (n=266) had a malignant tumor. Variable selection resulted in a set of 12 variables for the models: age, maximal diameter of the ovary, maximal diameter of the solid component, personal history of ovarian cancer, hormonal therapy, very strong intratumoral blood flow (i.e. color score 4), ascites, presumed ovarian origin of tumor, multilocular-solid tumor, blood flow within papillary projections, irregular internal cyst wall and acoustic shadows. Test set area under the receiver-operating characteristics curve (AUC) for all models exceeded 0.940, with a sensitivity above 90% and a specificity above 80% for all models. The least squares support vector machine model with linear kernel performed very well, with an AUC of 0.946, 91% sensitivity and 84% specificity. The models performed well in the test sets of all the centers. CONCLUSIONS: Bayesian kernel-based methods can accurately separate malignant from benign masses. The robustness of the models will be investigated in future studies.

Efficiently updating and tracking the dominant kernel principal components.

The dominant set of eigenvectors of the symmetrical kernel Gram matrix is used in many important kernel methods (like e.g. kernel Principal Component Analysis, feature approximation, denoising, compression, prediction) in the machine learning area. Yet in the case of dynamic and/or large-scale data, the batch calculation nature and computational demands of the eigenvector decomposition limit these methods in numerous applications. In this paper we present an efficient incremental approach for fast calculation of the dominant kernel eigenbasis, which allows us to track the kernel eigenspace dynamically. Experiments show that our updating scheme delivers a numerically stable and accurate approximation for eigenvalues and eigenvectors at every iteration in comparison to the batch algorithm.

Handling missing values in support vector machine classifiers.

This paper discusses the task of learning a classifier from observed data containing missing values amongst the inputs which are missing completely at random. A non-parametric perspective is adopted by defining a modified risk taking into account the uncertainty of the predicted outputs when missing values are involved. It is shown that this approach generalizes the approach of mean imputation in the linear case and the resulting kernel machine reduces to the standard Support Vector Machine (SVM) when no input values are missing. Furthermore, the method is extended to the multivariate case of fitting additive models using componentwise kernel machines, and an efficient implementation is based on the Least Squares Support Vector Machine (LS-SVM) classifier formulation.

The use of multivariate MR imaging intensities versus metabolic data from MR spectroscopic imaging for brain tumour classification.

This study investigated the value of information from both magnetic resonance imaging and magnetic resonance spectroscopic imaging (MRSI) to automated discrimination of brain tumours. The influence of imaging intensities and metabolic data was tested by comparing the use of MR spectra from MRSI, MR imaging intensities, peak integration values obtained from the MR spectra and a combination of the latter two. Three classification techniques were objectively compared: linear discriminant analysis, least squares support vector machines (LS-SVM) with a linear kernel as linear techniques and LS-SVM with radial basis function kernel as a nonlinear technique. Classifiers were evaluated over 100 stratified random splittings of the dataset into training and test sets. The area under the receiver operating characteristic (ROC) curve (AUC) was used as a global performance measure on test data. In general, all techniques obtained a high performance when using peak integration values with or without MR imaging intensities. For example for low- versus high-grade tumours, low- versus high-grade gliomas and gliomas versus meningiomas, the mean test AUC was higher than 0.91, 0.94, and 0.99, respectively, when both MR imaging intensities and peak integration values were used. The use of metabolic data from MRSI significantly improved automated classification of brain tumour types compared to the use of MR imaging intensities solely.

Classification of brain tumours using short echo time 1H MR spectra.

The purpose was to objectively compare the application of several techniques and the use of several input features for brain tumour classification using Magnetic Resonance Spectroscopy (MRS). Short echo time 1H MRS signals from patients with glioblastomas (n = 87), meningiomas (n = 57), metastases (n = 39), and astrocytomas grade II (n = 22) were provided by six centres in the European Union funded INTERPRET project. Linear discriminant analysis, least squares support vector machines (LS-SVM) with a linear kernel and LS-SVM with radial basis function kernel were applied and evaluated over 100 stratified random splittings of the dataset into training and test sets. The area under the receiver operating characteristic curve (AUC) was used to measure the performance of binary classifiers, while the percentage of correct classifications was used to evaluate the multiclass classifiers. The influence of several factors on the classification performance has been tested: L2- vs. water normalization, magnitude vs. real spectra and baseline correction. The effect of input feature reduction was also investigated by using only the selected frequency regions containing the most discriminatory information, and peak integrated values. Using L2-normalized complete spectra the automated binary classifiers reached a mean test AUC of more than 0.95, except for glioblastomas vs. metastases. Similar results were obtained for all classification techniques and input features except for water normalized spectra, where classification performance was lower. This indicates that data acquisition and processing can be simplified for classification purposes, excluding the need for separate water signal acquisition, baseline correction or phasing.

Brain tumor classification based on long echo proton MRS signals.

There has been a growing research interest in brain tumor classification based on proton magnetic resonance spectroscopy (1H MRS) signals. Four research centers within the EU funded INTERPRET project have acquired a significant number of long echo 1H MRS signals for brain tumor classification. In this paper, we present an objective comparison of several classification techniques applied to the discrimination of four types of brain tumors: meningiomas, glioblastomas, astrocytomas grade II and metastases. Linear and non-linear classifiers are compared: linear discriminant analysis (LDA), support vector machines (SVM) and least squares SVM (LS-SVM) with a linear kernel as linear techniques and LS-SVM with a radial basis function (RBF) kernel as a non-linear technique. Kernel-based methods can perform well in processing high dimensional data. This motivates the inclusion of SVM and LS-SVM in this study. The analysis includes optimal input variable selection, (hyper-) parameter estimation, followed by performance evaluation. The classification performance is evaluated over 200 stratified random samplings of the dataset into training and test sets. Receiver operating characteristic (ROC) curve analysis measures the performance of binary classification, while for multiclass classification, we consider the accuracy as performance measure. Based on the complete magnitude spectra, automated binary classifiers are able to reach an area under the ROC curve (AUC) of more than 0.9 except for the hard case glioblastomas versus metastases. Although, based on the available long echo 1H MRS data, we did not find any statistically significant difference between the performances of LDA and the kernel-based methods, the latter have the strength that no dimensionality reduction is required to obtain such a high performance.

Does the combination of magnetic resonance imaging and spectroscopic imaging improve the classification of brain tumours?

Magnetic Resonance Imaging (MRI) and magnetic resonance spectroscopic imaging (MRSI) play an important role in the noninvasive diagnosis of brain tumours. We investigate the use of both MRI and MRSI, separately and in combination with each other for classification of brain tissue types. Many clinically relevant classification problems are considered; for example healthy versus tumour tissues, low- versus high-grade tumours. Linear as well as nonlinear techniques are compared. The classification performance is evaluated by the area under the receiver operating characteristic (ROC) curve (AUC). In general, all techniques achieve a high performance, except when using MRI alone. For example, for low- versus high-grade tumours, low- versus high-grade gliomas, gliomas versus meningiomas, respectively a test AUC higher than 0.91, 0.93 and 0.98 is reached, when both MRI and MRSI data are used.

Preoperative prediction of malignancy of ovarian tumors using least squares support vector machines.

In this work, we develop and evaluate several least squares support vector machine (LS-SVM) classifiers within the Bayesian evidence framework, in order to preoperatively predict malignancy of ovarian tumors. The analysis includes exploratory data analysis, optimal input variable selection, parameter estimation, and performance evaluation via receiver operating characteristic (ROC) curve analysis. LS-SVM models with linear and radial basis function (RBF) kernels, and logistic regression models have been built on 265 training data, and tested on 160 newly collected patient data. The LS-SVM model with nonlinear RBF kernel achieves the best performance, on the test set with the area under the ROC curve (AUC), sensitivity and specificity equal to 0.92, 81.5% and 84.0%, respectively. The best averaged performance over 30 runs of randomized cross-validation is also obtained by an LS-SVM RBF model, with AUC, sensitivity and specificity equal to 0.94, 90.0% and 80.6%, respectively. These results show that the LS-SVM models have the potential to obtain a reliable preoperative distinction between benign and malignant ovarian tumors, and to assist the clinicians for making a correct diagnosis.

Bayesian framework for least-squares support vector machine classifiers, gaussian processes, and kernel Fisher discriminant analysis.

The Bayesian evidence framework has been successfully applied to the design of multilayer perceptrons (MLPs) in the work of MacKay. Nevertheless, the training of MLPs suffers from drawbacks like the nonconvex optimization problem and the choice of the number of hidden units. In support vector machines (SVMs) for classification, as introduced by Vapnik, a nonlinear decision boundary is obtained by mapping the input vector first in a nonlinear way to a high-dimensional kernel-induced feature space in which a linear large margin classifier is constructed. Practical expressions are formulated in the dual space in terms of the related kernel function, and the solution follows from a (convex) quadratic programming (QP) problem. In least-squares SVMs (LS-SVMs), the SVM problem formulation is modified by introducing a least-squares cost function and equality instead of inequality constraints, and the solution follows from a linear system in the dual space. Implicitly, the least-squares formulation corresponds to a regression formulation and is also related to kernel Fisher discriminant analysis. The least-squares regression formulation has advantages for deriving analytic expressions in a Bayesian evidence framework, in contrast to the classification formulations used, for example, in gaussian processes (GPs). The LS-SVM formulation has clear primal-dual interpretations, and without the bias term, one explicitly constructs a model that yields the same expressions as have been obtained with GPs for regression. In this article, the Bayesian evidence framework is combined with the LS-SVM classifier formulation. Starting from the feature space formulation, analytic expressions are obtained in the dual space on the different levels of Bayesian inference, while posterior class probabilities are obtained by marginalizing over the model parameters. Empirical results obtained on 10 public domain data sets show that the LS-SVM classifier designed within the Bayesian evidence framework consistently yields good generalization performances.