Anisotropic Gaussian kernel adaptive filtering by Lie-group dictionary learning.

The present paper proposes a novel kernel adaptive filtering algorithm, where each Gaussian kernel is parameterized by a center vector and a symmetric positive definite (SPD) precision matrix, which is regarded as a generalization of scalar width parameter. In fact, different from conventional kernel adaptive systems, the proposed filter is structured as a superposition of non-isotropic Gaussian kernels, whose non-isotropy makes the filter more flexible. The adaptation algorithm will search for optimal parameters in a wider parameter space. This generalization brings the need of special treatment of parameters that have a geometric structure. In fact, the main contribution of this paper is to establish update rules for precision matrices on the Lie group of SPD matrices in order to ensure their symmetry and positive-definiteness. The parameters of this filter are adapted on the basis of a least-squares criterion to minimize the filtering error, together with an ℓ1-type regularization criterion to avoid overfitting and to prevent the increase of dimensionality of the dictionary. Experimental results confirm the validity of the proposed method.

Robust Averaging of Covariances for EEG Recordings Classification in Motor Imagery Brain-Computer Interfaces.

The estimation of covariance matrices is of prime importance to analyze the distribution of multivariate signals. In motor imagery-based brain-computer interfaces (MI-BCI), covariance matrices play a central role in the extraction of features from recorded electroencephalograms (EEGs); therefore, correctly estimating covariance is crucial for EEG classification. This letter discusses algorithms to average sample covariance matrices (SCMs) for the selection of the reference matrix in tangent space mapping (TSM)-based MI-BCI. Tangent space mapping is a powerful method of feature extraction and strongly depends on the selection of a reference covariance matrix. In general, the observed signals may include outliers; therefore, taking the geometric mean of SCMs as the reference matrix may not be the best choice. In order to deal with the effects of outliers, robust estimators have to be used. In particular, we discuss and test the use of geometric medians and trimmed averages (defined on the basis of several metrics) as robust estimators. The main idea behind trimmed averages is to eliminate data that exhibit the largest distance from the average covariance calculated on the basis of all available data. The results of the experiments show that while the geometric medians show little differences from conventional methods in terms of classification accuracy in the classification of electroencephalographic recordings, the trimmed averages show significant improvement for all subjects.

Evaluation of tannins interactions in grape (Vitis vinifera L.) skins.

Tannins have a central role in grapevine berries both for their physiological and enological implications. In the skin tissue they can be in vacuolar solution, or associated to the cell walls through weak or strong physicochemical interactions. The present work aims to separate vacuolar, non-covalently and covalently bonded tannins fractions. A specific extraction procedure was developed. A first extraction in ethanol at low temperature allowed the quantification of vacuolar tannins. An urea treatment followed by an ethanol extraction at room temperature was able to separate non-covalently bonded compounds. Finally an acid catalysis was used to break down proanthocyanidin covalent bonds. The method was validated on ripe grape samples of three cultivars, on berries developed in two sun exposure conditions. The Ethephon treatment effect was also evaluated. Beside the method development, a preliminary evaluation of the cultivar, exposition and Ethephon treatment effects are discussed.

Grape colour phenotyping: development of a method based on the reflectance spectrum.

INTRODUCTION: The colour of fruit is an important quality factor for cultivar classification and phenotyping techniques. Besides the subjective visual evaluation, new instruments and techniques can be used. OBJECTIVES: This work aims at developping an objective, fast, easy and non-destructive method as a useful support for evaluating grapes' colour under different cultural and environmental conditions, as well as for breeding process and germplasm evaluation, supporting the plant characterization and the biodiversity preservation. MATERIALS AND METHODS: Colours of 120 grape varieties were studied using reflectance spectra. The classification was realized using cluster and discriminant analysis. Reflectance of the whole berries surface was also compared with absorption properties of single skin extracts. RESULTS: A phenotyping method based on the reflectance spectra was developed, producing reliable colour classifications. A cultivar-independent index for pigment content evaluation has also been obtained. CONCLUSIONS: This work allowed the classification of the berry colour using an objective method.

Extended Hamiltonian learning on Riemannian manifolds: numerical aspects.

This paper is the second part of a study initiated with the paper S. Fiori, "Extended Hamiltonian learning on Riemannian manifolds: Theoretical aspects," IEEE Trans. Neural Netw., vol. 22, no. 5, pp. 687-700, May 2011, which aimed at introducing a general framework to develop a theory of learning on differentiable manifolds by extended Hamiltonian stationary-action principle. This paper discusses the numerical implementation of the extended Hamiltonian learning paradigm by making use of notions from geometric numerical integration to numerically solve differential equations on manifolds. The general-purpose integration schemes and the discussion of several cases of interest show that the implementation of the dynamical learning equations exhibits a rich structure. The behavior of the discussed learning paradigm is illustrated via several numerical examples and discussions of case studies. The numerical examples confirm the theoretical developments presented in this paper as well as in its first part.

Riemannian-gradient-based learning on the complex matrix-hypersphere.

This brief tackles the problem of learning over the complex-valued matrix-hypersphere S(α)(n,p)(C). The developed learning theory is formulated in terms of Riemannian-gradient-based optimization of a regular criterion function and is implemented by a geodesic-stepping method. The stepping method is equipped with a geodesic-search sub-algorithm to compute the optimal learning stepsize at any step. Numerical results show the effectiveness of the developed learning method and of its implementation.

Extended Hamiltonian learning on Riemannian manifolds: theoretical aspects.

This paper introduces a general theory of extended Hamiltonian (second-order) learning on Riemannian manifolds, as an instance of learning by constrained criterion optimization. The dynamical learning equations are derived within the general framework of extended-Hamiltonian stationary-action principle and are expressed in a coordinate-free fashion. A theoretical analysis is carried out in order to compare the features of the dynamical learning theory with the features exhibited by the gradient-based ones. In particular, gradient-based learning is shown to be an instance of dynamical learning, and the classical gradient-based learning modified by a "momentum" term is shown to resemble discrete-time dynamical learning. Moreover, the convergence features of gradient-based and dynamical learning are compared on a theoretical basis. This paper discusses cases of learning by dynamical systems on manifolds of interest in the scientific literature, namely, the Stiefel manifold, the special orthogonal group, the Grassmann manifold, the group of symmetric positive definite matrices, the generalized flag manifold, and the real symplectic group of matrices.

Learning by natural gradient on noncompact matrix-type pseudo-Riemannian manifolds.

This paper deals with learning by natural-gradient optimization on noncompact manifolds. In a Riemannian manifold, the calculation of entities such as the closed form of geodesic curves over noncompact manifolds might be infeasible. For this reason, it is interesting to study the problem of learning by optimization over noncompact manifolds endowed with pseudo-Riemannian metrics, which may give rise to tractable calculations. A general theory for natural-gradient-based learning on noncompact manifolds as well as specific cases of interest of learning are discussed.

Lie-group-type neural system learning by manifold retractions.

The present manuscript treats the problem of adapting a neural signal processing system whose parameters belong to a curved manifold, which is assumed to possess the structure of a Lie group. Neural system parameter adapting is effected by optimizing a system performance criterion. Riemannian-gradient-based optimization is suggested, which cannot be performed by standard additive stepping because of the curved nature of the parameter space. Retraction-based stepping is discussed, instead, along with a companion stepsize-schedule selection procedure. A case-study of learning by optimization of a non-quadratic criterion is discussed in detail.

Asymmetric variate generation via a parameterless dual neural learning algorithm.

In a previous work (S. Fiori, 2006), we proposed a random number generator based on a tunable non-linear neural system, whose learning rule is designed on the basis of a cardinal equation from statistics and whose implementation is based on look-up tables (LUTs). The aim of the present manuscript is to improve the above-mentioned random number generation method by changing the learning principle, while retaining the efficient LUT-based implementation. The new method proposed here proves easier to implement and relaxes some previous limitations.

Learning by criterion optimization on a unitary unimodular matrix group.

The present manuscript aims at illustrating fundamental challenges and solutions arising in the design of learning theories by optimization on manifolds in the context of complex-valued neural systems. The special case of a unitary unimodular group of matrices is dealt with. The unitary unimodular group under analysis is a low dimensional and easy-to-handle matrix group. Notwithstanding, it exhibits a rich geometrical structure and gives rise to interesting speculations about methods to solve optimization problems on manifolds. Also, its low dimension allows us to treat most of the quantities involved in computation in closed form as well as to render them in graphical format. Some numerical experiments are presented and discussed within the paper, which deal with complex-valued independent component analysis.

A study on neural learning on manifold foliations: the case of the Lie group SU(3).

Learning on differential manifolds may involve the optimization of a function of many parameters. In this letter, we deal with Riemannian-gradient-based optimization on a Lie group, namely, the group of unitary unimodular matrices SU(3). In this special case, subalgebras of the associated Lie algebra su(3) may be individuated by computing pair-wise Gell-Mann matrices commutators. Subalgebras generate subgroups of a Lie group, as well as manifold foliation. We show that the Riemannian gradient may be projected over tangent structures to foliation, giving rise to foliation gradients. Exponentiations of foliation gradients may be computed in closed forms, which closely resemble Rodriguez forms for the special orthogonal group SO(3). We thus compare optimization by Riemannian gradient and foliation gradients.

Neural systems with numerically matched input-output statistic: isotonic bivariate statistical modeling.

Bivariate statistical modeling from incomplete data is a useful statistical tool that allows to discover the model underlying two data sets when the data in the two sets do not correspond in size nor in ordering. Such situation may occur when the sizes of the two data sets do not match (i.e., there are "holes" in the data) or when the data sets have been acquired independently. Also, statistical modeling is useful when the amount of available data is enough to show relevant statistical features of the phenomenon underlying the data. We propose to tackle the problem of statistical modeling via a neural (nonlinear) system that is able to match its input-output statistic to the statistic of the available data sets. A key point of the new implementation proposed here is that it is based on look-up-table (LUT) neural systems, which guarantee a computationally advantageous way of implementing neural systems. A number of numerical experiments, performed on both synthetic and real-world data sets, illustrate the features of the proposed modeling procedure.

Formulation and integration of learning differential equations on the stiefel manifold.

This letter aims at illustrating the relevance of numerical integration of learning differential equations on differential manifolds. In particular, the task of learning with orthonormality constraints is dealt with, which is naturally formulated as an optimization task with the compact Stiefel manifold as neural parameter space. Intrinsic properties of the derived learning algorithms, such as stability and constraints preservation, are illustrated through experiments on minor and independent component analysis (ICA).

Nonlinear complex-valued extensions of Hebbian learning: an essay.

The Hebbian paradigm is perhaps the best-known unsupervised learning theory in connectionism. It has inspired wide research activity in the artificial neural network field because it embodies some interesting properties such as locality and the capability of being applicable to the basic weight-and-sum structure of neuron models. The plain Hebbian principle, however, also presents some inherent theoretical limitations that make it impractical in most cases. Therefore, modifications of the basic Hebbian learning paradigm have been proposed over the past 20 years in order to design profitable signal and data processing algorithms. Such modifications led to the principal component analysis type class of learning rules along with their nonlinear extensions. The aim of this review is primarily to present part of the existing fragmented material in the field of principal component learning within a unified view and contextually to motivate and present extensions of previous works on Hebbian learning to complex-weighted linear neural networks. This work benefits from previous studies on linear signal decomposition by artificial neural networks, nonquadratic component optimization and reconstruction error definition, neural parameters adaptation by constrained optimization of learning criteria of complex-valued arguments, and orthonormality expression via the insertion of topological elements in the networks or by modifying the network learning criterion. In particular, the learning principles considered here and their analysis concern complex-valued principal/minor component/subspace linear/nonlinear rules for complex-weighted neural structures, both feedforward and laterally connected.

Relative uncertainty learning theory: an essay.

The aim of this manuscript is to present a detailed analysis of the algebraic and geometric properties of relative uncertainty theory (RUT) applied to neural networks learning. Through the algebraic analysis of the original learning criterion, it is shown that RUT gives rise to principal-subspace-analysis-type learning equations. Through an algebraic-geometric analysis, the behavior of such matrix-type learning equations is illustrated, with particular emphasis to the existence of certain invariant manifolds.

Fast fixed-point neural blind-deconvolution algorithm.

The aim of this letter is to introduce a new blind-deconvolution algorithm based on fixed-point optimization of a "Bussgang"-type cost function. The cost function relies on approximate Bayesian estimation achieved by an adaptive neuron. The main feature of the presented algorithm is fast convergence that guarantees good deconvolution performances with limited computational demand as compared with algorithms of the same class.

Stiefel-manifold learning by improved rigid-body theory applied to ICA.

In previous contributions we presented a new class of algorithms for orthonormal learning of a linear neural network with p inputs and m outputs, based on the equations describing the dynamics of a massive rigid frame in a submanifold of R(p). While exhibiting interesting features, such as intrinsic numerical stability, strongly binding to the orthonormal submanifolds, and good controllability of the learning dynamics, tested on principal/independent component analysis, the proposed algorithms were not completely satisfactory from a computational-complexity point of view. The main drawback was the need to repeatedly evaluate a matrix exponential map. With the aim to lessen the computational efforts pertaining to these algorithms, we propose here an improvement based on the closed-form Rodriguez formula for the exponential map. Such formula is available in the p = 3 and m = 3 case, which is discussed with details here. In particular, experimental results on independent component analysis (ICA), carried out with both synthetic and real-world data, help confirming the computational gain due to the proposed improvement.

Closed-form expressions of some stochastic adapting equations for nonlinear adaptive activation function neurons.

In recent work, we introduced nonlinear adaptive activation function (FAN) artificial neuron models, which learn their activation functions in an unsupervised way by information-theoretic adapting rules. We also applied networks of these neurons to some blind signal processing problems, such as independent component analysis and blind deconvolution. The aim of this letter is to study some fundamental aspects of FAN units' learning by investigating the properties of the associated learning differential equation systems.

Neural independent component analysis by 'maximum-mismatch' learning principle.

The aim of the present paper is to apply Sudjanto-Hassoun theory of Hebbian learning to neural independent component analysis. The basic learning theory is first recalled and expanded in order to make it suitable for a network of non-linear complex-weighted neurons; then its interpretation and application is shown in the context of blind separation of complex-valued sources. Numerical results are given in order to assess the effectiveness of the proposed learning theory and the related separation algorithm on telecommunication signals; a comparison with other existing techniques finally helps assessing the performances and computational requirements of the proposed algorithm.