Multiview Orthonormalized Partial Least Squares: Regularizations and Deep Extensions.

In this article, we establish a family of subspace-based learning methods for multiview learning using least squares as the fundamental basis. Specifically, we propose a novel unified multiview learning framework called multiview orthonormalized partial least squares (MvOPLSs) to learn a classifier over a common latent space shared by all views. The regularization technique is further leveraged to unleash the power of the proposed framework by providing three types of regularizers on its basic ingredients, including model parameters, decision values, and latent projected points. With a set of regularizers derived from various priors, we not only recast most existing multiview learning methods into the proposed framework with properly chosen regularizers but also propose two novel models. To further improve the performance of the proposed framework, we propose to learn nonlinear transformations parameterized by deep networks. Extensive experiments are conducted on multiview datasets in terms of both feature extraction and cross-modal retrieval. Results show that the subspace-based learning for a common latent space is effective and its nonlinear extension can further boost performance, and more importantly, one of two proposed methods with nonlinear extension can achieve better results than all compared methods.

A Self-Consistent-Field Iteration for Orthogonal Canonical Correlation Analysis.

We propose an efficient algorithm for solving orthogonal canonical correlation analysis (OCCA) in the form of trace-fractional structure and orthogonal linear projections. Even though orthogonality has been widely used and proved to be a useful criterion for visualization, pattern recognition and feature extraction, existing methods for solving OCCA problem are either numerically unstable by relying on a deflation scheme, or less efficient by directly using generic optimization methods. In this paper, we propose an alternating numerical scheme whose core is the sub-maximization problem in the trace-fractional form with an orthogonality constraint. A customized self-consistent-field (SCF) iteration for this sub-maximization problem is devised. It is proved that the SCF iteration is globally convergent to a KKT point and that the alternating numerical scheme always converges. We further formulate a new trace-fractional maximization problem for orthogonal multiset CCA and propose an efficient algorithm with an either Jacobi-style or Gauss-Seidel-style updating scheme based on the SCF iteration. Extensive experiments are conducted to evaluate the proposed algorithms against existing methods, including real-world applications of multi-label classification and multi-view feature extraction. Experimental results show that our methods not only perform competitively to or better than the existing methods but also are more efficient.

Probabilistic Structure Learning for EEG/MEG Source Imaging With Hierarchical Graph Priors.

Brain source imaging is an important method for noninvasively characterizing brain activity using Electroencephalogram (EEG) or Magnetoencephalography (MEG) recordings. Traditional EEG/MEG Source Imaging (ESI) methods usually assume the source activities at different time points are unrelated, and do not utilize the temporal structure in the source activation, making the ESI analysis sensitive to noise. Some methods may encourage very similar activation patterns across the entire time course and may be incapable of accounting the variation along the time course. To effectively deal with noise while maintaining flexibility and continuity among brain activation patterns, we propose a novel probabilistic ESI model based on a hierarchical graph prior. Under our method, a spanning tree constraint ensures that activity patterns have spatiotemporal continuity. An efficient algorithm based on an alternating convex search is presented to solve the resulting problem of the proposed model with guaranteed convergence. Comprehensive numerical studies using synthetic data on a realistic brain model are conducted under different levels of signal-to-noise ratio (SNR) from both sensor and source spaces. We also examine the EEG/MEG datasets in two real applications, in which our ESI reconstructions are neurologically plausible. All the results demonstrate significant improvements of the proposed method over benchmark methods in terms of source localization performance, especially at high noise levels.

Learning Low-Dimensional Latent Graph Structures: A Density Estimation Approach.

We aim to automatically learn a latent graph structure in a low-dimensional space from high-dimensional, unsupervised data based on a unified density estimation framework for both feature extraction and feature selection, where the latent structure is considered as a compact and informative representation of the high-dimensional data. Based on this framework, two novel methods are proposed with very different but intuitive learning criteria from existing methods. The proposed feature extraction method can learn a set of embedded points in a low-dimensional space by naturally integrating the discriminative information of the input data with structure learning so that multiple disconnected embedding structures of data can be uncovered. The proposed feature selection method preserves the pairwise distances only on the optimal set of features and selects these features simultaneously. It not only obtains the optimal set of features but also learns both the structure and embeddings for visualization. Extensive experiments demonstrate that our proposed methods can achieve competitive quantitative (often better) results in terms of discriminant evaluation performance and are able to obtain the embeddings of smooth skeleton structures and select optimal features to unveil the correct graph structures of high-dimensional data sets.

A block variational procedure for the iterative diagonalization of non-Hermitian random-phase approximation matrices.

We present a technique for the iterative diagonalization of random-phase approximation (RPA) matrices, which are encountered in the framework of time-dependent density-functional theory (TDDFT) and the Bethe-Salpeter equation. The non-Hermitian character of these matrices does not permit a straightforward application of standard iterative techniques used, i.e., for the diagonalization of ground state Hamiltonians. We first introduce a new block variational principle for RPA matrices. We then develop an algorithm for the simultaneous calculation of multiple eigenvalues and eigenvectors, with convergence and stability properties similar to techniques used to iteratively diagonalize Hermitian matrices. The algorithm is validated for simple systems (Na(2) and Na(4)) and then used to compute multiple low-lying TDDFT excitation energies of the benzene molecule.