On the Comparisons of Decorrelation Approaches for Non-Gaussian Neutral Vector Variables.

As a typical non-Gaussian vector variable, a neutral vector variable contains nonnegative elements only, and its l1 -norm equals one. In addition, its neutral properties make it significantly different from the commonly studied vector variables (e.g., the Gaussian vector variables). Due to the aforementioned properties, the conventionally applied linear transformation approaches [e.g., principal component analysis (PCA) and independent component analysis (ICA)] are not suitable for neutral vector variables, as PCA cannot transform a neutral vector variable, which is highly negatively correlated, into a set of mutually independent scalar variables and ICA cannot preserve the bounded property after transformation. In recent work, we proposed an efficient nonlinear transformation approach, i.e., the parallel nonlinear transformation (PNT), for decorrelating neutral vector variables. In this article, we extensively compare PNT with PCA and ICA through both theoretical analysis and experimental evaluations. The results of our investigations demonstrate the superiority of PNT for decorrelating the neutral vector variables.

Advanced Dropout: A Model-Free Methodology for Bayesian Dropout Optimization.

Due to lack of data, overfitting ubiquitously exists in real-world applications of deep neural networks (DNNs). We propose advanced dropout, a model-free methodology, to mitigate overfitting and improve the performance of DNNs. The advanced dropout technique applies a model-free and easily implemented distribution with parametric prior, and adaptively adjusts dropout rate. Specifically, the distribution parameters are optimized by stochastic gradient variational Bayes in order to carry out an end-to-end training. We evaluate the effectiveness of the advanced dropout against nine dropout techniques on seven computer vision datasets (five small-scale datasets and two large-scale datasets) with various base models. The advanced dropout outperforms all the referred techniques on all the datasets. We further compare the effectiveness ratios and find that advanced dropout achieves the highest one on most cases. Next, we conduct a set of analysis of dropout rate characteristics, including convergence of the adaptive dropout rate, the learned distributions of dropout masks, and a comparison with dropout rate generation without an explicit distribution. In addition, the ability of overfitting prevention is evaluated and confirmed. Finally, we extend the application of the advanced dropout to uncertainty inference, network pruning, text classification, and regression. The proposed advanced dropout is also superior to the corresponding referred methods. Codes are available at https://github.com/PRIS-CV/AdvancedDropout.

OSLNet: Deep Small-Sample Classification with an Orthogonal Softmax Layer.

A deep neural network of multiple nonlinear layers forms a large function space, which can easily lead to overfitting when it encounters small-sample data. To mitigate overfitting in small-sample classification, learning more discriminative features from small-sample data is becoming a new trend. To this end, this paper aims to find a subspace of neural networks that can facilitate a large decision margin. Specifically, we propose the Orthogonal Softmax Layer (OSL), which makes the weight vectors in the classification layer remain orthogonal during both the training and test processes. The Rademacher complexity of a network using the OSL is only 1/K, where K is the number of classes, of that of a network using the fully connected classification layer, leading to a tighter generalization error bound. Experimental results demonstrate that the proposed OSL has better performance than the methods used for comparison on four small-sample benchmark datasets, as well as its applicability to large-sample datasets. Codes are available at: https://github.com/dongliangchang/OSLNet.

Quasi-Hodgkin-Huxley Neurons with Leaky Integrate-and-Fire Functions Physically Realized with Memristive Devices.

Artificial neurons with functions such as leaky integrate-and-fire (LIF) and spike output are essential for brain-inspired computation with high efficiency. However, previously implemented artificial neurons, e.g., Hodgkin-Huxley (HH) neurons, integrate-and-fire (IF) neurons, and LIF neurons, only achieve partial functionality of a biological neuron. In this work, quasi-HH neurons with leaky integrate-and-fire functions are physically demonstrated with a volatile memristive device, W/WO3 /poly(3,4-ethylenedioxythiophene): polystyrene sulfonate/Pt. The resistive switching behavior of the device can be attributed to the migration of protons, unlike the migration of oxygen ions normally involved in oxide-based memristors. With multifunctions similar to their biological counterparts, quasi-HH neurons are advantageous over the reported HH and LIF neurons, demonstrating their potential for neuromorphic computing applications.

Spoofing Detection in Automatic Speaker Verification Systems Using DNN Classifiers and Dynamic Acoustic Features.

With the development of speech synthesis technology, automatic speaker verification (ASV) systems have encountered the serious challenge of spoofing attacks. In order to improve the security of ASV systems, many antispoofing countermeasures have been developed. In the front-end domain, much research has been conducted on finding effective features which can distinguish spoofed speech from genuine speech and the published results show that dynamic acoustic features work more effectively than static ones. In the back-end domain, Gaussian mixture model (GMM) and deep neural networks (DNNs) are the two most popular types of classifiers used for spoofing detection. The log-likelihood ratios (LLRs) generated by the difference of human and spoofing log-likelihoods are used as spoofing detection scores. In this paper, we train a five-layer DNN spoofing detection classifier using dynamic acoustic features and propose a novel, simple scoring method only using human log-likelihoods (HLLs) for spoofing detection. We mathematically prove that the new HLL scoring method is more suitable for the spoofing detection task than the classical LLR scoring method, especially when the spoofing speech is very similar to the human speech. We extensively investigate the performance of five different dynamic filter bank-based cepstral features and constant Q cepstral coefficients (CQCC) in conjunction with the DNN-HLL method. The experimental results show that, compared to the GMM-LLR method, the DNN-HLL method is able to significantly improve the spoofing detection accuracy. Compared with the CQCC-based GMM-LLR baseline, the proposed DNN-HLL model reduces the average equal error rate of all attack types to 0.045%, thus exceeding the performance of previously published approaches for the ASVspoof 2015 Challenge task. Fusing the CQCC-based DNN-HLL spoofing detection system with ASV systems, the false acceptance rate on spoofing attacks can be reduced significantly.

Decorrelation of Neutral Vector Variables: Theory and Applications.

In this paper, we propose novel strategies for neutral vector variable decorrelation. Two fundamental invertible transformations, namely, serial nonlinear transformation and parallel nonlinear transformation, are proposed to carry out the decorrelation. For a neutral vector variable, which is not multivariate-Gaussian distributed, the conventional principal component analysis cannot yield mutually independent scalar variables. With the two proposed transformations, a highly negatively correlated neutral vector can be transformed to a set of mutually independent scalar variables with the same degrees of freedom. We also evaluate the decorrelation performances for the vectors generated from a single Dirichlet distribution and a mixture of Dirichlet distributions. The mutual independence is verified with the distance correlation measurement. The advantages of the proposed decorrelation strategies are intensively studied and demonstrated with synthesized data and practical application evaluations.

Pavlovian conditioning demonstrated with neuromorphic memristive devices.

Pavlovian conditioning, a classical case of associative learning in a biological brain, is demonstrated using the Ni/Nb-SrTiO3/Ti memristive device with intrinsic forgetting properties in the framework of the asymmetric spike-timing-dependent plasticity of synapses. Three basic features of the Pavlovian conditioning, namely, acquisition, extinction and recovery, are implemented in detail. The effects of the temporal relation between conditioned and unconditioned stimuli as well as the time interval between individual training trials on the Pavlovian conditioning are investigated. The resulting change of the response strength, the number of training trials necessary for acquisition and the number of extinction trials are illustrated. This work clearly demonstrates the hardware implementation of the brain function of the associative learning.

Mimicking the brain functions of learning, forgetting and explicit/implicit memories with SrTiO3-based memristive devices.

To implement the complex brain functions of learning, forgetting and memory in a single electronic device is very advantageous for realizing artificial intelligence. As a proof of concept, memristive devices with a simple structure of Ni/Nb-SrTiO3/Ti were investigated in this work. The functions of learning, forgetting and memory were successfully mimicked using the memristive devices, and the "time-saving" effect of implicit memory was also demonstrated. The physics behind the brain functions is simply the modulation of the Schottky barrier at the Ni/SrTiO3 interface. The realization of various psychological functions in a single device simplifies the construction of the artificial neural network and facilitates the advent of artificial intelligence.

Revival of "dead" memristive devices: case of WO3-x.

Inappropriate operation could make a memristive device "dead" and cause the loss of resistive switching performance. In this study, the revival of "dead" devices was investigated in the case of WO3-x-based memristive devices. It is believed that inappropriate operation with a high-voltage pulse creates an ordered structure of oxygen vacancies and such an ordered structure makes the normal reset process fail. By precisely controlled voltage sweeping at certain compliance currents, a "dead" device can be revived. The revival operation disrupts the ordered structure by Joule heating and recovers Schottky-like barrier modulation-based switching.

Synaptic Metaplasticity Realized in Oxide Memristive Devices.

Metaplasticity, a higher order of synaptic plasticity, as well as a key issue in neuroscience, is realized with artificial synapses based on a WO3 thin film, and the activity-dependent metaplastic responses of the artificial synapses, such as spike-timing-dependent plasticity, are systematically investigated. This work has significant implications in neuromorphic computation.

Assessing the Potential Use of Eye-Tracking Triangulation for Evaluating the Usability of an Online Diabetes Exercise System.

The Online Diabetes Exercise System was developed to motivate people with Type 2 diabetes to do a 25 minutes low-volume high-intensity interval training program. In a previous multi-method evaluation of the system, several usability issues were identified and corrected. Despite the thorough testing, it was unclear whether all usability problems had been identified using the multi-method evaluation. Our hypothesis was that adding the eye-tracking triangulation to the multi-method evaluation would increase the accuracy and completeness when testing the usability of the system. The study design was an Eye-tracking Triangulation; conventional eye-tracking with predefined tasks followed by The Post-Experience Eye-Tracked Protocol (PEEP). Six Areas of Interests were the basis for the PEEP-session. The eye-tracking triangulation gave objective and subjective results, which are believed to be highly relevant for designing, implementing, evaluating and optimizing systems in the field of health informatics. Future work should include testing the method on a larger and more representative group of users and apply the method on different system types.

The role of Schottky barrier in the resistive switching of SrTiO3: direct experimental evidence.

Single crystalline SrTiO3 doped with 0.1 wt% Nb was used as a model system to evaluate the role of the Schottky barrier in the resistive switching of perovskites. The Ti bottom electrode formed an ohmic contact in the Ni/Nb:SrTiO3/Ti stack, whereas the Ni top electrode created a strong Schottky barrier, which was reflected in a huge semi-circle in the impedance spectrum of the stack. Bipolar switching was achieved in the voltage range of -4 to 4 V for the stack, two clear resistance states were created by electric pulses, and the Schottky barrier heights corresponding to the high/low resistance states were experimentally determined. A direct relationship between the resistance state and the Schottky barrier height was thus established.

Exploring super-Gaussianity toward robust information-theoretical time delay estimation.

Time delay estimation (TDE) is a fundamental component of speaker localization and tracking algorithms. Most of the existing systems are based on the generalized cross-correlation method assuming gaussianity of the source. It has been shown that the distribution of speech, captured with far-field microphones, is highly varying, depending on the noise and reverberation conditions. Thus the performance of TDE is expected to fluctuate depending on the underlying assumption for the speech distribution, being also subject to multi-path reflections and competitive background noise. This paper investigates the effect upon TDE when modeling the source signal with different speech-based distributions. An information theoretical TDE method indirectly encapsulating higher order statistics (HOS) formed the basis of this work. The underlying assumption of Gaussian distributed source has been replaced by that of generalized Gaussian distribution that allows evaluating the problem under a larger set of speech-shaped distributions, ranging from Gaussian to Laplacian and Gamma. Closed forms of the univariate and multivariate entropy expressions of the generalized Gaussian distribution are derived to evaluate the TDE. The results indicate that TDE based on the specific criterion is independent of the underlying assumption for the distribution of the source, for the same covariance matrix.