Independent vector analysis for source separation using a mixture of gaussians prior.

Convolutive mixtures of signals, which are common in acoustic environments, can be difficult to separate into their component sources. Here we present a uniform probabilistic framework to separate convolutive mixtures of acoustic signals using independent vector analysis (IVA), which is based on a joint distribution for the frequency components originating from the same source and is capable of preventing permutation disorder. Different gaussian mixture models (GMM) served as source priors, in contrast to the original IVA model, where all sources were modeled by identical multivariate Laplacian distributions. This flexible source prior enabled the IVA model to separate different type of signals. Three classes of models were derived and tested: noiseless IVA, online IVA, and noisy IVA. In the IVA model without sensor noise, the unmixing matrices were efficiently estimated by the expectation maximization (EM) algorithm. An online EM algorithm was derived for the online IVA algorithm to track the movement of the sources and separate them under nonstationary conditions. The noisy IVA model included the sensor noise and combined denoising with separation. An EM algorithm was developed that found the model parameters and separated the sources simultaneously. These algorithms were applied to separate mixtures of speech and music. Performance as measured by the signal-to-interference ratio (SIR) was substantial for all three models.

Bayesian machine learning classifiers for combining structural and functional measurements to classify healthy and glaucomatous eyes.

PURPOSE: To determine whether combining structural (optical coherence tomography, OCT) and functional (standard automated perimetry, SAP) measurements as input for machine learning classifiers (MLCs; relevance vector machine, RVM; and subspace mixture of Gaussians, SSMoG) improves diagnostic accuracy for detecting glaucomatous eyes compared with using each measurement method alone. METHODS: Sixty-nine eyes of 69 healthy control subjects (average age, 62.0, SD 9.7 years; visual field mean deviation [MD], -0.70, SD 1.41 dB) and 156 eyes of 156 patients with glaucoma (average age, 66.4, SD 10.2 years; visual field MD, -3.12, SD 3.43 dB) were imaged with OCT (Stratus OCT, Carl Zeiss Meditec, Inc., Dublin, CA) and tested with SAP (Humphrey Field Analyzer II with Swedish Interactive Thresholding Algorithm, SITA; Carl Zeiss Meditec, Inc.) within 3 months of each other. RVM and SSMoG MLCs were trained and tested on OCT-determined RNFL thickness measurements from 32 sectors ( approximately 11.25 degrees each) obtained in the circumpapillary area under the instrument-defined measurement ellipse and SAP pattern deviation values from 52 points from the 24-2 grid, independently and in combination. Tenfold cross-validation was used to train and test classifiers on unique subsets of the full 225-eye data set, and areas under the receiver operating characteristic curve (AUROC) for the classification of eyes in the test set were generated. AUROC results from classifiers trained on OCT and SAP alone and those trained on OCT and SAP in combination were compared. In addition, these results were compared to currently available OCT measurements (mean retinal nerve fiber layer [RNFL] thickness, inferior RNFL thickness, and superior RNFL thickness) and SAP indices (MD and pattern standard deviation [PSD]). RESULTS: The AUROCs for RVM trained on OCT parameters alone, SAP parameters alone and OCT and SAP parameters combined were 0.809, 0.815, and 0.845, respectively. The AUROCs for SSMoG trained on OCT parameters alone, SAP parameters alone, and OCT and SAP parameters combined were 0.817, 0.841, and 0.869, respectively. Combining techniques using both RVM and SSMoG significantly improved on MLC analysis of OCT, but not SAP, measurements alone. Classification performance using RVM and SSMoG was statistically similar. CONCLUSIONS: RVM and SSMoG Bayesian MLCs trained on OCT and SAP data can successfully discriminate between healthy and early glaucomatous eyes. Combining OCT and SAP measurements using RVM and SSMoG increased diagnostic performance marginally compared with MLC analysis of data obtained using each technology alone.

Cone selectivity derived from the responses of the retinal cone mosaic to natural scenes.

To achieve color vision, the brain has to process signals of the cones in the retinal photoreceptor mosaic in a cone-type-specific way. We investigated the possibility that cone-type-specific wiring is an adaptation to the statistics of the cone signals. We analyzed estimates of cone responses to natural scenes and found that there is sufficient information in the higher order statistics of L- and M-cone responses to distinguish between cones of different types, enabling unsupervised learning of cone-type specificity. This was not the case for a fourth cone type with spectral sensitivity between L and M cones, suggesting an explanation for the lack of strong tetrachromacy in heterozygous carriers of color deficiencies.

Relevance vector machine and support vector machine classifier analysis of scanning laser polarimetry retinal nerve fiber layer measurements.

PURPOSE: To classify healthy and glaucomatous eyes using relevance vector machine (RVM) and support vector machine (SVM) learning classifiers trained on retinal nerve fiber layer (RNFL) thickness measurements obtained by scanning laser polarimetry (SLP). METHODS: Seventy-two eyes of 72 healthy control subjects (average age = 64.3 +/- 8.8 years, visual field mean deviation = -0.71 +/- 1.2 dB) and 92 eyes of 92 patients with glaucoma (average age = 66.9 +/- 8.9 years, visual field mean deviation = -5.32 +/- 4.0 dB) were imaged with SLP with variable corneal compensation (GDx VCC; Laser Diagnostic Technologies, San Diego, CA). RVM and SVM learning classifiers were trained and tested on SLP-determined RNFL thickness measurements from 14 standard parameters and 64 sectors (approximately 5.6 degrees each) obtained in the circumpapillary area under the instrument-defined measurement ellipse (total 78 parameters). Ten-fold cross-validation was used to train and test RVM and SVM classifiers on unique subsets of the full 164-eye data set and areas under the receiver operating characteristic (AUROC) curve for the classification of eyes in the test set were generated. AUROC curve results from RVM and SVM were compared to those for 14 SLP software-generated global and regional RNFL thickness parameters. Also reported was the AUROC curve for the GDx VCC software-generated nerve fiber indicator (NFI). RESULTS: The AUROC curves for RVM and SVM were 0.90 and 0.91, respectively, and increased to 0.93 and 0.94 when the training sets were optimized with sequential forward and backward selection (resulting in reduced dimensional data sets). AUROC curves for optimized RVM and SVM were significantly larger than those for all individual SLP parameters. The AUROC curve for the NFI was 0.87. CONCLUSIONS: Results from RVM and SVM trained on SLP RNFL thickness measurements are similar and provide accurate classification of glaucomatous and healthy eyes. RVM may be preferable to SVM, because it provides a Bayesian-derived probability of glaucoma as an output. These results suggest that these machine learning classifiers show good potential for glaucoma diagnosis.

Using unsupervised learning with independent component analysis to identify patterns of glaucomatous visual field defects.

PURPOSE: Clustering by unsupervised learning with machine learning classifiers was shown to segment clusters of patterns in standard automated perimetry (SAP) for glaucoma in previous publications. In this study, unsupervised learning by independent component analysis decomposed SAP field patterns into axes, and the information represented by these axes was evaluated. METHODS: SAP fields were used that were obtained with the Humphrey Visual Field Analyzer (Carl Zeiss Meditec, Dublin, CA) from 189 normal eyes and 156 eyes with glaucomatous optic neuropathy (GON) determined by masked review with stereoscopic optic disc photographs. The variational Bayesian independent component analysis mixture model (vB-ICA-mm) partitioned the SAP fields into the most informative number of clusters. Simultaneously, the model learned an optimal number of maximally independent axes for each cluster. RESULTS: The most informative number of clusters in the SAP set was two. vB-ICA-mm placed 68.6% of the eyes with GON in a cluster labeled G and 98.4% of the eyes with normal optic discs in a cluster labeled N. Cluster G optimally contained six axes. Post hoc analysis of patterns generated at -1 SD and +2 SD from the cluster G mean on the six axes revealed defects similar to those identified by experts as indicative of glaucoma. SAP fields associated with an axis showed increasing severity, as they were located farther in the positive direction from the cluster G mean. CONCLUSIONS: vB-ICA-mm represented the SAP fields with patterns that were meaningful for glaucoma experts. This process also captured severity in the patterns uncovered. These findings should validate vB-ICA-mm as a data-mining technique for new and unfamiliar complex tests.

Unsupervised machine learning with independent component analysis to identify areas of progression in glaucomatous visual fields.

PURPOSE: To determine whether a variational Bayesian independent component analysis mixture model (vB-ICA-mm), a form of unsupervised machine learning, can be used to identify and quantify areas of progression in standard automated perimetry fields. METHODS: In an earlier study, it was shown that a model using vB-ICA-mm can separate normal fields from fields with six different patterns of visual field loss related to glaucomatous optic neuropathy (GON) along maximally independent axes. In the present study, an independent group of 191 patient eyes (66 with ocular hypertension (OHT), 12 with suspected glaucoma by field, 61 with suspected glaucoma by disc, and 52 with glaucoma) with five or more standard visual fields under observation for a mean of 6.24 +/- 2.65 years and 8.11 +/- 2.42 visual fields were evaluated with the vB-ICA-mm. In addition, eyes with progressive GON (PGON) were identified (n = 39). Each participant had a series of fields tested, with each field entered independently and placed along the axes of the previously developed model. This allowed change in one pattern of visual field defect (along one axis) to be assessed relative to results other areas of that same field (no change along other axes). Progression was based on a slope falling outside the 5th and the 95th percentile limits of all slopes, with at least two axes not showing such a deviation in a given individual's series of fields. Fields were also scored using Advanced Glaucoma Intervention Study (AGIS) and the Early Manifest Glaucoma Treatment Trial (EMGT) criteria. RESULTS: Thirty-two of 191 eyes progressed on vB-ICA-mm by this definition. Of the 32, 22 had field loss at baseline, 7 had only GON, 3 were OHTs and 12 were from the 39 eyes (31%) with PGON. The vB-ICA-mm identified a higher percentage of progressing eyes in each diagnostic category than did AGIS or and the EMGT. CONCLUSIONS: The vB-ICA-mm can quantitatively identify progression in eyes with glaucoma by evaluating change in one or more patterns of the visual field loss while other areas or patterns remain stable. This may enable each eye to contribute to the determination of whether change is caused by true progression or by variability.

Comparing neural networks and linear discriminant functions for glaucoma detection using confocal scanning laser ophthalmoscopy of the optic disc.

PURPOSE: To determine whether neural network techniques can improve differentiation between glaucomatous and nonglaucomatous eyes, using the optic disc topography parameters of the Heidelberg Retina Tomograph (HRT; Heidelberg Engineering, Heidelberg, Germany). METHODS: With the HRT, one eye was imaged from each of 108 patients with glaucoma (defined as having repeatable visual field defects with standard automated perimetry) and 189 subjects without glaucoma (no visual field defects with healthy-appearing optic disc and retinal nerve fiber layer on clinical examination) and the optic nerve topography was defined by 17 global and 66 regional HRT parameters. With all the HRT parameters used as input, receiver operating characteristic (ROC) curves were generated for the classification of eyes, by three neural network techniques: linear and Gaussian support vector machines (SVM linear and SVM Gaussian, respectively) and a multilayer perceptron (MLP), as well as four previously proposed linear discriminant functions (LDFs) and one LDF developed on the current data with all HRT parameters used as input. RESULTS: The areas under the ROC curves for SVM linear and SVM Gaussian were 0.938 and 0.945, respectively; for MLP, 0.941; for the current LDF, 0.906; and for the best previously proposed LDF, 0.890. With the use of forward selection and backward elimination optimization techniques, the areas under the ROC curves for SVM Gaussian and the current LDF were increased to approximately 0.96. CONCLUSIONS: Trained neural networks, with global and regional HRT parameters used as input, improve on previously proposed HRT parameter-based LDFs for discriminating between glaucomatous and nonglaucomatous eyes. The performance of both neural networks and LDFs can be improved with optimization of the features in the input. Neural network analyses show promise for increasing diagnostic accuracy of tests for glaucoma.

Heidelberg retina tomograph measurements of the optic disc and parapapillary retina for detecting glaucoma analyzed by machine learning classifiers.

PURPOSE: To determine whether topographical measurements of the parapapillary region analyzed by machine learning classifiers can detect early to moderate glaucoma better than similarly processed measurements obtained within the disc margin and to improve methods for optimization of machine learning classifier feature selection. METHODS: One eye of each of 95 patients with early to moderate glaucomatous visual field damage and of each of 135 normal subjects older than 40 years participating in the longitudinal Diagnostic Innovations in Glaucoma Study (DIGS) were included. Heidelberg Retina Tomograph (HRT; Heidelberg Engineering, Dossenheim, Germany) mean height contour was measured in 36 equal sectors, both along the disc margin and in the parapapillary region (at a mean contour line radius of 1.7 mm). Each sector was evaluated individually and in combination with other sectors. Gaussian support vector machine (SVM) learning classifiers were used to interpret HRT sector measurements along the disc margin and in the parapapillary region, to differentiate between eyes with normal and glaucomatous visual fields and to compare the results with global and regional HRT parameter measurements. The area under the receiver operating characteristic (ROC) curve was used to measure diagnostic performance of the HRT parameters and to evaluate the cross-validation strategies and forward selection and backward elimination optimization techniques that were used to generate the reduced feature sets. RESULTS: The area under the ROC curve for mean height contour of the 36 sectors along the disc margin was larger than that for the mean height contour in the parapapillary region (0.97 and 0.85, respectively). Of the 36 individual sectors along the disc margin, those in the inferior region between 240 degrees and 300 degrees, had the largest area under the ROC curve (0.85-0.91). With SVM Gaussian techniques, the regional parameters showed the best ability to discriminate between normal eyes and eyes with glaucomatous visual field damage, followed by the global parameters, mean height contour measures along the disc margin, and mean height contour measures in the parapapillary region. The area under the ROC curve was 0.98, 0.94, 0.93, and 0.85, respectively. Cross-validation and optimization techniques demonstrated that good discrimination (99% of peak area under the ROC curve) can be obtained with a reduced number of HRT parameters. CONCLUSIONS: Mean height contour measurements along the disc margin discriminated between normal and glaucomatous eyes better than measurements obtained in the parapapillary region.

Comparing machine learning classifiers for diagnosing glaucoma from standard automated perimetry.

PURPOSE: To determine which machine learning classifier learns best to interpret standard automated perimetry (SAP) and to compare the best of the machine classifiers with the global indices of STATPAC 2 and with experts in glaucoma. METHODS: Multilayer perceptrons (MLP), support vector machines (SVM), mixture of Gaussian (MoG), and mixture of generalized Gaussian (MGG) classifiers were trained and tested by cross validation on the numerical plot of absolute sensitivity plus age of 189 normal eyes and 156 glaucomatous eyes, designated as such by the appearance of the optic nerve. The authors compared performance of these classifiers with the global indices of STATPAC, using the area under the ROC curve. Two human experts were judged against the machine classifiers and the global indices by plotting their sensitivity-specificity pairs. RESULTS: MoG had the greatest area under the ROC curve of the machine classifiers. Pattern SD (PSD) and corrected PSD (CPSD) had the largest areas under the curve of the global indices. MoG had significantly greater ROC area than PSD and CPSD. Human experts were not better at classifying visual fields than the machine classifiers or the global indices. CONCLUSIONS: MoG, using the entire visual field and age for input, interpreted SAP better than the global indices of STATPAC. Machine classifiers may augment the global indices of STATPAC.

Using machine learning classifiers to identify glaucomatous change earlier in standard visual fields.

PURPOSE: To compare the ability of several machine learning classifiers to predict development of abnormal fields at follow-up in ocular hypertensive (OHT) eyes that had normal visual fields in baseline examination. METHODS: The visual fields of 114 eyes of 114 patients with OHT with four or more visual field tests with standard automated perimetry over three or more years and for whom stereophotographs were available were assessed. The mean (+/-SD) number of visual field tests was 7.89 +/- 3.04. The mean number of years covered (+/-SD) was 5.92 +/- 2.34 (range, 2.81-11.77). Fields were classified as normal or abnormal based on Statpac-like methods (Humphrey Instruments, Dublin, CA) and by several machine learning classifiers. The machine learning classifiers were two types of support vector machine (SVM), a mixture of Gaussian (MoG) classifier, a constrained MoG, and a mixture of generalized Gaussian (MGG). Specificity was set to 96% for all classifiers, using data from 94 normal eyes evaluated longitudinally. Specificity cutoffs required confirmation of abnormality. RESULTS: Thirty-two percent (36/114) of the eyes converted to abnormal fields during follow-up based on the Statpac-like methods. All 36 were identified by at least one machine classifier. In nearly all cases, the machine learning classifiers predicted the confirmed abnormality, on average, 3.92 +/- 0.55 years earlier than traditional Statpac-like methods. CONCLUSIONS: Machine learning classifiers can learn complex patterns and trends in data and adapt to create a decision surface without the constraints imposed by statistical classifiers. This adaptation allowed the machine learning classifiers to identify abnormality in visual field converts much earlier than the traditional methods.

Using unsupervised learning with variational bayesian mixture of factor analysis to identify patterns of glaucomatous visual field defects.

PURPOSE: To determine whether an unsupervised machine learning classifier can identify patterns of visual field loss in standard visual fields consistent with typical patterns learned by decades of human experience. METHODS: Standard perimetry thresholds for 52 locations plus age from one eye of each of 156 patients with glaucomatous optic neuropathy (GON) and 189 eyes of healthy subjects were clustered with an unsupervised machine classifier, variational Bayesian mixture of factor analysis (vbMFA). RESULTS: The vbMFA formed five distinct clusters. Cluster 5 held 186 of 189 fields from normal eyes plus 46 from eyes with GON. These fields were then judged within normal limits by several traditional methods. Each of the other four clusters could be described by the pattern of loss found within it. Cluster 1 (71 GON + 3 normal optic discs) included early, localized defects. A purely diffuse component was rare. Cluster 2 (26 GON) exhibited primarily deep superior hemifield defects, and cluster 3 (10 GON) held deep inferior hemifield defects only or in combination with lesser superior field defects. Cluster 4 (6 GON) showed deep defects in both hemifields. In other words, visual fields within a given cluster had similar patterns of loss that differed from the predominant pattern found in other clusters. The classifier separated the data based solely on the patterns of loss within the fields, without being guided by the diagnosis, placing 98.4% of the healthy eyes within the same cluster and spreading 70.5% of the eyes with GON across the other four clusters, in good agreement with a glaucoma expert and pattern standard deviation. CONCLUSIONS: Without training-based diagnosis (unsupervised learning), the vbMFA identified four important patterns of field loss in eyes with GON in a manner consistent with years of clinical experience.

Confocal scanning laser ophthalmoscopy classifiers and stereophotograph evaluation for prediction of visual field abnormalities in glaucoma-suspect eyes.

PURPOSE: To determine whether Heidelberg Retina Tomograph (HRT; Heidelberg Engineering, Dossenheim, Germany) classification techniques and investigational support vector machine (SVM) analyses can detect optic disc abnormalities in glaucoma-suspect eyes before the development of visual field abnormalities. METHODS: Glaucoma-suspect eyes (n = 226) were classified as converts or nonconverts based on the development of repeatable (either two or three consecutive) standard automated perimetry (SAP)-detected abnormalities over the course of the study (mean follow-up, approximately 4.5 years). Hazard ratios for development of SAP abnormalities were calculated based on baseline classification results, follow-up time, and end point status (convert, nonconvert). Classification techniques applied were HRT classification (HRTC), Moorfields Regression Analysis, forward-selection optimized SVM (SVM fwd) and backward elimination-optimized SVM (SVM back) analysis of HRT data, and stereophotograph assessment. RESULTS: Univariate analyses indicated that all classification techniques were predictors of the development of two repeatable abnormal SAP results, with hazards ratios (95% confidence interval [CI]) ranging from 1.32 (1.00-1.75) for HRTC to 2.0 (1.48-2.76) for stereophotograph assessment (all P < or = 0.05). Only SVM (SVM fwd and SVM back) analysis of HRT data and stereophotograph assessment were univariate predictors of the development of three repeatable abnormal SAP results, with hazard ratios (95% CI) ranging from 1.73 (1.16-2.82) for SVM fwd to 1.82 (1.19-3.12) for SVM back (both P < 0.007). Multivariate analyses including each classification technique individually in a model with age, baseline SAP pattern standard deviation [PSD], and baseline IOP indicated that all classification techniques except HRTC (P = 0.06) were predictors of the development of two repeatable abnormal SAP results with hazards ratios ranging from 1.30 (0.99, 1.73) for HRTC to 1.90 (1.37, 2.69) for stereophotograph assessment. Only SVM (SVM fwd and SVM back) analysis of HRT data and stereophotograph assessment were significant predictors of the development of three repeatable abnormal SAP results in multivariate analyses; hazard ratios of 1.57 (1.03, 2.59) and 1.70 (1.18, 2.51), respectively. SAP PSD was a significant predictor of two repeatable abnormal SAP results in multivariate models with all classification techniques, with hazard ratios ranging from 3.31 (1.39, 7.89) to 4.70 (2.02, 10.93) per 1-dB increase. CONCLUSIONS: HRT classifications techniques and stereophotograph assessment can detect optic disc topography abnormalities in glaucoma-suspect eyes before the development of SAP abnormalities. These data support strongly the importance of optic disc examination for early glaucoma diagnosis.

Color opponency is an efficient representation of spectral properties in natural scenes.

The human visual system encodes the chromatic signals conveyed by the three types of retinal cone photoreceptors in an opponent fashion. This opponency is thought to reduce redundant information by decorrelating the photoreceptor signals. Correlations in the receptor signals are caused by the substantial overlap of the spectral sensitivities of the receptors, but it is not clear to what extent the properties of natural spectra contribute to the correlations. To investigate the influences of natural spectra and photoreceptor spectral sensitivities, we attempted to find linear codes with minimal redundancy for trichromatic images assuming human cone spectral sensitivities, or hypothetical non-overlapping cone sensitivities, respectively. The resulting properties of basis functions are similar in both cases. They are non-orthogonal, show strong opponency along an achromatic direction (luminance edges) and along chromatic directions, and they achieve a highly efficient encoding of natural chromatic signals. Thus, color opponency arises for the encoding of human cone signals, i.e. with strongly overlapping spectral sensitivities, but also under the assumption of non-overlapping spectral sensitivities. Our results suggest that color opponency may in part be a result of the properties of natural spectra and not solely a consequence of the cone spectral sensitivities.

Imaging Brain Dynamics Using Independent Component Analysis.

The analysis of electroencephalographic (EEG) and magnetoencephalographic (MEG) recordings is important both for basic brain research and for medical diagnosis and treatment. Independent component analysis (ICA) is an effective method for removing artifacts and separating sources of the brain signals from these recordings. A similar approach is proving useful for analyzing functional magnetic resonance brain imaging (fMRI) data. In this paper, we outline the assumptions underlying ICA and demonstrate its application to a variety of electrical and hemodynamic recordings from the human brain.

Comparison of machine learning and traditional classifiers in glaucoma diagnosis.

Glaucoma is a progressive optic neuropathy with characteristic structural changes in the optic nerve head reflected in the visual field. The visual-field sensitivity test is commonly used in a clinical setting to evaluate glaucoma. Standard automated perimetry (SAP) is a common computerized visual-field test whose output is amenable to machine learning. We compared the performance of a number of machine learning algorithms with STATPAC indexes mean deviation, pattern standard deviation, and corrected pattern standard deviation. The machine learning algorithms studied included multilayer perceptron (MLP), support vector machine (SVM), and linear (LDA) and quadratic discriminant analysis (QDA), Parzen window, mixture of Gaussian (MOG), and mixture of generalized Gaussian (MGG). MLP and SVM are classifiers that work directly on the decision boundary and fall under the discriminative paradigm. Generative classifiers, which first model the data probability density and then perform classification via Bayes' rule, usually give deeper insight into the structure of the data space. We have applied MOG, MGG, LDA, QDA, and Parzen window to the classification of glaucoma from SAP. Performance of the various classifiers was compared by the areas under their receiver operating characteristic curves and by sensitivities (true-positive rates) at chosen specificities (true-negative rates). The machine-learning-type classifiers showed improved performance over the best indexes from STATPAC. Forward-selection and backward-elimination methodology further improved the classification rate and also has the potential to reduce testing time by diminishing the number of visual-field location measurements.

Unsupervised image classification, segmentation, and enhancement using ICA mixture models.

An unsupervised classification algorithm is derived by modeling observed data as a mixture of several mutually exclusive classes that are each described by linear combinations of independent, non-Gaussian densities. The algorithm estimates the data density in each class by using parametric nonlinear functions that fit to the non-Gaussian structure of the data. This improves classification accuracy compared with standard Gaussian mixture models. When applied to images, the algorithm can learn efficient codes (basis functions) for images that capture the statistically significant structure intrinsic in the images. We apply this technique to the problem of unsupervised classification, segmentation, and denoising of images. We demonstrate that this method was effective in classifying complex image textures such as natural scenes and text. It was also useful for denoising and filling in missing pixels in images with complex structures. The advantage of this model is that image codes can be learned with increasing numbers of classes thus providing greater flexibility in modeling structure and in finding more image features than in either Gaussian mixture models or standard independent component analysis (ICA) algorithms.

A negentropy minimization approach to adaptive equalization for digital communication systems.

In this paper, we introduce and investigate a new adaptive equalization method based on minimizing approximate negentropy of the estimation error for a finite-length equalizer. We consider an approximate negentropy using nonpolynomial expansions of the estimation error as a new performance criterion to improve performance of a linear equalizer based on minimizing minimum mean squared error (MMSE). Negentropy includes higher order statistical information and its minimization provides improved converge, performance and accuracy compared to traditional methods such as MMSE in terms of bit error rate (BER). The proposed negentropy minimization (NEGMIN) equalizer has two kinds of solutions, the MMSE solution and the other one, depending on the ratio of the normalization parameters. The NEGMIN equalizer has best BER performance when the ratio of the normalization parameters is properly adjusted to maximize the output power(variance) of the NEGMIN equalizer. Simulation experiments show that BER performance of the NEGMIN equalizer with the other solution than the MMSE one has similar characteristics to the adaptive minimum bit error rate (AMBER) equalizer. The main advantage of the proposed equalizer is that it needs significantly fewer training symbols than the AMBER equalizer. Furthermore, the proposed equalizer is more robust to nonlinear distortions than the MMSE equalizer.

Speech Enhancement Using Gaussian Scale Mixture Models.

This paper presents a novel probabilistic approach to speech enhancement. Instead of a deterministic logarithmic relationship, we assume a probabilistic relationship between the frequency coefficients and the log-spectra. The speech model in the log-spectral domain is a Gaussian mixture model (GMM). The frequency coefficients obey a zero-mean Gaussian whose covariance equals to the exponential of the log-spectra. This results in a Gaussian scale mixture model (GSMM) for the speech signal in the frequency domain, since the log-spectra can be regarded as scaling factors. The probabilistic relation between frequency coefficients and log-spectra allows these to be treated as two random variables, both to be estimated from the noisy signals. Expectation-maximization (EM) was used to train the GSMM and Bayesian inference was used to compute the posterior signal distribution. Because exact inference of this full probabilistic model is computationally intractable, we developed two approaches to enhance the efficiency: the Laplace method and a variational approximation. The proposed methods were applied to enhance speech corrupted by Gaussian noise and speech-shaped noise (SSN). For both approximations, signals reconstructed from the estimated frequency coefficients provided higher signal-to-noise ratio (SNR) and those reconstructed from the estimated log-spectra produced lower word recognition error rate because the log-spectra fit the inputs to the recognizer better. Our algorithms effectively reduced the SSN, which algorithms based on spectral analysis were not able to suppress.

Combining functional and structural tests improves the diagnostic accuracy of relevance vector machine classifiers.

PURPOSE: To investigate whether combining optic disc topography and short-wavelength automated perimetry (SWAP) data improves the diagnostic accuracy of relevance vector machine (RVM) classifiers for detecting glaucomatous eyes compared with using each test alone. METHODS: One eye of 144 glaucoma patients and 68 healthy controls from the Diagnostic Innovations in Glaucoma Study were included. RVM were trained and tested with cross-validation on optimized (backward elimination) SWAP features (thresholds plus age; pattern deviation; and total deviation) and on Heidelberg retina tomograph II (HRT) optic disc topography features, independently and in combination. RVM performance was also compared with 2 HRT linear discriminant functions and to SWAP mean deviation and pattern standard deviation. Classifier performance was measured by the area under the receiver operating characteristic curves (AUROCs) generated for each feature set and by the sensitivities at set specificities of 75%, 90%, and 96%. RESULTS: RVM trained on combined HRT and SWAP thresholds plus age had significantly higher AUROC (0.93) than RVM trained on HRT (0.88) and SWAP (0.76) alone. AUROCs for the SWAP global indices (mean deviation: 0.68; pattern standard deviation: 0.72) offered no advantage over SWAP thresholds plus age, whereas the linear discriminant functions AUROCs were significantly lower than RVM trained on the combined SWAP and HRT feature set and on HRT alone feature set. CONCLUSIONS: Training RVM on combined optimized HRT and SWAP data improved diagnostic accuracy compared with training on SWAP and HRT parameters alone. Future research may identify other combinations of tests and classifiers that can also improve diagnostic accuracy.

Speech Enhancement, Gain, and Noise Spectrum Adaptation Using Approximate Bayesian Estimation.

This paper presents a new approximate Bayesian estimator for enhancing a noisy speech signal. The speech model is assumed to be a Gaussian mixture model (GMM) in the log-spectral domain. This is in contrast to most current models in frequency domain. Exact signal estimation is a computationally intractable problem. We derive three approximations to enhance the efficiency of signal estimation. The Gaussian approximation transforms the log-spectral domain GMM into the frequency domain using minimal Kullback-Leiber (KL)-divergency criterion. The frequency domain Laplace method computes the maximum a posteriori (MAP) estimator for the spectral amplitude. Correspondingly, the log-spectral domain Laplace method computes the MAP estimator for the log-spectral amplitude. Further, the gain and noise spectrum adaptation are implemented using the expectation-maximization (EM) algorithm within the GMM under Gaussian approximation. The proposed algorithms are evaluated by applying them to enhance the speeches corrupted by the speech-shaped noise (SSN). The experimental results demonstrate that the proposed algorithms offer improved signal-to-noise ratio, lower word recognition error rate, and less spectral distortion.