Data reduction by randomization subsampling for the study of large hyperspectral datasets.

Large amount of information in hyperspectral images (HSI) generally makes their analysis (e.g., principal component analysis, PCA) time consuming and often requires a lot of random access memory (RAM) and high computing power. This is particularly problematic for analysis of large images, containing millions of pixels, which can be created by augmenting series of single images (e.g., in time series analysis). This tutorial explores how data reduction can be used to analyze time series hyperspectral images much faster without losing crucial analytical information. Two of the most common data reduction methods have been chosen from the recent research. The first one uses a simple randomization method called randomized sub-sampling PCA (RSPCA). The second implies a more robust randomization method based on local-rank approximations (rPCA). This manuscript exposes the major benefits and drawbacks of both methods with the spirit of being as didactical as possible for a reader. A comprehensive comparison is made considering the amount of information retained by the PCA models at different compression degrees and the performance time. Extrapolation is also made to the case where the effect of time and any other factor are to be studied simultaneously.

Subsampled Directed-Percolation Models Explain Scaling Relations Experimentally Observed in the Brain.

Recent experimental results on spike avalanches measured in the urethane-anesthetized rat cortex have revealed scaling relations that indicate a phase transition at a specific level of cortical firing rate variability. The scaling relations point to critical exponents whose values differ from those of a branching process, which has been the canonical model employed to understand brain criticality. This suggested that a different model, with a different phase transition, might be required to explain the data. Here we show that this is not necessarily the case. By employing two different models belonging to the same universality class as the branching process (mean-field directed percolation) and treating the simulation data exactly like experimental data, we reproduce most of the experimental results. We find that subsampling the model and adjusting the time bin used to define avalanches (as done with experimental data) are sufficient ingredients to change the apparent exponents of the critical point. Moreover, experimental data is only reproduced within a very narrow range in parameter space around the phase transition.

Choice of field and laboratory methods affects the detection of anthropogenic disturbances using stream macroinvertebrate assemblages.

Accurate and precise detection of anthropogenic impacts on stream ecosystems using macroinvertebrates as biological indicators depends on the use of appropriate field and laboratory methods. We assessed the responsiveness to anthropogenic disturbances of assemblage metrics and composition by comparing commonly employed alternative combinations of field sampling and individuals counting methods. Four datasets were derived by, in the field 1) conducting multihabitat sampling (MH) or 2) targeting samples in leaf packs (single-habitat sampling - SH) and, in the laboratory A) counting all individuals of the samples, or B) simulating subsampling of 300 individuals per sample. We collected our data from 39 headwater stream sites in a drainage basin located in the Brazilian Cerrado. We used a previously published quantitative integrated disturbance index (IDI), based on both local and catchment disturbance measurements, to characterize the intensity of anthropogenic alterations at each site. Family richness and % Ephemeroptera, Plecoptera and Trichoptera (% EPT) individuals obtained from each dataset were tested against the IDI through simple linear regressions, and the differences in assemblage composition between least- and most-disturbed sites was tested using Permutational Multivariate Analysis of Variance (PERMANOVA). When counting all individuals, differences in taxonomic richness and assemblage composition of macroinvertebrate assemblages between least- and most-disturbed sites were more pronounced in the MH than in the SH sampling method. Leaf packs seemed to concentrate high abundance and diversity of macroinvertebrates in highly disturbed sites, acting as 'biodiversity hotbeds' in these situations, which likely reduced the response of the assemblages to the disturbance gradient when this substrate was targeted. However, MH sampling produced weaker results than SH when subsampling was performed. The % EPT individuals responded better to the disturbance gradient when SH was employed, and its efficiency was not affected by the subsampling procedure. We conclude that no single method was the best in all situations, and the efficiency of a sampling protocol depends on the combination of field and laboratory methods being used. Although the total count of individuals with multihabitat sampling obtained the best results for most of the evaluated variables, the decision of which procedures to use depends on the amount of time and resources available, on the variables of interest, on the availability of habitat types in the sites sampled, and on the other methods being employed in the sampling protocol.

Novel Data Mining Methodology for Healthcare Applied to a New Model to Diagnose Metabolic Syndrome without a Blood Test.

Metabolic Syndrome (MetS) is a cluster of risk factors that increase the likelihood of heart disease and diabetes mellitus. It is crucial to get diagnosed with time to take preventive measures, especially for patients in locations without proper access to laboratories and medical consultations. This work presented a new methodology to diagnose diseases using data mining that documents all the phases thoroughly for further improvement of the resulting models. We used the methodology to create a new model to diagnose the syndrome without using biochemical variables. We compared similar classification models, using their reported variables and previously obtained data from a study in Colombia. We built a new model and compared it to previous models using the holdout, and random subsampling validation methods to get performance evaluation indicators between the models. Our resulting ANN model used three hidden layers and only Hip Circumference, dichotomous Waist Circumference, and dichotomous blood pressure variables. It gave an Area Under Curve (AUC) of 87.75% by the IDF and 85.12% by HMS MetS diagnosis criteria, higher than previous models. Thanks to our new methodology, diagnosis models can be thoroughly documented for appropriate future comparisons, thus benefiting the diagnosis of the studied diseases.

Estimation of a Difference in Finite Populations with Missing Observations.

Sampling strategies for the difference are constructed when missing observations are present. Two different situations are analyzed. One of them is related with a non random device settled by the statistician for reducing costs. The other is a non response problem. An unbiased minimum variance estimator is obtained in the first case and an approximation to it is deduced. The unbiased estimation in the second is associated with subsampling tactics.