Evaluation of the Use of P Values in Neurosurgical Literature: from Statistical Significance to Clinical Irrelevance.

The application and interpretation of P values have caused debate for several decades, and this debate has become particularly relevant in the past few years. The P value represents the probability of seeing results as extreme or more extreme than those observed in a data analysis, were the null hypothesis and other underlying assumptions to be true. While P values are useful in pointing out where an effect may be present, they have often been misused in an attempt to oversell "statistically significant" findings. As P values rely on the spread and number of measurements, a smaller P value does not necessarily imply a larger effect size, which is better assessed via an effect estimate and confidence interval interpreted in the context of the study. The clinical relevance of a computed P value is context dependent. We investigated the current use of P values in a small sample of recent neurosurgical literature. Only a minority of manuscripts that reported statistical significance described confounder adjustment, or effect sizes. A common, incorrect assumption often observed was that statistical significance equals clinical relevance. To enable correct interpretation of clinical significance, it is crucial that authors describe the clinical implications of their findings.

An integrative multivariate approach for predicting functional recovery using magnetic resonance imaging parameters in a translational pig ischemic stroke model.

Magnetic resonance imaging (MRI) is a clinically relevant, real-time imaging modality that is frequently utilized to assess stroke type and severity. However, specific MRI biomarkers that can be used to predict long-term functional recovery are still a critical need. Consequently, the present study sought to examine the prognostic value of commonly utilized MRI parameters to predict functional outcomes in a porcine model of ischemic stroke. Stroke was induced via permanent middle cerebral artery occlusion. At 24 hours post-stroke, MRI analysis revealed focal ischemic lesions, decreased diffusivity, hemispheric swelling, and white matter degradation. Functional deficits including behavioral abnormalities in open field and novel object exploration as well as spatiotemporal gait impairments were observed at 4 weeks post-stroke. Gaussian graphical models identified specific MRI outputs and functional recovery variables, including white matter integrity and gait performance, that exhibited strong conditional dependencies. Canonical correlation analysis revealed a prognostic relationship between lesion volume and white matter integrity and novel object exploration and gait performance. Consequently, these analyses may also have the potential of predicting patient recovery at chronic time points as pigs and humans share many anatomical similarities (e.g., white matter composition) that have proven to be critical in ischemic stroke pathophysiology. The study was approved by the University of Georgia (UGA) Institutional Animal Care and Use Committee (IACUC; Protocol Number: A2014-07-021-Y3-A11 and 2018-01-029-Y1-A5) on November 22, 2017.

Finding common task-related regions in fMRI data from multiple subjects by periodogram clustering and clustering ensemble.

We propose an innovative and practically relevant clustering method to find common task-related brain regions among different subjects who respond to the same set of stimuli. Using functional magnetic resonance imaging (fMRI) time series data, we first cluster the voxels within each subject on a voxel by voxel basis. To extract signals out of noisy data, we estimate a new periodogram at each voxel using multi-tapering and low-rank spline smoothing and then use the periodogram as the main feature for clustering. We apply a divisive hierarchical clustering algorithm to the estimated periodograms within a single subject and identify the task-related region as the cluster of voxels that have periodograms with a peak frequency matching that of the stimulus sequence. Finally, we apply a machine learning technique called clustering ensemble to find common task-related regions across different subjects. The efficacy of the proposed approach is illustrated via a simulation study and a real fMRI data set. Copyright © 2016 John Wiley & Sons, Ltd.

A Meta-Analysis of fMRI Activation Differences during Episodic Memory in Alzheimer's Disease and Mild Cognitive Impairment.

Functional MRI (fMRI) has the potential to be used as a tool to detect biomarkers related to classifying Alzheimer's disease (AD) and its prodromal stage, mild cognitive impairment (MCI). Previous meta-analyses suggest that during episodic memory tasks, MCI patients exhibit hyperactivation in the medial temporal lobe (MTL) while AD patients exhibit hypoactivation, compared to healthy older adults (HOAs). However, these previous studies have methodological weaknesses that limit the generalizability of the results. This quantitative meta-analysis re-examines the activation associated with episodic memory in AD and MCI as compared to cognitively normal populations to assess these commonly cited activation differences. A whole-brain activation likelihood estimation based meta-analysis was conducted on fMRI studies that examined episodic memory in HOA (n = 200), MCI (n = 131), and AD populations (n = 89; total n = 409). Diffuse activation was exhibited in the HOA sample, while activation was more limited in the clinical populations. Additionally, the HOA sample showed more activation in the right hippocampus compared to the AD sample. The MCI studies showed greater activation in the cerebellum compared to the HOA sample, potentially indicating a compensatory mechanism for verbal encoding. MTL hypoactivation in the AD sample is consistent with previous studies, but more evidence of MCI hyperactivation is needed before considering MTL activation as an early biomarker for the AD disease process.

Nonparametric variogram modeling with hole effect structure in analyzing the spatial characteristics of fMRI data.

When analyzing functional neuroimaging data, it is particularly important to consider the spatial structure of the brain. Some researchers have applied geostatistical methods in the analysis of functional magnetic resonance imaging (fMRI) data to enhance the detection of activated brain regions. In this paper, we propose a nonparametric variogram model for the complicated spatial characteristics of fMRI data. The new periodic variogram model can well describe the fluctuating spatial structure of fMRI data, considering both the nonlinear physical relationship between the proximate voxels and the functional relationship between distant voxels. We demonstrate the effectiveness of the new variogram model using fMRI data from a saccade study.

Incorporating spatial dependence into Bayesian multiple testing of statistical parametric maps in functional neuroimaging.

The analysis of functional neuroimaging data often involves the simultaneous testing for activation at thousands of voxels, leading to a massive multiple testing problem. This is true whether the data analyzed are time courses observed at each voxel or a collection of summary statistics such as statistical parametric maps (SPMs). It is known that classical multiplicity corrections become strongly conservative in the presence of a massive number of tests. Some more popular approaches for thresholding imaging data, such as the Benjamini-Hochberg step-up procedure for false discovery rate control, tend to lose precision or power when the assumption of independence of the data does not hold. Bayesian approaches to large scale simultaneous inference also often rely on the assumption of independence. We introduce a spatial dependence structure into a Bayesian testing model for the analysis of SPMs. By using SPMs rather than the voxel time courses, much of the computational burden of Bayesian analysis is mitigated. Increased power is demonstrated by using the dependence model to draw inference on a real dataset collected in a fMRI study of cognitive control. The model also is shown to lead to improved identification of neural activation patterns known to be associated with eye movement tasks.

Computing Critical Values of Exact Tests by Incorporating Monte Carlo Simulations Combined with Statistical Tables.

Various exact tests for statistical inference are available for powerful and accurate decision rules provided that corresponding critical values are tabulated or evaluated via Monte Carlo methods. This article introduces a novel hybrid method for computing p-values of exact tests by combining Monte Carlo simulations and statistical tables generated a priori. To use the data from Monte Carlo generations and tabulated critical values jointly, we employ kernel density estimation within Bayesian-type procedures. The p-values are linked to the posterior means of quantiles. In this framework, we present relevant information from the Monte Carlo experiments via likelihood-type functions, whereas tabulated critical values are used to reflect prior distributions. The local maximum likelihood technique is employed to compute functional forms of prior distributions from statistical tables. Empirical likelihood functions are proposed to replace parametric likelihood functions within the structure of the posterior mean calculations to provide a Bayesian-type procedure with a distribution-free set of assumptions. We derive the asymptotic properties of the proposed nonparametric posterior means of quantiles process. Using the theoretical propositions, we calculate the minimum number of needed Monte Carlo resamples for desired level of accuracy on the basis of distances between actual data characteristics (e.g. sample sizes) and characteristics of data used to present corresponding critical values in a table. The proposed approach makes practical applications of exact tests simple and rapid. Implementations of the proposed technique are easily carried out via the recently developed STATA and R statistical packages.

Practice-related changes in neural activation patterns investigated via wavelet-based clustering analysis.

OBJECTIVES: To evaluate brain activation using functional magnetic resonance imaging (fMRI) and specifically, activation changes across time associated with practice-related cognitive control during eye movement tasks. EXPERIMENTAL DESIGN: Participants were engaged in antisaccade performance (generating a glance away from a cue) while fMR images were acquired during two separate test sessions: (1) at pre-test before any exposure to the task and (2) at post-test, after 1 week of daily practice on antisaccades, prosaccades (glancing toward a target), or fixation (maintaining gaze on a target). PRINCIPAL OBSERVATIONS: The three practice groups were compared across the two test sessions, and analyses were conducted via the application of a model-free clustering technique based on wavelet analysis. This series of procedures was developed to avoid analysis problems inherent in fMRI data and was composed of several steps: detrending, data aggregation, wavelet transform and thresholding, no trend test, principal component analysis (PCA), and K-means clustering. The main clustering algorithm was built in the wavelet domain to account for temporal correlation. We applied a no trend test based on wavelets to significantly reduce the high dimension of the data. We clustered the thresholded wavelet coefficients of the remaining voxels using PCA K-means clustering. CONCLUSION: Over the series of analyses, we found that the antisaccade practice group was the only group to show decreased activation from pre-test to post-test in saccadic circuitry, particularly evident in supplementary eye field, frontal eye fields, superior parietal lobe, and cuneus.

Bootstrapping GEE models for fMRI regional connectivity.

An Alzheimer's fMRI study has motivated us to evaluate inter-regional correlations during rest between groups. We apply generalized estimating equation (GEE) models to test for differences in regional correlations across groups. Both the GEE marginal model and GEE transition model are evaluated and compared to the standard pooling Fisher-z approach using simulation studies. Standard errors of all methods are estimated both theoretically (model-based) and empirically (bootstrap). Of all the methods, we find that the transition models have the best statistical properties. Overall, the model-based standard errors and bootstrap standard errors perform about the same. We also demonstrate the methods with a functional connectivity study in a healthy cognitively normal population of ApoE4+ participants and ApoE4- participants who are recruited from the Adult Children's Study conducted at the Washington University Knight Alzheimer's Disease Research Center.

Sparse geostatistical analysis in clustering fMRI time series.

Clustering is used in fMRI time series data analysis to find the active regions in the brain related to a stimulus. However, clustering algorithms usually do not work well for ill-balanced data, i.e., when only a small proportion of the voxels in the brain respond to the stimulus. This is the typical situation in fMRI--most voxels do not, in fact, respond to the specific task. We propose a new method of sparse geostatistical analysis in clustering, which first uses sparse principal component analysis (SPCA) to perform data reduction, followed by geostatistical clustering. The proposed method is model-free and data-driven; in particular it does not require prior knowledge of the hemodynamic response function, nor of the experimental paradigm. Our data analysis shows that the spatial and temporal structures of the task-related activation produced by our new approach are more stable compared with other methods (e.g., GLM analysis with geostatistical clustering). Sparse geostatistical analysis appears to be a promising tool for exploratory clustering of fMRI time series.

A generalized estimating equations approach for resting-state functional MRI group analysis.

An Alzheimer's fMRI study has motivated us to evaluate inter-regional correlations between groups. The overall objective is to assess inter-regional correlations at a resting-state with no stimulus or task. We propose using a generalized estimating equation (GEE) transition model and a GEE marginal model to model the within-subject correlation for each region. Residuals calculated from the GEE models are used to correlate brain regions and assess between group differences. The standard pooling approach of group averages of the Fisher-z transformation assuming temporal independence is a typical approach used to compare group correlations. The GEE approaches and standard Fisher-z pooling approach are demonstrated with an Alzheimer's disease (AD) connectivity study in a population of AD subjects and healthy control subjects. We also compare these methods using simulation studies and show that the transition model may have better statistical properties.

A multiscale analysis of the temporal characteristics of resting-state fMRI data.

In this paper, we conduct an investigation of the null hypothesis distribution for functional magnetic resonance imaging (fMRI) time series using multiscale analysis tools, SiZer (significance of zero crossings of the derivative) and wavelets. Most current approaches to the analysis of fMRI data assume simple models for temporal (short term or long term) dependence structure. Such simplifications are to some extent necessary due to the complex, high-dimensional nature of the data, but to date there have been few systematic studies of the dependence structures under a range of possible null hypotheses, using data sets gathered specifically for that purpose. We aim to address some of these issues by analyzing the detrended data with a long enough time horizon to study possible long-range temporal dependence. Our multiscale approach shows that even for resting-state data, data, i.e. "null" or ambient thought, some voxel time series cannot be modeled by white noise and need long-range dependent type error structure. This finding suggests the use of different time series models in different parts of the brain in fMRI studies.

Geostatistical analysis in clustering fMRI time series.

Clustering of functional magnetic resonance imaging (fMRI) time series--either directly or through characteristic features such as the cross-correlation with the experimental protocol signal--has been extensively used for the identification of active regions in the brain. Both approaches have drawbacks; clustering of the time series themselves may identify voxels with similar temporal behavior that is unrelated to the stimulus, whereas cross-correlation requires knowledge of the stimulus presentation protocol. In this paper we propose the use of autocorrelation structure instead--an idea borrowed from geostatistics; this approach does not suffer from the deficits associated with previous clustering methods. We first formalize the traditional classification methods as three steps: feature extraction, choice of classification metric and choice of classification algorithm. The use of different characteristics to effect the clustering (cross-correlation, autocorrelation, and so forth) relates to the first of these three steps. We then demonstrate the efficacy of autocorrelation clustering on a simple visual task and on resting data. A byproduct of our analysis is the finding that masking prior to clustering, as is commonly done, may degrade the quality of the discovered clusters, and we offer an explanation for this phenomenon.

Discussion of "Puzzlingly High Correlations in fMRI Studies of Emotion, Personality, and Social Cognition" by Vul et al. (2009).

In their article, Vul, Harris, Winkielman, and Pashler (2009), (this issue) raise the issue of nonindependent analysis in behavioral neuroimaging, whereby correlations are artificially inflated as a result of spurious statistical procedures. In this comment, I note that the phenomenon in question is a type of selection bias and hence is neither new nor unique to fMRI. The use of massive, complex data sets (common in modern applications) to answer increasingly intricate scientific questions presents many potential pitfalls to valid statistical analysis. Strong collaboration between statisticians and scientists and the development of statistical methods specific to the types of data encountered in practice can help researchers avoid these pitfalls.

Hypothesis testing, power and sample size determination for between group comparisons in fMRI experiments.

Modern methods for imaging the human brain, such as functional magnetic resonance imaging (fMRI) present a range of challenging statistical problems. In this paper, we first develop a large sample based test for between group comparisons and use it to determine the necessary sample size in order to obtain a target power via simulation under various alternatives for a given pre-specified significance level. Both testing and sample size calculations are particularly critical for neuroscientists who use these new techniques, since each subject is expensive to image.

Estimation and Classification of BOLD Responses Over Multiple Trials.

In this article, we model functional magnetic resonance imaging (fMRI) data for event-related experiment data using a fourth degree spline to fit voxel specific blood oxygenation level-dependent (BOLD) responses. The data are preprocessed for removing long term temporal components such as drifts using wavelet approximations. The spatial dependence is incorporated in the data by the application of 3D Gaussian spatial filter. The methodology assigns an activation score to each trial based on the voxel specific characteristics of the response curve. The proposed procedure has a capability of being fully automated and it produces activation images based on overall scores assigned to each voxel. The methodology is illustrated on real data from an event-related design experiment of visually guided saccades (VGS).

Maturation of cognitive processes from late childhood to adulthood.

To characterize cognitive maturation through adolescence, processing speed, voluntary response suppression, and spatial working memory were measured in 8- to 30-year-old (N = 245) healthy participants using oculomotor tasks. Development progressed with a steep initial improvement in performance followed by stabilization in adolescence. Adult-level mature performance began at approximately 15, 14, and 19 years of age for processing speed, response inhibition, and working memory, respectively. Although processes developed independently, processing speed influenced the development of working memory whereas the development of response suppression and working memory were interdependent. These results indicate that processing speed, voluntary response suppression, and working memory mature through late childhood and into adolescence. How brain maturation specific to adolescence may support cognitive maturation is discussed.

Estimation and classification of fMRI hemodynamic response patterns.

In this paper, we propose an approach to modeling functional magnetic resonance imaging (fMRI) data that combines hierarchical polynomial models, Bayes estimation, and clustering. A cubic polynomial is used to fit the voxel time courses of event-related design experiments. The coefficients of the polynomials are estimated by Bayes estimation, in a two-level hierarchical model, which allows us to borrow strength from all voxels. The voxel-specific Bayes polynomial coefficients are then transformed to the times and magnitudes of the minimum and maximum points on the hemodynamic response curve, which are in turn used to classify the voxels as being activated or not. The procedure is demonstrated on real data from an event-related design experiment of visually guided saccades and shown to be an effective alternative to existing methods.

Assessing the sensitivity of fMRI group maps.

Group maps created from individual functional maps provide useful summaries of patterns of brain activation. Different methods for combining information have been proposed in the statistical literature and have been recently applied to fMRI data. Since these group maps are statistics, it is natural to ask how robust they are, that is, are they sensitive to the effects of unusual subjects? "Unusual" might be in terms of extent, location, or strength of activation. Our approach in this paper is to jackknife group maps formed by different combining procedures; the jackknife method, which involves deleting each observation (subject) in turn and recalculating the statistic (the group map), is commonly used for the purpose of assessing sensitivity. We examine the theoretical properties of four combining methods. In addition, via a collection of measures defined on the difference between group maps based on the entire sample and based on the jackknifed samples, we evaluate the robustness of these same methods on data from an fMRI experiment. Results indicate that there is a type of tradeoff in the combining techniques we consider, between robustness and conservativeness: methods that are liberal, in that they allow for the discovery of many active voxels, tend also to be more sensitive to the influences of individual subjects.

Combining brains: a survey of methods for statistical pooling of information.

More than one subject is scanned in a typical functional brain imaging experiment. How can the scientist make best use of the acquired data to map the specific areas of the brain that become active during the performance of different tasks? It is clear that we can gain both scientific and statistical power by pooling the images from multiple subjects; furthermore, for the comparison of groups of subjects (clinical patients vs healthy controls, children of different ages, left-handed people vs right-handed people, as just some examples), it is essential to have a "group map" to represent each population and to form the basis of a statistical test. While the importance of combining images for these purposes has been recognized, there has not been an organized attempt on the part of neuroscientists to understand the different statistical approaches to this problem, which have various strengths and weaknesses. In this paper we review some popular methods for combining information, and demonstrate the surveyed techniques on a sample data set. Given a combination of brain images, the researcher needs to interpret the result and decide on areas of activation; the question of thresholding is critical here and is also explored.