EVM: Incorporating Model Checking into Exploratory Visual Analysis.

Visual analytics (VA) tools support data exploration by helping analysts quickly and iteratively generate views of data which reveal interesting patterns. However, these tools seldom enable explicit checks of the resulting interpretations of data-e.g., whether patterns can be accounted for by a model that implies a particular structure in the relationships between variables. We present EVM, a data exploration tool that enables users to express and check provisional interpretations of data in the form of statistical models. EVM integrates support for visualization-based model checks by rendering distributions of model predictions alongside user-generated views of data. In a user study with data scientists practicing in the private and public sector, we evaluate how model checks influence analysts' thinking during data exploration. Our analysis characterizes how participants use model checks to scrutinize expectations about data generating process and surfaces further opportunities to scaffold model exploration in VA tools.

Causal Support: Modeling Causal Inferences with Visualizations.

Analysts often make visual causal inferences about possible data-generating models. However, visual analytics (VA) software tends to leave these models implicit in the mind of the analyst, which casts doubt on the statistical validity of informal visual "insights". We formally evaluate the quality of causal inferences from visualizations by adopting causal support-a Bayesian cognition model that learns the probability of alternative causal explanations given some data-as a normative benchmark for causal inferences. We contribute two experiments assessing how well crowdworkers can detect (1) a treatment effect and (2) a confounding relationship. We find that chart users' causal inferences tend to be insensitive to sample size such that they deviate from our normative benchmark. While interactively cross-filtering data in visualizations can improve sensitivity, on average users do not perform reliably better with common visualizations than they do with textual contingency tables. These experiments demonstrate the utility of causal support as an evaluation framework for inferences in VA and point to opportunities to make analysts' mental models more explicit in VA software.

Visual Reasoning Strategies for Effect Size Judgments and Decisions.

Uncertainty visualizations often emphasize point estimates to support magnitude estimates or decisions through visual comparison. However, when design choices emphasize means, users may overlook uncertainty information and misinterpret visual distance as a proxy for effect size. We present findings from a mixed design experiment on Mechanical Turk which tests eight uncertainty visualization designs: 95% containment intervals, hypothetical outcome plots, densities, and quantile dotplots, each with and without means added. We find that adding means to uncertainty visualizations has small biasing effects on both magnitude estimation and decision-making, consistent with discounting uncertainty. We also see that visualization designs that support the least biased effect size estimation do not support the best decision-making, suggesting that a chart user's sense of effect size may not necessarily be identical when they use the same information for different tasks. In a qualitative analysis of users' strategy descriptions, we find that many users switch strategies and do not employ an optimal strategy when one exists. Uncertainty visualizations which are optimally designed in theory may not be the most effective in practice because of the ways that users satisfice with heuristics, suggesting opportunities to better understand visualization effectiveness by modeling sets of potential strategies.

Boba: Authoring and Visualizing Multiverse Analyses.

Multiverse analysis is an approach to data analysis in which all "reasonable" analytic decisions are evaluated in parallel and interpreted collectively, in order to foster robustness and transparency. However, specifying a multiverse is demanding because analysts must manage myriad variants from a cross-product of analytic decisions, and the results require nuanced interpretation. We contribute Baba: an integrated domain-specific language (DSL) and visual analysis system for authoring and reviewing multiverse analyses. With the Boba DSL, analysts write the shared portion of analysis code only once, alongside local variations defining alternative decisions, from which the compiler generates a multiplex of scripts representing all possible analysis paths. The Boba Visualizer provides linked views of model results and the multiverse decision space to enable rapid, systematic assessment of consequential decisions and robustness, including sampling uncertainty and model fit. We demonstrate Boba's utility through two data analysis case studies, and reflect on challenges and design opportunities for multiverse analysis software.

The time course of different surround suppression mechanisms.

What we see depends on the spatial context in which it appears. Previous work has linked the suppression of perceived contrast by surrounding stimuli to reduced neural responses in early visual cortex. This surround suppression depends on at least two separable neural mechanisms, "low-level" and "higher level," which can be differentiated by their response characteristics. We used electroencephalography to demonstrate for the first time that human occipital neural responses show evidence of these two suppression mechanisms. Eighteen adults (10 women, 8 men) each participated in three experimental sessions, in which they viewed visual stimuli through a mirror stereoscope. The first session was used to identify the C1 component, while the second and third comprised the main experiment. Event-related potentials were measured in response to center gratings either with no surround or with surrounding gratings oriented parallel or orthogonal, and presented in either the same eye (monoptic) or the opposite eye (dichoptic). We found that the earliest component of an event-related potential (C1; ∼60 ms) was suppressed by surrounding stimuli, but that suppression did not depend on surround configuration. This suggests a suppression mechanism that is not tuned for relative orientation acting on the earliest cortical response to the target. A later response component (N1; ∼160 ms) showed stronger suppression for parallel and monoptic surrounds, consistent with our earlier psychophysical results and a second form of suppression that is binocular and orientation tuned. We conclude that these two forms of surround suppression have distinct response time courses in the human visual system, which can be differentiated using electrophysiology.

Glutamatergic facilitation of neural responses in MT enhances motion perception in humans.

There is large individual variability in human neural responses and perceptual abilities. The factors that give rise to these individual differences, however, remain largely unknown. To examine these factors, we measured fMRI responses to moving gratings in the motion-selective region MT, and perceptual duration thresholds for motion direction discrimination. Further, we acquired MR spectroscopy data, which allowed us to quantify an index of neurotransmitter levels in the region of area MT. These three measurements were conducted in separate experimental sessions within the same group of male and female subjects. We show that stronger Glx (glutamate + glutamine) signals in the MT region are associated with both higher fMRI responses and superior psychophysical task performance. Our results suggest that greater baseline levels of glutamate within MT facilitate motion perception by increasing neural responses in this region.

In Pursuit of Error: A Survey of Uncertainty Visualization Evaluation.

Understanding and accounting for uncertainty is critical to effectively reasoning about visualized data. However, evaluating the impact of an uncertainty visualization is complex due to the difficulties that people have interpreting uncertainty and the challenge of defining correct behavior with uncertainty information. Currently, evaluators of uncertainty visualization must rely on general purpose visualization evaluation frameworks which can be ill-equipped to provide guidance with the unique difficulties of assessing judgments under uncertainty. To help evaluators navigate these complexities, we present a taxonomy for characterizing decisions made in designing an evaluation of an uncertainty visualization. Our taxonomy differentiates six levels of decisions that comprise an uncertainty visualization evaluation: the behavioral targets of the study, expected effects from an uncertainty visualization, evaluation goals, measures, elicitation techniques, and analysis approaches. Applying our taxonomy to 86 user studies of uncertainty visualizations, we find that existing evaluation practice, particularly in visualization research, focuses on Performance and Satisfaction-based measures that assume more predictable and statistically-driven judgment behavior than is suggested by research on human judgment and decision making. We reflect on common themes in evaluation practice concerning the interpretation and semantics of uncertainty, the use of confidence reporting, and a bias toward evaluating performance as accuracy rather than decision quality. We conclude with a concrete set of recommendations for evaluators designed to reduce the mismatch between the conceptualization of uncertainty in visualization versus other fields.

Sex Differences in Visual Motion Processing.

The importance of sex as a biological variable has recently been emphasized by major funding organizations [1] and within the neuroscience community [2]. Critical sex-based neural differences are indicated by, for example, conditions such as autism spectrum disorder (ASD) that have a strong sex bias with a higher prevalence among males [51, 3]. Motivated by this broader context, we report a marked sex difference in a visual motion perception task among neurotypical adults. Motion duration thresholds [4, 5]-the minimum duration needed to accurately perceive motion direction-were considerably shorter for males than females. We replicated this result across three laboratories and 263 total participants. This type of enhanced performance has previously been observed only in special populations including ASD, depression, and senescence [6-8]. The observed sex difference cannot be explained by general differences in speed of visual processing, overall visual discrimination abilities, or potential motor-related differences. We also show that while individual differences in motion duration thresholds are associated with differences in fMRI responsiveness of human MT+, surprisingly, MT+ response magnitudes did not differ between males and females. Thus, we reason that sex differences in motion perception are not captured by an MT+ fMRI measure that predicts within-sex individual differences in perception. Overall, these results show how sex differences can manifest unexpectedly, highlighting the importance of sex as a factor in the design and analysis of perceptual and cognitive studies.

Hypothetical Outcome Plots Help Untrained Observers Judge Trends in Ambiguous Data.

Animated representations of outcomes drawn from distributions (hypothetical outcome plots, or HOPs) are used in the media and other public venues to communicate uncertainty. HOPs greatly improve multivariate probability estimation over conventional static uncertainty visualizations and leverage the ability of the visual system to quickly, accurately, and automatically process the summary statistical properties of ensembles. However, it is unclear how well HOPs support applied tasks resembling real world judgments posed in uncertainty communication. We identify and motivate an appropriate task to investigate realistic judgments of uncertainty in the public domain through a qualitative analysis of uncertainty visualizations in the news. We contribute two crowdsourced experiments comparing the effectiveness of HOPs, error bars, and line ensembles for supporting perceptual decision-making from visualized uncertainty. Participants infer which of two possible underlying trends is more likely to have produced a sample of time series data by referencing uncertainty visualizations which depict the two trends with variability due to sampling error. By modeling each participant's accuracy as a function of the level of evidence presented over many repeated judgments, we find that observers are able to correctly infer the underlying trend in samples conveying a lower level of evidence when using HOPs rather than static aggregate uncertainty visualizations as a decision aid. Modeling approaches like ours contribute theoretically grounded and richly descriptive accounts of user perceptions to visualization evaluation.

Luminance-dependent long-term chromatic adaptation.

There is theoretical and empirical support for long-term adaptation of human vision to chromatic regularities in the environment. The current study investigates whether relationships of luminance and chromaticity in the natural environment could drive chromatic adaptation independently and differently for bright and dark colors. This is motivated by psychophysical evidence of systematic difference shifts in red-green chromatic sensitivities between contextually bright- versus dark-colored stimuli. For some broad classes of scene content, consistent shifts in chromaticity are found between high and low light levels within images. Especially in those images in which sky and terrain are juxtaposed, this shift has direction and magnitude consistent with the observed psychophysical shifts in the red-green balance between bright and dark colors. Taken together, these findings suggest that relative weighting of M- and L-cone signals could be adapted, in a luminance-dependent fashion, to regularities in the natural environment.