Are We Closing the Loop Yet? Gaps in the Generalizability of VIS4ML Research.

Visualization for machine learning (VIS4ML) research aims to help experts apply their prior knowledge to develop, understand, and improve the performance of machine learning models. In conceiving VIS4ML systems, researchers characterize the nature of human knowledge to support human-in-the-loop tasks, design interactive visualizations to make ML components interpretable and elicit knowledge, and evaluate the effectiveness of human-model interchange. We survey recent VIS4ML papers to assess the generalizability of research contributions and claims in enabling human-in-the-loop ML. Our results show potential gaps between the current scope of VIS4ML research and aspirations for its use in practice. We find that while papers motivate that VIS4ML systems are applicable beyond the specific conditions studied, conclusions are often overfitted to non-representative scenarios, are based on interactions with a small set of ML experts and well-understood datasets, fail to acknowledge crucial dependencies, and hinge on decisions that lack justification. We discuss approaches to close the gap between aspirations and research claims and suggest documentation practices to report generality constraints that better acknowledge the exploratory nature of VIS4ML research.

How Do Viewers Synthesize Conflicting Information from Data Visualizations?

Scientific knowledge develops through cumulative discoveries that build on, contradict, contextualize, or correct prior findings. Scientists and journalists often communicate these incremental findings to lay people through visualizations and text (e.g., the positive and negative effects of caffeine intake). Consequently, readers need to integrate diverse and contrasting evidence from multiple sources to form opinions or make decisions. However, the underlying mechanism for synthesizing information from multiple visualizations remains under-explored. To address this knowledge gap, we conducted a series of four experiments ( N=1166) in which participants synthesized empirical evidence from a pair of line charts presented sequentially. In Experiment 1, we administered a baseline condition with charts depicting no specific context where participants held no strong belief. To test for the generalizability, we introduced real-world scenarios to our visualizations in Experiment 2 and added accompanying text descriptions similar to online news articles or blog posts in Experiment 3. In all three experiments, we varied the relative direction and magnitude of line slopes within the chart pairs. We found that participants tended to weigh the positive slope more when the two charts depicted relationships in the opposite direction (e.g., one positive slope and one negative slope). Participants tended to weigh the less steep slope more when the two charts depicted relationships in the same direction (e.g., both positive). Through these experiments, we characterize participants' synthesis behaviors depending on the relationship between the information they viewed, contribute to theories describing underlying cognitive mechanisms in information synthesis, and describe design implications for data storytelling.

Evaluating longitudinal relationships between parental monitoring and substance use in a multi-year, intensive longitudinal study of 670 adolescent twins.

Introduction: Parental monitoring is a key intervention target for adolescent substance use, however this practice is largely supported by causally uninformative cross-sectional or sparse-longitudinal observational research designs. Methods: We therefore evaluated relationships between adolescent substance use (assessed weekly) and parental monitoring (assessed every two months) in 670 adolescent twins for two years. This allowed us to assess how individual-level parental monitoring and substance use trajectories were related and, via the twin design, to quantify genetic and environmental contributions to these relationships. Furthermore, we attempted to devise additional measures of parental monitoring by collecting quasi-continuous GPS locations and calculating a) time spent at home between midnight and 5am and b) time spent at school between 8am-3pm. Results: ACE-decomposed latent growth models found alcohol and cannabis use increased with age while parental monitoring, time at home, and time at school decreased. Baseline alcohol and cannabis use were correlated (r = .65) and associated with baseline parental monitoring (r = -.24 to -.29) but not with baseline GPS measures (r = -.06 to -.16). Longitudinally, changes in substance use and parental monitoring were not significantly correlated. Geospatial measures were largely unrelated to parental monitoring, though changes in cannabis use and time at home were highly correlated (r = -.53 to -.90), with genetic correlations suggesting their relationship was substantially genetically mediated. Due to power constraints, ACE estimates and biometric correlations were imprecisely estimated. Most of the substance use and parental monitoring phenotypes were substantially heritable, but genetic correlations between them were not significantly different from 0. Discussion: Overall, we found developmental changes in each phenotype, baseline correlations between substance use and parental monitoring, co-occurring changes and mutual genetic influences for time at home and cannabis use, and substantial genetic influences on many substance use and parental monitoring phenotypes. However, our geospatial variables were mostly unrelated to parental monitoring, suggesting they poorly measured this construct. Furthermore, though we did not detect evidence of genetic confounding, changes in parental monitoring and substance use were not significantly correlated, suggesting that, at least in community samples of mid-to-late adolescents, the two may not be causally related.

SmartCues: A Multitouch Query Approach for Details-on-Demand through Dynamically Computed Overlays.

Details-on-demand is a crucial feature in the visual information-seeking process but is often only implemented in highly constrained settings. The most common solution, hover queries (i.e., tooltips), are fast and expressive but are usually limited to single mark (e.g., a bar in a bar chart). 'Queries' to retrieve details for more complex sets of objects (e.g., comparisons between pairs of elements, averages across multiple items, trend lines, etc.) are difficult for end-users to invoke explicitly. Further, the output of these queries require complex annotations and overlays which need to be displayed and dismissed on demand to avoid clutter. In this work we introduce SmartCues, a library to support details-on-demand through dynamically computed overlays. For end-users, SmartCues provides multitouch interactions to construct complex queries for a variety of details. For designers, SmartCues offers an interaction library that can be used out-of-the-box, and can be extended for new charts and detail types. We demonstrate how SmartCues can be implemented across a wide array of visualization types and, through a lab study, show that end users can effectively use SmartCues.