Evaluating Text Reading Speed in VR Scenes and 3D Particle Visualizations.

This work reports how text size and other rendering conditions affect reading speeds in a virtual reality environment and a scientific data analysis application. Displaying text legibly yet space-efficiently is a challenging problem in immersive displays. Effective text displays that enable users to read at their maximum speed must consider the variety of virtual reality (VR) display hardware and possible visual exploration tasks. We investigate how text size and display parameters affect reading speed and legibility in three state-of-the-art VR displays: two head-mounted displays and one CAVE. In our perception experiments, we establish limits where reading speed declines as the text size approaches the so-called critical print sizes (CPS) of individual displays, which can inform the design of uniform reading experiences across different VR systems. We observe an inverse correlation between display resolution and CPS. Yet, even in high-fidelity VR systems, the measured CPS was larger than in comparable physical text displays, highlighting the value of increased VR display resolutions in certain visualization scenarios. Our findings indicate that CPS can be an effective metric for evaluating VR display usability. Additionally, we evaluate the effects of text panel placement, orientation, and occlusion-reducing rendering methods on reading speeds in generic volumetric particle visualizations. Our study provides insights into the trade-off between text representation and legibility in cluttered immersive environments with specific suggestions for visualization designers and highlight areas for further research.

Memory-efficient semantic segmentation of large microscopy images using graph-based neural networks.

We present a graph neural network (GNN)-based framework applied to large-scale microscopy image segmentation tasks. While deep learning models, like convolutional neural networks (CNNs), have become common for automating image segmentation tasks, they are limited by the image size that can fit in the memory of computational hardware. In a GNN framework, large-scale images are converted into graphs using superpixels (regions of pixels with similar color/intensity values), allowing us to input information from the entire image into the model. By converting images with hundreds of millions of pixels to graphs with thousands of nodes, we can segment large images using memory-limited computational resources. We compare the performance of GNN- and CNN-based segmentation in terms of accuracy, training time and required graphics processing unit memory. Based on our experiments with microscopy images of biological cells and cell colonies, GNN-based segmentation used one to three orders-of-magnitude fewer computational resources with only a change in accuracy of $-2\;%$ to $+0.3\;%$. Furthermore, errors due to superpixel generation can be reduced by either using better superpixel generation algorithms or increasing the number of superpixels, thereby allowing for improvement in the GNN framework's accuracy. This trade-off between accuracy and computational cost over CNN models makes the GNN framework attractive for many large-scale microscopy image segmentation tasks in biology.

Visual Cue Effects on a Classification Accuracy Estimation Task in Immersive Scatterplots.

Immersive visualization in virtual reality (VR) allows us to exploit visual cues for perception in 3D space, yet few existing studies have measured the effects of visual cues. Across a desktop monitor and a head-mounted display (HMD), we assessed scatterplot designs which vary their use of visual cues-motion, shading, perspective (graphical projection), and dimensionality-on two sets of data. We conducted a user study with a summary task in which 32 participants estimated the classification accuracy of an artificial neural network from the scatterplots. With Bayesian multilevel modeling, we capture the intricate visual effects and find that no cue alone explains all the variance in estimation error. Visual motion cues generally reduce participants' estimation error; besides this motion, using other cues may increase participants' estimation error. Using an HMD, adding visual motion cues, providing a third data dimension, or showing a more complicated dataset leads to longer response times. We speculate that most visual cues may not strongly affect perception in immersive analytics unless they change people's mental model about data. In summary, by studying participants as they interpret the output from a complicated machine learning model, we advance our understanding of how to use the visual cues in immersive analytics.

Virtual and augmented reality: New tools for visualizing, analyzing, and communicating complex morphology.

Virtual and augmented reality (VR/AR) are new technologies with the power to revolutionize the study of morphology. Modern imaging approaches such as computed tomography, laser scanning, and photogrammetry have opened up a new digital world, enabling researchers to share and analyze morphological data electronically and in great detail. Because this digital data exists on a computer screen, however, it can remain difficult to understand and unintuitive to interact with. VR/AR technologies bridge the analog-to-digital divide by presenting 3D data to users in a very similar way to how they would interact with actual anatomy, while also providing a more immersive experience and greater possibilities for exploration. This manuscript describes VR/AR hardware, software, and techniques, and is designed to give practicing morphologists and educators a primer on using these technologies in their research, pedagogy, and communication to a wide variety of audiences. We also include a series of case studies from the presentations and workshop given at the 2019 International Congress of Vertebrate Morphology, and suggest best practices for the use of VR/AR in comparative morphology.

A Virtual Reality Memory Palace Variant Aids Knowledge Retrieval from Scholarly Articles.

We present exploratory research of virtual reality techniques and mnemonic devices to assist in retrieving knowledge from scholarly articles. We used abstracts of scientific publications to represent knowledge in scholarly articles; participants were asked to read, remember, and retrieve knowledge from a set of abstracts. We conducted an experiment to compare participants' recall and recognition performance in three different conditions: a control condition without a pre-specified strategy to test baseline individual memory ability, a condition using an image-based variant of a mnemonic called a "memory palace," and a condition using a virtual reality-based variant of a memory palace. Our analyses show that using a virtual reality-based memory palace variant greatly increased the amount of knowledge retrieved and retained over the baseline, and it shows a moderate improvement over the other image-based memory palace variant. Anecdotal feedback from participants suggested that personalizing a memory palace variant would be appreciated. Our results support the value of virtual reality for some high-level cognitive tasks and help improve future applications of virtual reality and visualization.

Tractography Processing with the Sparse Closest Point Transform.

We propose a novel approach for processing diffusion MRI tractography datasets using the sparse closest point transform (SCPT). Tractography enables the 3D geometry of white matter pathways to be reconstructed; however, algorithms for processing them are often highly customized, and thus, do not leverage the existing wealth of machine learning (ML) algorithms. We investigated a vector-space tractography representation that aims to bridge this gap by using the SCPT, which consists of two steps: first, extracting sparse and representative landmarks from a tractography dataset, and second transforming curves relative to these landmarks with a closest point transform. We explore its use in three typical tasks: fiber bundle clustering, simplification, and selection across a population. The clustering algorithm groups fibers from single whole-brain datasets using a non-parametric k-means clustering algorithm, with performance compared with three alternative methods and across four datasets. The simplification algorithm removes redundant curves to improve interactive visualization, with performance gauged relative to random subsampling. The selection algorithm extracts bundles across a population using a one-class Gaussian classifier derived from an atlas prototype, with performance gauged by scan-rescan reliability and sensitivity to normal aging, as compared to manual mask-based selection. Our results demonstrate how the SCPT enables the novel application of existing vector-space ML algorithms to create effective and efficient tools for tractography processing. Our experimental data is available online, and our software implementation is available in the Quantitative Imaging Toolkit.

Visualization of 3D Stress Tensor Fields Using Superquadric Glyphs on Displacement Streamlines.

Stress tensor fields play a central role in solid mechanics studies, but their visualization in 3D space remains challenging as the information-dense multi-variate tensor needs to be sampled in 3D space while avoiding clutter. Taking cues from current tensor visualizations, we adapted glyph-based visualization for stress tensors in 3D space. We also developed a testing framework and performed user studies to evaluate the various glyph-based tensor visualizations for objective accuracy measures, and subjective user feedback for each visualization method. To represent the stress tensor, we color encoded the original superquadric glyph, and in the user study, we compared it to superquadric glyphs developed for second-order symmetric tensors. We found that color encoding improved the user accuracy measures, while the users also rated our method the highest. We compared our method of placing stress tensor glyphs on displacement streamlines to the glyph placement on a 3D grid. In the visualization, we modified the glyph to show both the stress tensor and the displacement vector at each sample point. The participants preferred our method of glyph placement on displacement streamlines as it highlighted the underlying continuous structure in the tensor field.

Topic-Based Exploration and Embedded Visualizations for Research Idea Generation.

This work analyzes sensemaking frameworks and experiments with an iteratively designed visual analysis tool to identify design implications for facilitating research idea generation using visualizations. Our tool, ThoughtFlow, structures and visualizes literature collections using topic models to bridge the information gap between core activities during research ideation. To help users stay focused on a topic while discovering relevant documents, we designed and analyzed usage patterns for two types of embedded visualization that help determine document relevance while minimizing distraction. We analyzed how research ideation outcomes and processes differ when using ThoughtFlow and conventional search engines by augmenting insight-based evaluation with concept-map analysis. Our results suggest that operations afforded by topic models match well with later ideation stages when coherent topics have emerged, but not with early stages when users are still relying heavily on individual keywords to gather background knowledge. We also present qualitative evidence that citation sparklines encourage more exploration of recommended references, and that a preference for paper thumbnails may depend on the consistency between the evidence and the current mental frame.

Measuring the Effects of Scalar and Spherical Colormaps on Ensembles of DMRI Tubes.

We report empirical study results on the color encoding of ensemble scalar and orientation to visualize diffusion magnetic resonance imaging (DMRI) tubes. The experiment tested six scalar colormaps for average fractional anisotropy (FA) tasks (grayscale, blackbody, diverging, isoluminant-rainbow, extended-blackbody, and coolwarm) and four three-dimensional (3D) spherical colormaps for tract tracing tasks (uniform gray, absolute, eigenmaps, and Boy's surface embedding). We found that extended-blackbody, coolwarm, and blackbody remain the best three approaches for identifying ensemble average in 3D. Isoluminant-rainbow colormap led to the same ensemble mean accuracy as other colormaps. However, more than 50 percent of the answers consistently had higher estimates of the ensemble average, independent of the mean values. The number of hues, not luminance, influences ensemble estimates of mean values. For ensemble orientation-tracing tasks, we found that both Boy's surface embedding (greatest spatial resolution and contrast) and absolute colormaps (lowest spatial resolution and contrast) led to more accurate answers than the eigenmaps scheme (medium resolution and contrast), acting as the uncanny-valley phenomenon of visualization design in terms of accuracy. Absolute colormap broadly used in brain science is a good default spherical colormap. We could conclude from our study that human visual processing of a chunk of colors differs from that of single colors.

Towards Placental Surface Vasculature Exploration in Virtual Reality.

We present a case study evaluating the potential for interactively identifying placental surface blood vessels using magnetic resonance imaging (MRI) scans in virtual reality (VR) environments. We visualized the MRI data using direct volume rendering in a high-fidelity CAVE-like VR system, allowing medical professionals to identify relevant placental vessels directly from volume visualizations in the VR system, without prior vessel segmentation. Participants were able to trace most of the observable vascular structure, and consistently identified blood vessels down to diameters of 1 mm, an important requirement in diagnosing vascular diseases. Qualitative feedback from our participants suggests that our VR visualization is easy to understand and allows intuitive data exploration, but complex user interactions remained a challenge. Using these observations, we discuss implications and requirements for spatial tracing user interaction methods in VR environments. We believe that VR MRI visualizations are the next step towards effective surgery planning for prenatal diseases.

Predicting Carpal Bone Kinematics Using an Expanded Digital Database of Wrist Carpal Bone Anatomy and Kinematics.

The wrist can be considered a 2 degrees-of-freedom joint with all movements reflecting the combination of flexion-extension and radial-ulnar deviation. Wrist motions are accomplished by the kinematic reduction of the 42 degrees-of-freedom of the individual carpal bones. While previous studies have demonstrated the minimal motion of the scaphoid and lunate as the wrist moves along the dart-thrower's path or small relative motion between hamate-capitate-trapezoid, an understanding of the kinematics of the complete carpus across all wrist motions remains lacking. To address this, we assembled an open-source database of in vivo carpal motions and developed mathematical models of the carpal kinematics as a function of wrist motion. Quadratic surfaces were trained for each of the 42-carpal bone degrees-of-freedom and the goodness of fits were evaluated. Using the models, paths of wrist motion that generated minimal carpal rotations or translations were determined. Model predictions were best for flexion-extension, radial-ulnar deviation, and volar-dorsal translations for all carpal bones with R 2 > 0.8, while the estimates were least effective for supination-pronation with R 2 < 0.6. The wrist path of motion's analysis indicated that the distal row of carpal bones moves rigidly together (<3° motion), along the anatomical axis of wrist motion, while the bones in the proximal row undergo minimal motion when the wrist moves in a path oblique to the main axes. The open-source dataset along with its graphical user interface and mathematical models should facilitate clinical visualization and enable new studies of carpal kinematics and function. © 2019 Orthopaedic Research Society. Published by Wiley Periodicals, Inc. J Orthop Res 37:2661-2670, 2019.

Behavioral inhibition corresponds to white matter fiber bundle integrity in older adults.

Little is known about the contribution of white matter integrity to inhibitory cognitive control, particularly in healthy aging. The present study examines the correspondence between white matter fiber bundle length and behavioral inhibition in 37 community-dwelling older adults (aged 51-78 years). Participants underwent neuroimaging with 3 Tesla MRI, and completed a behavioral test of inhibition (i.e., Go/NoGo task). Quantitative tractography derived from diffusion tensor imaging (qtDTI) was used to measure white matter fiber bundle lengths (FBLs) in tracts known to innervate frontal brain regions, including the anterior corpus callosum (AntCC), the cingulate gyrus segment of the cingulum bundle (CING), uncinate fasciculus (UNC), and the superior longitudinal fasciculus (SLF). Performance on the Go/NoGo task was measured by the number of commission errors standardized to reaction time. Hierarchical regression models revealed that shorter FBLs in the CING (p < 0.05) and the bilateral UNC (p < 0.01) were associated with lower inhibitory performance after adjusting for multiple comparisons, supporting a disconnection model of response inhibition in older adults. Prospective longitudinal studies are needed to examine the evolution of inhibitory errors in older adult populations and potential for therapeutic intervention.

Developing Virtual Reality Visualizations for Unsteady Flow Analysis of Dinosaur Track Formation using Scientific Sketching.

We present the results of a two-year design study to developing virtual reality (VR) flow visualization tools for the analysis of dinosaur track creation in a malleable substrate. Using Scientific Sketching methodology, we combined input from illustration artists, visualization experts, and domain scientists to create novel visualization methods. By iteratively improving visualization concepts at multiple levels of abstraction we helped domain scientists to gain insights into the relationship between dinosaur foot movements and substrate deformations. We involved over 20 art and computer science students from a VR design course in a rapid visualization sketching cycle, guided by our paleontologist collaborators through multiple critique sessions. This allowed us to explore a wide range of potential visualization methods and select the most promising methods for actual implementation. Our resulting visualization methods provide paleontologists with effective tools to analyze their data through particle, pathline and time surface visualizations. We also introduce a set of visual metaphors to compare foot motion in relation to substrate deformation by using pathsurfaces. This is one of the first large-scale projects using Scientific Sketching as a development methodology. We discuss how the research questions of our collaborators have evolved during the sketching and prototyping phases. Finally, we provide lessons learned and usage considerations for Scientific Sketching based on the experiences gathered during this project.

Preliminary mapping of the structural effects of age in pediatric bipolar disorder with multimodal MR imaging.

This study investigates multimodal structural MR imaging biomarkers of development trajectories in pediatric bipolar disorder. T1-weighted and diffusion-weighted MR imaging was conducted to investigate cross-sectional group differences with age between typically developing controls (N = 26) and youths diagnosed with bipolar disorder (N = 26). Region-based analysis was used to examine cortical thickness of gray matter and diffusion tensor parameters in superficial white matter, and tractography-based analysis was used to examine deep white matter fiber bundles. Patients and controls showed significantly different maturation trajectories across brain areas; however, the magnitude of differences varied by region. The rate of cortical thinning with age was greater in patients than controls in the left frontal pole. While controls showed increasing fractional anisotropy (FA) and axial diffusivity (AD) with age, patients showed an opposite trend of decreasing FA and AD with age in fronto-temporal-striatal regions located in both superficial and deep white matter. The findings support fronto-temporal-striatal alterations in the developmental trajectories of youths diagnosed with bipolar disorder, and further, show the value of multimodal computational techniques in the assessment of neuropsychiatric disorders. These preliminary results warrant further investigation into longitudinal changes and the effects of treatment in the brain areas identified in this study.

An Analysis of Automated Visual Analysis Classification: Interactive Visualization Task Inference of Cancer Genomics Domain Experts.

We show how mouse interaction log classification can help visualization toolsmiths understand how their tools are used "in the wild" through an evaluation of MAGI - a cancer genomics visualization tool. Our primary contribution is an evaluation of twelve visual analysis task classifiers, which compares predictions to task inferences made by pairs of genomics and visualization experts. Our evaluation uses common classifiers that are accessible to most visualization evaluators: -nearest neighbors, linear support vector machines, and random forests. By comparing classifier predictions to visual analysis task inferences made by experts, we show that simple automated task classification can have up to 73 percent accuracy and can separate meaningful logs from "junk" logs with up to 91 percent accuracy. Our second contribution is an exploration of common MAGI interaction trends using classification predictions, which expands current knowledge about ecological cancer genomics visualization tasks. Our third contribution is a discussion of how automated task classification can inform iterative tool design. These contributions suggest that mouse interaction log analysis is a viable method for (1) evaluating task requirements of client-side-focused tools, (2) allowing researchers to study experts on larger scales than is typically possible with in-lab observation, and (3) highlighting potential tool evaluation bias.

White matter fiber bundle lengths are shorter in cART naive HIV: an analysis of quantitative diffusion tractography in South Africa.

This study examines white matter microstructure using quantitative tractography diffusion magnetic resonance imaging (qtdMRI) in HIV+ individuals from South Africa who were naïve or early in the initiation of antiretroviral therapy. Fiber bundle length (FBL) metrics, generated from qtdMRI, for whole brain and six white matter tracts of interest (TOI) were assessed for 135 HIV+ and 21 HIV- individuals. The association between FBL metrics, measures of disease burden, and neuropsychological performance were also investigated. Results indicate significantly reduced sum of whole brain fiber bundle lengths (FBL, p < 0.001), but not average whole brain FBL in the HIV+ group compared to the HIV- controls. The HIV+ group exhibited significantly shorter sum of FBL in all six TOIs examined: the anterior thalamic radiation, cingulum bundle, inferior and superior longitudinal fasciculi, inferior frontal occipital fasciculus, and the uncinate fasciculus. Additionally, average FBLs were significantly shorter select TOIs including the inferior longitudinal fasciculus, cingulum bundle, and the anterior thalamic radiation. Shorter whole brain FBL sum metrics were associated with poorer neuropsychological performance, but were not associated with markers of disease burden. Taken together these findings suggest HIV affects white matter architecture primarily through reductions in white matter fiber numbers and, to a lesser degree, the shortening of fibers along a bundle path.

Vulnerability of white matter tracts and cognition to the SOD2 polymorphism: A preliminary study of antioxidant defense genes in brain aging.

Oxidative stress is a key mechanism of the aging process that can cause damage to brain white matter and cognitive functions. Polymorphisms in the superoxide dismutase 2 (SOD2) and catalase (CAT) genes have been associated with abnormalities in antioxidant enzyme activity in the aging brain, suggesting a risk for enhanced oxidative damage to white matter and cognition among older individuals with these genetic variants. The present study compared differences in white matter microstructure and cognition among 96 older adults with and without genetic risk factors of SOD2 (rs4880) and CAT (rs1001179). Results revealed higher radial diffusivity in the anterior thalamic radiation among SOD2 CC genotypes compared to CT/TT genotypes. Further, the CC genotype moderated the relationship between the hippocampal cingulum and processing speed, though this did not survive multiple test correction. The CAT polymorphism was not associated with brain outcomes in this cohort. These results suggest that the CC genotype of SOD2 is an important genetic marker of suboptimal brain aging in healthy individuals.

Topological Organization of Whole-Brain White Matter in HIV Infection.

Infection with human immunodeficiency virus (HIV) is associated with neuroimaging alterations. However, little is known about the topological organization of whole-brain networks and the corresponding association with cognition. As such, we examined structural whole-brain white matter connectivity patterns and cognitive performance in 29 HIV+ young adults (mean age = 25.9) with limited or no HIV treatment history. HIV+ participants and demographically similar HIV- controls (n = 16) residing in South Africa underwent magnetic resonance imaging (MRI) and neuropsychological testing. Structural network models were constructed using diffusion MRI-based multifiber tractography and T1-weighted MRI-based regional gray matter segmentation. Global network measures included whole-brain structural integration, connection strength, and structural segregation. Cognition was measured using a neuropsychological global deficit score (GDS) as well as individual cognitive domains. Results revealed that HIV+ participants exhibited significant disruptions to whole-brain networks, characterized by weaker structural integration (characteristic path length and efficiency), connection strength, and structural segregation (clustering coefficient) than HIV- controls (p < 0.05). GDSs and performance on learning/recall tasks were negatively correlated with the clustering coefficient (p < 0.05) in HIV+ participants. Results from this study indicate disruption to brain network integrity in treatment-limited HIV+ young adults with corresponding abnormalities in cognitive performance.

Cognitive reserve moderates the relationship between neuropsychological performance and white matter fiber bundle length in healthy older adults.

Recent work using novel neuroimaging methods has revealed shorter white matter fiber bundle length (FBL) in older compared to younger adults. Shorter FBL also corresponds to poorer performance on cognitive measures sensitive to advanced age. However, it is unclear if individual factors such as cognitive reserve (CR) effectively moderate the relationship between FBL and cognitive performance. This study examined CR as a potential moderator of cognitive performance and brain integrity as defined by FBL. Sixty-three healthy adults underwent neuropsychological evaluation and 3T brain magnetic resonance imaging. Cognitive performance was measured using the Repeatable Battery of Assessment of Neuropsychological Status (RBANS). FBL was quantified from tractography tracings of white matter fiber bundles, derived from the diffusion tensor imaging. CR was determined by estimated premorbid IQ. Analyses revealed that lower scores on the RBANS were associated with shorter whole brain FBL (p = 0.04) and lower CR (p = 0.01) CR moderated the relationship between whole brain FBL and RBANS score (p < 0.01). Tract-specific analyses revealed that CR also moderated the association between FBL in the hippocampal segment of the cingulum and RBANS performance (p = 0.03). These results demonstrate that lower cognitive performance on the RBANS is more common with low CR and short FBL. On the contrary, when individuals have high CR, the relationship between FBL and cognitive performance is attenuated. Overall, CR protects older adults against lower cognitive performance despite age-associated reductions in FBL.

Fauxvea: Crowdsourcing Gaze Location Estimates for Visualization Analysis Tasks.

We present the design and evaluation of a method for estimating gaze locations during the analysis of static visualizations using crowdsourcing. Understanding gaze patterns is helpful for evaluating visualizations and user behaviors, but traditional eye-tracking studies require specialized hardware and local users. To avoid these constraints, we developed a method called Fauxvea, which crowdsources visualization tasks on the Web and estimates gaze fixations through cursor interactions without eye-tracking hardware. We ran experiments to evaluate how gaze estimates from our method compare with eye-tracking data. First, we evaluated crowdsourced estimates for three common types of information visualizations and basic visualization tasks using Amazon Mechanical Turk (MTurk). In another, we reproduced findings from a previous eye-tracking study on tree layouts using our method on MTurk. Results from these experiments show that fixation estimates using Fauxvea are qualitatively and quantitatively similar to eye tracking on the same stimulus-task pairs. These findings suggest that crowdsourcing visual analysis tasks with static information visualizations could be a viable alternative to traditional eye-tracking studies for visualization research and design.