Segmentor: a tool for manual refinement of 3D microscopy annotations.

BACKGROUND: Recent advances in tissue clearing techniques, combined with high-speed image acquisition through light sheet microscopy, enable rapid three-dimensional (3D) imaging of biological specimens, such as whole mouse brains, in a matter of hours. Quantitative analysis of such 3D images can help us understand how changes in brain structure lead to differences in behavior or cognition, but distinguishing densely packed features of interest, such as nuclei, from background can be challenging. Recent deep learning-based nuclear segmentation algorithms show great promise for automated segmentation, but require large numbers of accurate manually labeled nuclei as training data. RESULTS: We present Segmentor, an open-source tool for reliable, efficient, and user-friendly manual annotation and refinement of objects (e.g., nuclei) within 3D light sheet microscopy images. Segmentor employs a hybrid 2D-3D approach for visualizing and segmenting objects and contains features for automatic region splitting, designed specifically for streamlining the process of 3D segmentation of nuclei. We show that editing simultaneously in 2D and 3D using Segmentor significantly decreases time spent on manual annotations without affecting accuracy as compared to editing the same set of images with only 2D capabilities. CONCLUSIONS: Segmentor is a tool for increased efficiency of manual annotation and refinement of 3D objects that can be used to train deep learning segmentation algorithms, and is available at https://www.nucleininja.org/ and https://github.com/RENCI/Segmentor .

Using Counterfactuals to Improve Causal Inferences From Visualizations.

Traditional approaches to data visualization have often focused on comparing different subsets of data, and this is reflected in the many techniques developed and evaluated over the years for visual comparison. Similarly, common workflows for exploratory visualization are built upon the idea of users interactively applying various filter and grouping mechanisms in search of new insights. This paradigm has proven effective at helping users identify correlations between variables that can inform thinking and decision-making. However, recent studies show that consumers of visualizations often draw causal conclusions even when not supported by the data. Motivated by these observations, this article highlights recent advances from a growing community of researchers exploring methods that aim to directly support visual causal inference. However, many of these approaches have their own limitations, which limit their use in many real-world scenarios. This article, therefore, also outlines a set of key open challenges and corresponding priorities for new research to advance the state of the art in visual causal inference.

ImmCellFie: A user-friendly web-based platform to infer metabolic function from omics data.

Understanding cellular metabolism is important across biotechnology and biomedical research and has critical implications in a broad range of normal and pathological conditions. Here, we introduce the user-friendly web-based platform ImmCellFie, which allows the comprehensive analysis of metabolic functions inferred from transcriptomic or proteomic data. We explain how to set up a run using publicly available omics data and how to visualize the results. The ImmCellFie algorithm pushes beyond conventional statistical enrichment and incorporates complex biological mechanisms to quantify cell activity. For complete details on the use and execution of this protocol, please refer to Richelle et al. (2021).1.

Image and statistical analysis of melanocytic histology.

AIMS: We applied digital image analysis techniques to study selected types of melanocytic lesions. METHODS AND RESULTS: We used advanced digital image analysis to compare melanocytic lesions as follows: (i) melanoma to nevi, (ii) melanoma subtypes to nevi, (iii) severely dysplastic nevi to other nevi and (iv) melanoma to severely dysplastic nevi. We were successful in differentiating melanoma from nevi [receiver operating characteristic area (ROC) 0.95] using image-derived features, among which those related to nuclear size and shape and distance between nuclei were most important. Dividing melanoma into subtypes, even greater separation was obtained (ROC area 0.98 for superficial spreading melanoma; 0.95 for lentigo maligna melanoma; and 0.99 for unclassified). Severely dysplastic nevi were best differentiated from conventional and mildly dysplastic nevi by differences in cellular staining qualities (ROC area 0.84). We found that melanomas were separated from severely dysplastic nevi by features related to shape and staining qualities (ROC area 0.95). All comparisons were statistically significant (P < 0.0001). CONCLUSIONS: We offer a unique perspective into the evaluation of melanocytic lesions and demonstrate a technological application with increasing prevalence, and with potential use as an adjunct to traditional diagnosis in the future.

Improving Visualization Interpretation Using Counterfactuals.

Complex, high-dimensional data is used in a wide range of domains to explore problems and make decisions. Analysis of high-dimensional data, however, is vulnerable to the hidden influence of confounding variables, especially as users apply ad hoc filtering operations to visualize only specific subsets of an entire dataset. Thus, visual data-driven analysis can mislead users and encourage mistaken assumptions about causality or the strength of relationships between features. This work introduces a novel visual approach designed to reveal the presence of confounding variables via counterfactual possibilities during visual data analysis. It is implemented in CoFact, an interactive visualization prototype that determines and visualizes counterfactual subsets to better support user exploration of feature relationships. Using publicly available datasets, we conducted a controlled user study to demonstrate the effectiveness of our approach; the results indicate that users exposed to counterfactual visualizations formed more careful judgments about feature-to-outcome relationships.

Computational Metrics Can Provide Quantitative Values to Characterize Arthroscopic Field of View.

Purpose: The purpose of this study was to determine the inter-rater reliability of arthroscopic video quality, determine correlation between surgeon rating and computational image metrics, and facilitate a quantitative methodology for assessing video quality. Methods: Five orthopaedic surgeons reviewed 60 clips from deidentified arthroscopic shoulder videos and rated each on a four-point Likert scale from poor to excellent view. The videos were randomized, and the process was completed a total of three times. Each user rating was averaged to provide a user rating per clip. Each video frame was processed to calculate brightness, local contrast, redness (used to represent bleeding), and image entropy. Each metric was then averaged over each frame per video clip, providing four image quality metrics per clip. Results: Inter-rater reliability for grading video quality had an intraclass correlation of .974. Improved image quality rating was positively correlated with increased entropy (.8142; P < .001), contrast (.8013; P < .001), and brightness (.6120; P < .001), and negatively correlated with redness (-.8626; P < .001). A multiple linear regression model was calculated with the image metrics used as predictors for the image quality ranking, with an R-squared value of .775 and root mean square error of .42. Conclusions: Our study demonstrates strong inter-rater reliability between surgeons when describing image quality and strong correlations between image quality and the computed image metrics. A model based on these metrics enables automatic quantification of image quality. Clinical Relevance: Video quality during arthroscopic cases can impact the ease and duration of the case which could contribute to swelling and complication risk. This pilot study provides a quantitative method to assess video quality. Future works can objectively determine factors that affect visualization during arthroscopy and identify options for improvement.

Selection-Bias-Corrected Visualization via Dynamic Reweighting.

The collection and visual analysis of large-scale data from complex systems, such as electronic health records or clickstream data, has become increasingly common across a wide range of industries. This type of retrospective visual analysis, however, is prone to a variety of selection bias effects, especially for high-dimensional data where only a subset of dimensions is visualized at any given time. The risk of selection bias is even higher when analysts dynamically apply filters or perform grouping operations during ad hoc analyses. These bias effects threaten the validity and generalizability of insights discovered during visual analysis as the basis for decision making. Past work has focused on bias transparency, helping users understand when selection bias may have occurred. However, countering the effects of selection bias via bias mitigation is typically left for the user to accomplish as a separate process. Dynamic reweighting (DR) is a novel computational approach to selection bias mitigation that helps users craft bias-corrected visualizations. This paper describes the DR workflow, introduces key DR visualization designs, and presents statistical methods that support the DR process. Use cases from the medical domain, as well as findings from domain expert user interviews, are also reported.

Enabling Longitudinal Exploratory Analysis of Clinical COVID Data.

As the COVID-19 pandemic continues to impact the world, data is being gathered and analyzed to better understand the disease. Recognizing the potential for visual analytics technologies to support exploratory analysis and hypothesis generation from longitudinal clinical data, a team of collaborators worked to apply existing event sequence visual analytics technologies to a longitudinal clinical data from a cohort of 998 patients with high rates of COVID-19 infection. This paper describes the initial steps toward this goal, including: (1) the data transformation and processing work required to prepare the data for visual analysis, (2) initial findings and observations, and (3) qualitative feedback and lessons learned which highlight key features as well as limitations to address in future work.

Model-based assessment of mammalian cell metabolic functionalities using omics data.

Omics experiments are ubiquitous in biological studies, leading to a deluge of data. However, it is still challenging to connect changes in these data to changes in cell functions because of complex interdependencies between genes, proteins, and metabolites. Here, we present a framework allowing researchers to infer how metabolic functions change on the basis of omics data. To enable this, we curated and standardized lists of metabolic tasks that mammalian cells can accomplish. Genome-scale metabolic networks were used to define gene sets associated with each metabolic task. We further developed a framework to overlay omics data on these sets and predict pathway usage for each metabolic task. We demonstrated how this approach can be used to quantify metabolic functions of diverse biological samples from the single cell to whole tissues and organs by using multiple transcriptomic datasets. To facilitate its adoption, we integrated the approach into GenePattern (www.genepattern.org-CellFie).

Selection Bias Tracking and Detailed Subset Comparison for High-Dimensional Data.

The collection of large, complex datasets has become common across a wide variety of domains. Visual analytics tools increasingly play a key role in exploring and answering complex questions about these large datasets. However, many visualizations are not designed to concurrently visualize the large number of dimensions present in complex datasets (e.g. tens of thousands of distinct codes in an electronic health record system). This fact, combined with the ability of many visual analytics systems to enable rapid, ad-hoc specification of groups, or cohorts, of individuals based on a small subset of visualized dimensions, leads to the possibility of introducing selection bias-when the user creates a cohort based on a specified set of dimensions, differences across many other unseen dimensions may also be introduced. These unintended side effects may result in the cohort no longer being representative of the larger population intended to be studied, which can negatively affect the validity of subsequent analyses. We present techniques for selection bias tracking and visualization that can be incorporated into high-dimensional exploratory visual analytics systems, with a focus on medical data with existing data hierarchies. These techniques include: (1) tree-based cohort provenance and visualization, including a user-specified baseline cohort that all other cohorts are compared against, and visual encoding of cohort "drift", which indicates where selection bias may have occurred, and (2) a set of visualizations, including a novel icicle-plot based visualization, to compare in detail the per-dimension differences between the baseline and a user-specified focus cohort. These techniques are integrated into a medical temporal event sequence visual analytics tool. We present example use cases and report findings from domain expert user interviews.

Visual Analysis of High-Dimensional Event Sequence Data via Dynamic Hierarchical Aggregation.

Temporal event data are collected across a broad range of domains, and a variety of visual analytics techniques have been developed to empower analysts working with this form of data. These techniques generally display aggregate statistics computed over sets of event sequences that share common patterns. Such techniques are often hindered, however, by the high-dimensionality of many real-world event sequence datasets which can prevent effective aggregation. A common coping strategy for this challenge is to group event types together prior to visualization, as a pre-process, so that each group can be represented within an analysis as a single event type. However, computing these event groupings as a pre-process also places significant constraints on the analysis. This paper presents a new visual analytics approach for dynamic hierarchical dimension aggregation. The approach leverages a predefined hierarchy of dimensions to computationally quantify the informativeness, with respect to a measure of interest, of alternative levels of grouping within the hierarchy at runtime. This information is then interactively visualized, enabling users to dynamically explore the hierarchy to select the most appropriate level of grouping to use at any individual step within an analysis. Key contributions include an algorithm for interactively determining the most informative set of event groupings for a specific analysis context, and a scented scatter-plus-focus visualization design with an optimization-based layout algorithm that supports interactive hierarchical exploration of alternative event type groupings. We apply these techniques to high-dimensional event sequence data from the medical domain and report findings from domain expert interviews.

Being the Victim of Intimate Partner Violence in Virtual Reality: First- Versus Third-Person Perspective.

Immersive virtual reality is widely used for research and clinical purposes. Here we explored the impact of an immersive virtual scene of intimate partner violence experienced from the victim's perspective (first person), as opposed to witnessing it as an observer (third person). We are ultimately interested in the potential of this approach to rehabilitate batterers and in understanding the mechanisms underlying this process. For this, non-offender men experienced the scene either from the perspective of the victim's virtual body (a female avatar), which moved synchronously with the participants' real movements, or from the perspective of an observer, while we recorded their behavior and physiological responses. We also evaluated through questionnaires, interviews and implicit association tests their subjective impressions and potential pre/post changes in implicit gender bias following the experience. We found that in all participants, regardless of perspective, the magnitude of the physiological reactions to virtual threatening stimuli was related to how vulnerable they felt for being a woman, the sensation that they could be assaulted, how useful the scene could be for batterer rehabilitation, and how different it would have been to experience the scenario on TV. Furthermore, we found that their level of identification with the female avatar correlated with the decrease in prejudice against women. Although the first-person perspective (1PP) facilitated taking the scene personally, generated a sensation of fear, helplessness, and vulnerability, and tended to induce greater behavioral and physiological reactions, we show that the potential for batterer rehabilitation originates from presence and identification with the victim, which in turn is more easily, but not exclusively, achieved through 1PP. This study is relevant for the development of advanced virtual reality tools for clinical purposes.

Ontology-Based Interactive Visualization of Patient-Generated Research Questions.

BACKGROUND: Crohn's disease and colitis are chronic conditions that affect every facet of patients' lives (e.g., social interaction, family, work, diet, and sleep). Thus, treatment consists largely of disease management. The University of North Carolina at Chapel Hill chapter of the Crohn's and Colitis Foundation-IBD Partners-has created an interactive website that, in addition to providing helpful information and disease management tools, provides a discussion forum for patients to talk about their experiences and suggest new lines of research into Crohn's disease and colitis. OBJECTIVES: The primary objective of this work is to enable researchers to more effectively browse the forum content. Researchers wish to identify important/popular patient-suggested research topics, appreciate the full breadth of the research topics, and see connections between them, in order to more effectively prioritize research agendas. METHODS: To help structure the forum content we have developed an ontology describing the major themes in the discussion forum. We have also created a prototype interactive visualization tool that leverages the ontology to help researchers identify common themes and related patient-generated research topics via linked views of (1) the ontology, (2) a research topic overview clustered by relevant ontology terms, and (3) a detailed view of the discussion forum content. RESULTS: We discuss visualizations and interactions enabled by the visualization tool, provide an example scenario using the tool, and discuss limitations and future work based on feedback from potential users. CONCLUSION: The integration of a user-community specific ontology with an interactive visualization tool is a promising approach for enabling researchers to more effectively study user-generated research questions.

Evaluating visual analytics for health informatics applications: a systematic review from the American Medical Informatics Association Visual Analytics Working Group Task Force on Evaluation.

OBJECTIVE: This article reports results from a systematic literature review related to the evaluation of data visualizations and visual analytics technologies within the health informatics domain. The review aims to (1) characterize the variety of evaluation methods used within the health informatics community and (2) identify best practices. METHODS: A systematic literature review was conducted following PRISMA guidelines. PubMed searches were conducted in February 2017 using search terms representing key concepts of interest: health care settings, visualization, and evaluation. References were also screened for eligibility. Data were extracted from included studies and analyzed using a PICOS framework: Participants, Interventions, Comparators, Outcomes, and Study Design. RESULTS: After screening, 76 publications met the review criteria. Publications varied across all PICOS dimensions. The most common audience was healthcare providers (n = 43), and the most common data gathering methods were direct observation (n = 30) and surveys (n = 27). About half of the publications focused on static, concentrated views of data with visuals (n = 36). Evaluations were heterogeneous regarding setting and measurements used. DISCUSSION: When evaluating data visualizations and visual analytics technologies, a variety of approaches have been used. Usability measures were used most often in early (prototype) implementations, whereas clinical outcomes were most common in evaluations of operationally-deployed systems. These findings suggest opportunities for both (1) expanding evaluation practices, and (2) innovation with respect to evaluation methods for data visualizations and visual analytics technologies across health settings. CONCLUSION: Evaluation approaches are varied. New studies should adopt commonly reported metrics, context-appropriate study designs, and phased evaluation strategies.

Contextual Visualization.

Unseen information can lead to various "threats to validity" when analyzing complex datasets using visual tools, resulting in potentially biased findings. We enumerate sources of unseen information and argue that a new focus on contextual visualization methods is needed to inform users of these threats and to mitigate their effects.

The Cell Cycle Browser: An Interactive Tool for Visualizing, Simulating, and Perturbing Cell-Cycle Progression.

The cell cycle is driven by precise temporal coordination among many molecular activities. To understand and explore this process, we developed the Cell Cycle Browser (CCB), an interactive web interface based on real-time reporter data collected in proliferating human cells. This tool facilitates visualizing, organizing, simulating, and predicting the outcomes of perturbing cell-cycle parameters. Time-series traces from individual cells can be combined to build a multi-layered timeline of molecular activities. Users can simulate the cell cycle using computational models that capture the dynamics of molecular activities and phase transitions. By adjusting individual expression levels and strengths of molecular relationships, users can predict effects on the cell cycle. Virtual assays, such as growth curves and flow cytometry, provide familiar outputs to compare cell-cycle behaviors for data and simulations. The CCB serves to unify our understanding of cell-cycle dynamics and provides a platform for generating hypotheses through virtual experiments.

Virtual pyeloscopy using volumetric depth peeling.

RATIONALE AND OBJECTIVES: The aim of the study is to test a new volume-rendering method, volumetric depth peeling (VDP), for use in virtual pyeloscopy. MATERIALS AND METHODS: VDP was applied to axial contrast-enhanced source computed tomographic (CT) images and coronal reformatted maximum intensity projections of three contrast-filled gloves containing objects of varying density. Similar renderings were performed on CT urograms performed to evaluate hematuria (n = 20). Renderings were assessed for anatomic appearance of ureters and specific calyces in comparison with source images. RESULTS: Objects of soft-tissue and calcific density ranging in size from 4 to 20 mm were identified by using VDP within the glove phantoms. Normal and deformed renal calyces were well visualized by using VDP; however, two stones were not identified. The minimal ureteral width that could be visualized was 3 mm. CONCLUSION: VDP may be a useful technique for virtual pyeloscopy providing that a robust and user-friendly computer interface can be developed.

Data-Driven Healthcare: Challenges and Opportunities for Interactive Visualization.

The healthcare industry's widespread digitization efforts are reshaping one of the largest sectors of the world's economy. This transformation is enabling systems that promise to use ever-improving data-driven evidence to help doctors make more precise diagnoses, institutions identify at risk patients for intervention, clinicians develop more personalized treatment plans, and researchers better understand medical outcomes within complex patient populations. Given the scale and complexity of the data required to achieve these goals, advanced data visualization tools have the potential to play a critical role. This article reviews a number of visualization challenges unique to the healthcare discipline.

Innovative information visualization of electronic health record data: a systematic review.

OBJECTIVE: This study investigates the use of visualization techniques reported between 1996 and 2013 and evaluates innovative approaches to information visualization of electronic health record (EHR) data for knowledge discovery. METHODS: An electronic literature search was conducted May-July 2013 using MEDLINE and Web of Knowledge, supplemented by citation searching, gray literature searching, and reference list reviews. General search terms were used to assure a comprehensive document search. RESULTS: Beginning with 891 articles, the number of articles was reduced by eliminating 191 duplicates. A matrix was developed for categorizing all abstracts and to assist with determining those to be excluded for review. Eighteen articles were included in the final analysis. DISCUSSION: Several visualization techniques have been extensively researched. The most mature system is LifeLines and its applications as LifeLines2, EventFlow, and LifeFlow. Initially, research focused on records from a single patient and visualization of the complex data related to one patient. Since 2010, the techniques under investigation are for use with large numbers of patient records and events. Most are linear and allow interaction through scaling and zooming to resize. Color, density, and filter techniques are commonly used for visualization. CONCLUSIONS: With the burgeoning increase in the amount of electronic healthcare data, the potential for knowledge discovery is significant if data are managed in innovative and effective ways. We identify challenges discovered by previous EHR visualization research, which will help researchers who seek to design and improve visualization techniques.

Appearance normalization of histology slides.

This paper presents a method for automatic color and intensity normalization of digitized histology slides stained with two different agents. In comparison to previous approaches, prior information on the stain vectors is used in the plane estimation process, resulting in improved stability of the estimates. Due to the prevalence of hematoxylin and eosin staining for histology slides, the proposed method has significant practical utility. In particular, it can be used as a first step to standardize appearance across slides and is effective at countering effects due to differing stain amounts and protocols and counteracting slide fading. The approach is validated against non-prior plane-fitting using synthetic experiments and 13 real datasets. Results of application of the method to adjustment of faded slides are given, and the effectiveness of the method in aiding statistical classification is shown.