Residency Octree: A Hybrid Approach for Scalable Web-Based Multi-Volume Rendering.

We present a hybrid multi-volume rendering approach based on a novel Residency Octree that combines the advantages of out-of-core volume rendering using page tables with those of standard octrees. Octree approaches work by performing hierarchical tree traversal. However, in octree volume rendering, tree traversal and the selection of data resolution are intrinsically coupled. This makes fine-grained empty-space skipping costly. Page tables, on the other hand, allow access to any cached brick from any resolution. However, they do not offer a clear and efficient strategy for substituting missing high-resolution data with lower-resolution data. We enable flexible mixed-resolution out-of-core multi-volume rendering by decoupling the cache residency of multi-resolution data from a resolution-independent spatial subdivision determined by the tree. Instead of one-to-one node-to-brick correspondences, each residency octree node is mapped to a set of bricks from different resolution levels. This makes it possible to efficiently and adaptively choose and mix resolutions, adapt sampling rates, and compensate for cache misses. At the same time, residency octrees support fine-grained empty-space skipping, independent of the data subdivision used for caching. Finally, to facilitate collaboration and outreach, and to eliminate local data storage, our implementation is a web-based, pure client-side renderer using WebGPU and WebAssembly. Our method is faster than prior approaches and efficient for many data channels with a flexible and adaptive choice of data resolution.

Fuzzy Spreadsheet: Understanding and Exploring Uncertainties in Tabular Calculations.

Spreadsheet-based tools provide a simple yet effective way of calculating values, which makes them the number-one choice for building and formalizing simple models for budget planning and many other applications. A cell in a spreadsheet holds one specific value and gives a discrete, overprecise view of the underlying model. Therefore, spreadsheets are of limited use when investigating the inherent uncertainties of such models and answering what-if questions. Existing extensions typically require a complex modeling process that cannot easily be embedded in a tabular layout. In Fuzzy Spreadsheet, a cell can hold and display a distribution of values. This integrated uncertainty-handling immediately conveys sensitivity and robustness information. The fuzzification of the cells enables calculations not only with precise values but also with distributions, and probabilities. We conservatively added and carefully crafted visuals to maintain the look and feel of a traditional spreadsheet while facilitating what-if analyses. Given a user-specified reference cell, Fuzzy Spreadsheet automatically extracts and visualizes contextually relevant information, such as impact, uncertainty, and degree of neighborhood, for the selected and related cells. To evaluate its usability and the perceived mental effort required, we conducted a user study. The results show that our approach outperforms traditional spreadsheets in terms of answer correctness, response time, and perceived mental effort in almost all tasks tested.

Nanotilus: Generator of Immersive Guided-Tours in Crowded 3D Environments.

Immersive virtual reality environments are gaining popularity for studying and exploring crowded three-dimensional structures. When reaching very high structural densities, the natural depiction of the scene produces impenetrable clutter and requires visibility and occlusion management strategies for exploration and orientation. Strategies developed to address the crowdedness in desktop applications, however, inhibit the feeling of immersion. They result in nonimmersive, desktop-style outside-in viewing in virtual reality. This article proposes Nanotilus-a new visibility and guidance approach for very dense environments that generates an endoscopic inside-out experience instead of outside-in viewing, preserving the immersive aspect of virtual reality. The approach consists of two novel, tightly coupled mechanisms that control scene sparsification simultaneously with camera path planning. The sparsification strategy is localized around the camera and is realized as a multi-scale, multi-shell, variety-preserving technique. When Nanotilus dives into the structures to capture internal details residing on multiple scales, it guides the camera using depth-based path planning. In addition to sparsification and path planning, we complete the tour generation with an animation controller, textual annotation, and text-to-visualization conversion. We demonstrate the generated guided tours on mesoscopic biological models - SARS-CoV-2 and HIV. We evaluate the Nanotilus experience with a baseline outside-in sparsification and navigational technique in a formal user study with 29 participants. While users can maintain a better overview using the outside-in sparsification, the study confirms our hypothesis that Nanotilus leads to stronger engagement and immersion.

Watertight Incremental Heightfield Tessellation.

In this article, we propose a method for the interactive visualization of medium-scale dynamic heightfields without visual artifacts. Our data fall into a category too large to be rendered directly at full resolution, but small enough to fit into GPU memory without pre-filtering and data streaming. We present the real-world use case of unfiltered flood simulation data of such medium scale that need to be visualized in real time for scientific purposes. Our solution facilitates compute shaders to maintain a guaranteed watertight triangulation in GPU memory that approximates the interpolated heightfields with view-dependent, continuous levels of detail. In each frame, the triangulation is updated incrementally by iteratively refining the cached result of the previous frame to minimize the computational effort. In particular, we minimize the number of heightfield sampling operations to make adaptive and higher-order interpolations viable options. We impose no restriction on the number of subdivisions and the achievable level of detail to allow for extreme zoom ranges required in geospatial visualization. Our method provides a stable runtime performance and can be executed with a limited time budget. We present a comparison of our method to three state-of-the-art methods, in which our method is competitive to previous non-watertight methods in terms of runtime, while outperforming them in terms of accuracy.

Ten Open Challenges in Medical Visualization.

The medical domain has been an inspiring application area in visualization research for many years already, but many open challenges remain. The driving forces of medical visualization research have been strengthened by novel developments, for example, in deep learning, the advent of affordable VR technology, and the need to provide medical visualizations for broader audiences. At IEEE VIS 2020, we hosted an Application Spotlight session to highlight recent medical visualization research topics. With this article, we provide the visualization community with ten such open challenges, primarily focused on challenges related to the visualization of medical imaging data. We first describe the unique nature of medical data in terms of data preparation, access, and standardization. Subsequently, we cover open visualization research challenges related to uncertainty, multimodal and multiscale approaches, and evaluation. Finally, we emphasize challenges related to users focusing on explainable AI, immersive visualization, P4 medicine, and narrative visualization.

Multi-Scale Procedural Animations of Microtubule Dynamics Based on Measured Data.

Biologists often use computer graphics to visualize structures, which due to physical limitations are not possible to image with a microscope. One example for such structures are microtubules, which are present in every eukaryotic cell. They are part of the cytoskeleton maintaining the shape of the cell and playing a key role in the cell division. In this paper, we propose a scientifically-accurate multi-scale procedural model of microtubule dynamics as a novel application scenario for procedural animation, which can generate visualizations of their overall shape, molecular structure, as well as animations of the dynamic behaviour of their growth and disassembly. The model is spanning from tens of micrometers down to atomic resolution. All the aspects of the model are driven by scientific data. The advantage over a traditional, manual animation approach is that when the underlying data change, for instance due to new evidence, the model can be recreated immediately. The procedural animation concept is presented in its generic form, with several novel extensions, facilitating an easy translation to other domains with emergent multi-scale behavior.

Dynamic Volume Lines: Visual Comparison of 3D Volumes through Space-filling Curves.

The comparison of many members of an ensemble is difficult, tedious, and error-prone, which is aggravated by often just subtle differences. In this paper, we introduce Dynamic Volume Lines for the interactive visual analysis and comparison of sets of 3D volumes. Each volume is linearized along a Hilbert space-filling curve into a 1D Hilbert line plot, which depicts the intensities over the Hilbert indices. We present a nonlinear scaling of these 1D Hilbert line plots based on the intensity variations in the ensemble of 3D volumes, which enables a more effective use of the available screen space. The nonlinear scaling builds the basis for our interactive visualization techniques. An interactive histogram heatmap of the intensity frequencies serves as overview visualization. When zooming in, the frequencies are replaced by detailed 1D Hilbert line plots and optional functional boxplots. To focus on important regions of the volume ensemble, nonlinear scaling is incorporated into the plots. An interactive scaling widget depicts the local ensemble variations. Our brushing and linking interface reveals, for example, regions with a high ensemble variation by showing the affected voxels in a 3D spatial view. We show the applicability of our concepts using two case studies on ensembles of 3D volumes resulting from tomographic reconstruction. In the first case study, we evaluate an artificial specimen from simulated industrial 3D X-ray computed tomography (XCT). In the second case study, a real-world XCT foam specimen is investigated. Our results show that Dynamic Volume Lines can identify regions with high local intensity variations, allowing the user to draw conclusions, for example, about the choice of reconstruction parameters. Furthermore, it is possible to detect ring artifacts in reconstructions volumes.

COZOID: contact zone identifier for visual analysis of protein-protein interactions.

BACKGROUND: Studying the patterns of protein-protein interactions (PPIs) is fundamental for understanding the structure and function of protein complexes. The exploration of the vast space of possible mutual configurations of interacting proteins and their contact zones is very time consuming and requires the proteomic expert knowledge. RESULTS: In this paper, we propose a novel tool containing a set of visual abstraction techniques for the guided exploration of PPI configuration space. It helps proteomic experts to select the most relevant configurations and explore their contact zones at different levels of detail. The system integrates a set of methods that follow and support the workflow of proteomics experts. The first visual abstraction method, the Matrix view, is based on customized interactive heat maps and provides the users with an overview of all possible residue-residue contacts in all PPI configurations and their interactive filtering. In this step, the user can traverse all input PPI configurations and obtain an overview of their interacting amino acids. Then, the models containing a particular pair of interacting amino acids can be selectively picked and traversed. Detailed information on the individual amino acids in the contact zones and their properties is presented in the Contact-Zone list-view. The list-view provides a comparative tool to rank the best models based on the similarity of their contacts to the template-structure contacts. All these techniques are interactively linked with other proposed methods, the Exploded view and the Open-Book view, which represent individual configurations in three-dimensional space. These representations solve the high overlap problem associated with many configurations. Using these views, the structural alignment of the best models can also be visually confirmed. CONCLUSIONS: We developed a system for the exploration of large sets of protein-protein complexes in a fast and intuitive way. The usefulness of our system has been tested and verified on several docking structures covering the three major types of PPIs, including coiled-coil, pocket-string, and surface-surface interactions. Our case studies prove that our tool helps to analyse and filter protein-protein complexes in a fraction of the time compared to using previously available techniques.

Vis-A-Ware: Integrating Spatial and Non-Spatial Visualization for Visibility-Aware Urban Planning.

3D visibility analysis plays a key role in urban planning for assessing the visual impact of proposed buildings on the cityscape. A call for proposals typically yields around 30 candidate buildings that need to be evaluated with respect to selected viewpoints. Current visibility analysis methods are very time-consuming and limited to a small number of viewpoints. Further, analysts neither have measures to evaluate candidates quantitatively, nor to compare them efficiently. The primary contribution of this work is the design study of Vis-A-Ware, a visualization system to qualitatively and quantitatively evaluate, rank, and compare visibility data of candidate buildings with respect to a large number of viewpoints. Vis-A-Ware features a 3D spatial view of an urban scene and non-spatial views of data derived from visibility evaluations, which are tightly integrated by linked interaction. To enable a quantitative evaluation we developed four metrics in accordance with experts from urban planning. We illustrate the applicability of Vis-A-Ware on the basis of a use case scenario and present results from informal feedback sessions with domain experts from urban planning and development. This feedback suggests that Vis-A-Ware is a valuable tool for visibility analysis allowing analysts to answer complex questions more efficiently and objectively.

LiteVis: Integrated Visualization for Simulation-Based Decision Support in Lighting Design.

State-of-the-art lighting design is based on physically accurate lighting simulations of scenes such as offices. The simulation results support lighting designers in the creation of lighting configurations, which must meet contradicting customer objectives regarding quality and price while conforming to industry standards. However, current tools for lighting design impede rapid feedback cycles. On the one side, they decouple analysis and simulation specification. On the other side, they lack capabilities for a detailed comparison of multiple configurations. The primary contribution of this paper is a design study of LiteVis, a system for efficient decision support in lighting design. LiteVis tightly integrates global illumination-based lighting simulation, a spatial representation of the scene, and non-spatial visualizations of parameters and result indicators. This enables an efficient iterative cycle of simulation parametrization and analysis. Specifically, a novel visualization supports decision making by ranking simulated lighting configurations with regard to a weight-based prioritization of objectives that considers both spatial and non-spatial characteristics. In the spatial domain, novel concepts support a detailed comparison of illumination scenarios. We demonstrate LiteVis using a real-world use case and report qualitative feedback of lighting designers. This feedback indicates that LiteVis successfully supports lighting designers to achieve key tasks more efficiently and with greater certainty.

Chameleon: Dynamic Color Mapping for Multi-Scale Structural Biology Models.

Visualization of structural biology data uses color to categorize or separate dense structures into particular semantic units. In multiscale models of viruses or bacteria, there are atoms on the finest level of detail, then amino-acids, secondary structures, macromolecules, up to the compartment level and, in all these levels, elements can be visually distinguished by color. However, currently only single scale coloring schemes are utilized that show information for one particular scale only. We present a novel technology which adaptively, based on the current scale level, adjusts the color scheme to depict or distinguish the currently best visible structural information. We treat the color as a visual resource that is distributed given a particular demand. The changes of the color scheme are seamlessly interpolated between the color scheme from the previous views into a given new one. With such dynamic multi-scale color mapping we ensure that the viewer is able to distinguish structural detail that is shown on any given scale. This technique has been tested by users with an expertise in structural biology and has been overall well received.

Visual analytics and rendering for tunnel crack analysis: A methodological approach for integrating geometric and attribute data.

The visual analysis of surface cracks plays an essential role in tunnel maintenance when assessing the condition of a tunnel. To identify patterns of cracks, which endanger the structural integrity of its concrete surface, analysts need an integrated solution for visual analysis of geometric and multivariate data to decide if issuing a repair project is necessary. The primary contribution of this work is a design study, supporting tunnel crack analysis by tightly integrating geometric and attribute views to allow users a holistic visual analysis of geometric representations and multivariate attributes. Our secondary contribution is Visual Analytics and Rendering, a methodological approach which addresses challenges and recurring design questions in integrated systems. We evaluated the tunnel crack analysis solution in informal feedback sessions with experts from tunnel maintenance and surveying. We substantiated the derived methodology by providing guidelines and linking it to examples from the literature.

The Spinel Explorer--Interactive Visual Analysis of Spinel Group Minerals.

Geologists usually deal with rocks that are up to several thousand million years old. They try to reconstruct the tectonic settings where these rocks were formed and the history of events that affected them through the geological time. The spinel group minerals provide useful information regarding the geological environment in which the host rocks were formed. They constitute excellent indicators of geological environments (tectonic settings) and are of invaluable help in the search for mineral deposits of economic interest. The current workflow requires the scientists to work with different applications to analyze spine data. They do use specific diagrams, but these are usually not interactive. The current workflow hinders domain experts to fully exploit the potentials of tediously and expensively collected data. In this paper, we introduce the Spinel Explorer-an interactive visual analysis application for spinel group minerals. The design of the Spinel Explorer and of the newly introduced interactions is a result of a careful study of geologists' tasks. The Spinel Explorer includes most of the diagrams commonly used for analyzing spinel group minerals, including 2D binary plots, ternary plots, and 3D Spinel prism plots. Besides specific plots, conventional information visualization views are also integrated in the Spinel Explorer. All views are interactive and linked. The Spinel Explorer supports conventional statistics commonly used in spinel minerals exploration. The statistics views and different data derivation techniques are fully integrated in the system. Besides the Spinel Explorer as newly proposed interactive exploration system, we also describe the identified analysis tasks, and propose a new workflow. We evaluate the Spinel Explorer using real-life data from two locations in Argentina: the Frontal Cordillera in Central Andes and Patagonia. We describe the new findings of the geologists which would have been much more difficult to achieve using the current workflow only. Very positive feedback from geologists confirms the usefulness of the Spinel Explorer.

Visual analysis and steering of flooding simulations.

We present a visualization tool for the real-time analysis of interactively steered ensemble-simulation runs, and apply it to flooding simulations. Simulations are performed on-the-fly, generating large quantities of data. The user wants to make sense of the data as it is created. The tool facilitates understanding of what happens in all scenarios, where important events occur, and how simulation runs are related. We combine different approaches to achieve this goal. To maintain an overview, data are aggregated and embedded into the simulation rendering, showing trends, outliers, and robustness. For a detailed view, we use information-visualization views and interactive visual analysis techniques. A selection mechanism connects the two approaches. Points of interest are selected by clicking on aggregates, supplying data for visual analysis. This allows the user to maintain an overview of the ensemble and perform analysis even as new data are supplied through simulation steering. Unexpected or unwanted developments are detected easily, and the user can focus the exploration on them. The solution was evaluated with two case studies focusing on placing and testing flood defense measures. Both were evaluated by a consortium of flood simulation and defense experts, who found the system to be both intuitive and relevant.

High-performance GPU-based rendering for real-time, rigid 2D/3D-image registration and motion prediction in radiation oncology.

A common problem in image-guided radiation therapy (IGRT) of lung cancer as well as other malignant diseases is the compensation of periodic and aperiodic motion during dose delivery. Modern systems for image-guided radiation oncology allow for the acquisition of cone-beam computed tomography data in the treatment room as well as the acquisition of planar radiographs during the treatment. A mid-term research goal is the compensation of tumor target volume motion by 2D/3D Registration. In 2D/3D registration, spatial information on organ location is derived by an iterative comparison of perspective volume renderings, so-called digitally rendered radiographs (DRR) from computed tomography volume data, and planar reference x-rays. Currently, this rendering process is very time consuming, and real-time registration, which should at least provide data on organ position in less than a second, has not come into existence. We present two GPU-based rendering algorithms which generate a DRR of 512×512 pixels size from a CT dataset of 53 MB size at a pace of almost 100 Hz. This rendering rate is feasible by applying a number of algorithmic simplifications which range from alternative volume-driven rendering approaches - namely so-called wobbled splatting - to sub-sampling of the DRR-image by means of specialized raycasting techniques. Furthermore, general purpose graphics processing unit (GPGPU) programming paradigms were consequently utilized. Rendering quality and performance as well as the influence on the quality and performance of the overall registration process were measured and analyzed in detail. The results show that both methods are competitive and pave the way for fast motion compensation by rigid and possibly even non-rigid 2D/3D registration and, beyond that, adaptive filtering of motion models in IGRT.

Semantics by analogy for illustrative volume visualization.

We present an interactive graphical approach for the explicit specification of semantics for volume visualization. This explicit and graphical specification of semantics for volumetric features allows us to visually assign meaning to both input and output parameters of the visualization mapping. This is in contrast to the implicit way of specifying semantics using transfer functions. In particular, we demonstrate how to realize a dynamic specification of semantics which allows to flexibly explore a wide range of mappings. Our approach is based on three concepts. First, we use semantic shader augmentation to automatically add rule-based rendering functionality to static visualization mappings in a shader program, while preserving the visual abstraction that the initial shader encodes. With this technique we extend recent developments that define a mapping between data attributes and visual attributes with rules, which are evaluated using fuzzy logic. Second, we let users define the semantics by analogy through brushing on renderings of the data attributes of interest. Third, the rules are specified graphically in an interface that provides visual clues for potential modifications. Together, the presented methods offer a high degree of freedom in the specification and exploration of rule-based mappings and avoid the limitations of a linguistic rule formulation.

Unified Boundary-Aware Texturing for Interactive Volume Rendering.

In this paper, we describe a novel approach for applying texture mapping to volumetric data sets. In contrast to previous approaches, the presented technique enables a unified integration of 2D and 3D textures and thus allows to emphasize material boundaries as well as volumetric regions within a volumetric data set at the same time. One key contribution of this paper is a parametrization technique for volumetric data sets, which takes into account material boundaries and volumetric regions. Using this technique, the resulting parametrizations of volumetric data sets enable texturing effects which create a higher degree of realism in volume rendered images. We evaluate the quality of the parametrization and demonstrate the usefulness of the proposed concepts by combining volumetric texturing with volumetric lighting models to generate photorealistic volume renderings. Furthermore, we show the applicability in the area of illustrative visualization.

Interactive volume visualization of general polyhedral grids.

This paper presents a novel framework for visualizing volumetric data specified on complex polyhedral grids, without the need to perform any kind of a priori tetrahedralization. These grids are composed of polyhedra that often are non-convex and have an arbitrary number of faces, where the faces can be non-planar with an arbitrary number of vertices. The importance of such grids in state-of-the-art simulation packages is increasing rapidly. We propose a very compact, face-based data structure for representing such meshes for visualization, called two-sided face sequence lists (TSFSL), as well as an algorithm for direct GPU-based ray-casting using this representation. The TSFSL data structure is able to represent the entire mesh topology in a 1D TSFSL data array of face records, which facilitates the use of efficient 1D texture accesses for visualization. In order to scale to large data sizes, we employ a mesh decomposition into bricks that can be handled independently, where each brick is then composed of its own TSFSL array. This bricking enables memory savings and performance improvements for large meshes. We illustrate the feasibility of our approach with real-world application results, by visualizing highly complex polyhedral data from commercial state-of-the-art simulation packages.

World lines.

In this paper we present World Lines as a novel interactive visualization that provides complete control over multiple heterogeneous simulation runs. In many application areas, decisions can only be made by exploring alternative scenarios. The goal of the suggested approach is to support users in this decision making process. In this setting, the data domain is extended to a set of alternative worlds where only one outcome will actually happen. World Lines integrate simulation, visualization and computational steering into a single unified system that is capable of dealing with the extended solution space. World Lines represent simulation runs as causally connected tracks that share a common time axis. This setup enables users to interfere and add new information quickly. A World Line is introduced as a visual combination of user events and their effects in order to present a possible future. To quickly find the most attractive outcome, we suggest World Lines as the governing component in a system of multiple linked views and a simulation component. World Lines employ linking and brushing to enable comparative visual analysis of multiple simulations in linked views. Analysis results can be mapped to various visual variables that World Lines provide in order to highlight the most compelling solutions. To demonstrate this technique we present a flooding scenario and show the usefulness of the integrated approach to support informed decision making.

Visual optimality and stability analysis of 3DCT scan positions.

Industrial cone-beam X-Ray computed tomography (CT) systems often face problems due to artifacts caused by a bad placement of the specimen on the rotary plate. This paper presents a visual-analysis tool for CT systems, which provides a simulation-based preview and estimates artifacts and deviations of a specimen's placement using the corresponding 3D geometrical surface model as input. The presented tool identifies potentially good or bad placements of a specimen and regions of a specimen, which cause the major portion of artefacts. The tool can be used for a preliminary analysis of the specimen before CT scanning, in order to determine the optimal way of placing the object. The analysis includes: penetration lengths, placement stability and an investigation in Radon space. Novel visualization techniques are applied to the simulation data. A stability widget is presented for determining the placement parameters' robustness. The performance and the comparison of results provided by the tool compared with real world data is demonstrated using two specimens.