Fabrication of moiré on curved surfaces.

Moiré is an appealing visual effect observable when two or more repetitive patterns are superposed. Fabrication of moiré effects has already proven to be useful in a range of applications, from art to engineering. Here, we introduce a method for designing and fabricating level-line moirés on curved surfaces. These moiré shapes are obtained by superposing a partly absorbing layer and a layer formed by an array of cylindrical lenses or by two layers of cylindrical lenses. We formulate the problem of placing an array of cylindrical lenses on a curved surface as a design problem with a small number of dimensions. The range of possible solutions can therefore be explored by a human observer. We demonstrate the quality of our method by rendered simulations and by fabrication. The resulting static displays can be manufactured using different fabrication techniques, from multi-material 3D printing to molding.

Mo2Cap2: Real-time Mobile 3D Motion Capture with a Cap-mounted Fisheye Camera.

We propose the first real-time system for the egocentric estimation of 3D human body pose in a wide range of unconstrained everyday activities. This setting has a unique set of challenges, such as mobility of the hardware setup, and robustness to long capture sessions with fast recovery from tracking failures. We tackle these challenges based on a novel lightweight setup that converts a standard baseball cap to a device for high-quality pose estimation based on a single cap-mounted fisheye camera. From the captured egocentric live stream, our CNN based 3D pose estimation approach runs at 60 Hz on a consumer-level GPU. In addition to the lightweight hardware setup, our other main contributions are: 1) a large ground truth training corpus of top-down fisheye images and 2) a disentangled 3D pose estimation approach that takes the unique properties of the egocentric viewpoint into account. As shown by our evaluation, we achieve lower 3D joint error as well as better 2D overlay than the existing baselines.

A Perception-driven Hybrid Decomposition for Multi-layer Accommodative Displays.

Multi-focal plane and multi-layered light-field displays are promising solutions for addressing all visual cues observed in the real world. Unfortunately, these devices usually require expensive optimizations to compute a suitable decomposition of the input light field or focal stack to drive individual display layers. Although these methods provide near-correct image reconstruction, a significant computational cost prevents real-time applications. A simple alternative is a linear blending strategy which decomposes a single 2D image using depth information. This method provides real-time performance, but it generates inaccurate results at occlusion boundaries and on glossy surfaces. This paper proposes a perception-based hybrid decomposition technique which combines the advantages of the above strategies and achieves both real-time performance and high-fidelity results. The fundamental idea is to apply expensive optimizations only in regions where it is perceptually superior, e.g., depth discontinuities at the fovea, and fall back to less costly linear blending otherwise. We present a complete, perception-informed analysis and model that locally determine which of the two strategies should be applied. The prediction is later utilized by our new synthesis method which performs the image decomposition. The results are analyzed and validated in user experiments on a custom multi-plane display.

Perceptual Real-Time 2D-to-3D Conversion Using Cue Fusion.

We propose a system to infer binocular disparity from a monocular video stream in real-time. Different from classic reconstruction of physical depth in computer vision, we compute perceptually plausible disparity, that is numerically inaccurate, but results in a very similar overall depth impression with plausible overall layout, sharp edges, fine details and agreement between luminance and disparity. We use several simple monocular cues to estimate disparity maps and confidence maps of low spatial and temporal resolution in real-time. These are complemented by spatially-varying, appearance-dependent and class-specific disparity prior maps, learned from example stereo images. Scene classification selects this prior at runtime. Fusion of prior and cues is done by means of robust MAP inference on a dense spatio-temporal conditional random field with high spatial and temporal resolution. Using normal distributions allows this in constant-time, parallel per-pixel work. We compare our approach to previous 2D-to-3D conversion systems in terms of different metrics, as well as a user study and validate our notion of perceptually plausible disparity.

Expressive Single Scattering for Light Shaft Stylization.

Light scattering in participating media is a natural phenomenon that is increasingly featured in movies and games, as it is visually pleasing and lends realism to a scene. In art, it may further be used to express a certain mood or emphasize objects. Here, artists often rely on stylization when creating scattering effects, not only because of the complexity of physically correct scattering, but also to increase expressiveness. Little research, however, focuses on artistically influencing the simulation of the scattering process in a virtual 3D scene. We propose novel stylization techniques, enabling artists to change the appearance of single scattering effects such as light shafts. Users can add, remove, or enhance light shafts using occluder manipulation. The colors of the light shafts can be stylized and animated using easily modifiable transfer functions. Alternatively, our system can optimize a light map given a simple user input for a number of desired views in the 3D world. Finally, we enable artists to control the heterogeneity of the underlying medium. Our stylized scattering solution is easy to use and compatible with standard rendering pipelines. It works for animated scenes and can be executed in real time to provide the artist with quick feedback.

Multi-field Pattern Matching based on Sparse Feature Sampling.

We present an approach to pattern matching in 3D multi-field scalar data. Existing pattern matching algorithms work on single scalar or vector fields only, yet many numerical simulations output multi-field data where only a joint analysis of multiple fields describes the underlying phenomenon fully. Our method takes this into account by bundling information from multiple fields into the description of a pattern. First, we extract a sparse set of features for each 3D scalar field using the 3D SIFT algorithm (Scale-Invariant Feature Transform). This allows for a memory-saving description of prominent features in the data with invariance to translation, rotation, and scaling. Second, the user defines a pattern as a set of SIFT features in multiple fields by e.g. brushing a region of interest. Third, we locate and rank matching patterns in the entire data set. Experiments show that our algorithm is efficient in terms of required memory and computational efforts.

MovExp: A Versatile Visualization Tool for Human-Computer Interaction Studies with 3D Performance and Biomechanical Data.

In Human-Computer Interaction (HCI), experts seek to evaluate and compare the performance and ergonomics of user interfaces. Recently, a novel cost-efficient method for estimating physical ergonomics and performance has been introduced to HCI. It is based on optical motion capture and biomechanical simulation. It provides a rich source for analyzing human movements summarized in a multidimensional data set. Existing visualization tools do not sufficiently support the HCI experts in analyzing this data. We identified two shortcomings. First, appropriate visual encodings are missing particularly for the biomechanical aspects of the data. Second, the physical setup of the user interface cannot be incorporated explicitly into existing tools. We present MovExp, a versatile visualization tool that supports the evaluation of user interfaces. In particular, it can be easily adapted by the HCI experts to include the physical setup that is being evaluated, and visualize the data on top of it. Furthermore, it provides a variety of visual encodings to communicate muscular loads, movement directions, and other specifics of HCI studies that employ motion capture and biomechanical simulation. In this design study, we follow a problem-driven research approach. Based on a formalization of the visualization needs and the data structure, we formulate technical requirements for the visualization tool and present novel solutions to the analysis needs of the HCI experts. We show the utility of our tool with four case studies from the daily work of our HCI experts.

Fast and Memory-Efficient Topological Denoising of 2D and 3D Scalar Fields.

Data acquisition, numerical inaccuracies, and sampling often introduce noise in measurements and simulations. Removing this noise is often necessary for efficient analysis and visualization of this data, yet many denoising techniques change the minima and maxima of a scalar field. For example, the extrema can appear or disappear, spatially move, and change their value. This can lead to wrong interpretations of the data, e.g., when the maximum temperature over an area is falsely reported being a few degrees cooler because the denoising method is unaware of these features. Recently, a topological denoising technique based on a global energy optimization was proposed, which allows the topology-controlled denoising of 2D scalar fields. While this method preserves the minima and maxima, it is constrained by the size of the data. We extend this work to large 2D data and medium-sized 3D data by introducing a novel domain decomposition approach. It allows processing small patches of the domain independently while still avoiding the introduction of new critical points. Furthermore, we propose an iterative refinement of the solution, which decreases the optimization energy compared to the previous approach and therefore gives smoother results that are closer to the input. We illustrate our technique on synthetic and real-world 2D and 3D data sets that highlight potential applications.

Property and Lighting Manipulations for Static Volume Stylization Using a Painting Metaphor.

Although volumetric phenomena are important for realistic rendering and can even be a crucial component in the image, the artistic control of the volume's appearance is challenging. Appropriate tools to edit volume properties are missing, which can make it necessary to use simulation results directly. Alternatively, high-level modifications that are rarely intuitive, e.g., the tweaking of noise function parameters, can be utilized. Our work introduces a solution to stylize single-scattering volumetric effects in static volumes. Hereby, an artistic and intuitive control of emission, scattering and extinction becomes possible, while ensuring a smooth and coherent appearance when changing the viewpoint. Our method is based on tomographic reconstruction, which we link to the volumetric rendering equation. It analyzes a number of target views provided by the artist and adapts the volume properties to match the appearance for the given perspectives. Additionally, we describe how we can optimize for the environmental lighting to match a desired scene appearance, while keeping volume properties constant. Finally, both techniques can be combined. We demonstrate several use cases of our approach and illustrate its effectiveness.

Markerless motion capture of multiple characters using multiview image segmentation.

Capturing the skeleton motion and detailed time-varying surface geometry of multiple, closely interacting peoples is a very challenging task, even in a multicamera setup, due to frequent occlusions and ambiguities in feature-to-person assignments. To address this task, we propose a framework that exploits multiview image segmentation. To this end, a probabilistic shape and appearance model is employed to segment the input images and to assign each pixel uniquely to one person. Given the articulated template models of each person and the labeled pixels, a combined optimization scheme, which splits the skeleton pose optimization problem into a local one and a lower dimensional global one, is applied one by one to each individual, followed with surface estimation to capture detailed nonrigid deformations. We show on various sequences that our approach can capture the 3D motion of humans accurately even if they move rapidly, if they wear wide apparel, and if they are engaged in challenging multiperson motions, including dancing, wrestling, and hugging.

Perceptually driven visibility optimization for categorical data visualization.

Visualization techniques often use color to present categorical differences to a user. When selecting a color palette, the perceptual qualities of color need careful consideration. Large coherent groups visually suppress smaller groups and are often visually dominant in images. This paper introduces the concept of class visibility used to quantitatively measure the utility of a color palette to present coherent categorical structure to the user. We present a color optimization algorithm based on our class visibility metric to make categorical differences clearly visible to the user. We performed two user experiments on user preference and visual search to validate our visibility measure over a range of color palettes. The results indicate that visibility is a robust measure, and our color optimization can increase the effectiveness of categorical data visualizations.

Real-time screen-space scattering in homogeneous environments.

The proposed screen-space algorithm approximates light scattering in homogeneous participating environments, such as water. Instead of simulating full global illumination, this method models scattering by a physically based point spread function. A discrete hierarchical convolution in a texture MIP map makes the algorithm efficient, and a custom anisotropic incremental filter prevents illumination leaking.

Precomputed Safety Shapes for Efficient and Accurate Height-Field Rendering.

Height fields have become an important element of realistic real-time image synthesis to represent surface details. In this paper, we focus on the frequent case of static height-field data, for which we can precompute acceleration structures. While many rendering algorithms exist that impose tradeoffs between speed and accuracy, we show that even accurate rendering can be combined with high performance. A careful analysis of the surface defined by the height values, leads to an efficient and accurate precomputation method. As a result, each texel stores a safety shape inside which a ray cannot cross the surface twice. This property ensures that no intersections are missed during the efficient marching method. Our analysis is general and can even consider visibility constraints that are robustly integrated into the precomputation. Further, we propose a particular instance of safety shapes with little memory overhead, which results in a rendering algorithm that outperforms existing methods, both in terms of accuracy and performance.

Crease surfaces: from theory to extraction and application to diffusion tensor MRI.

Crease surfaces are two-dimensional manifolds along which a scalar field assumes a local maximum (ridge) or a local minimum (valley) in a constrained space. Unlike isosurfaces, they are able to capture extremal structures in the data. Creases have a long tradition in image processing and computer vision, and have recently become a popular tool for visualization. When extracting crease surfaces, degeneracies of the Hessian (i.e., lines along which two eigenvalues are equal) have so far been ignored. We show that these loci, however, have two important consequences for the topology of crease surfaces: First, creases are bounded not only by a side constraint on eigenvalue sign, but also by Hessian degeneracies. Second, crease surfaces are not, in general, orientable. We describe an efficient algorithm for the extraction of crease surfaces which takes these insights into account and demonstrate that it produces more accurate results than previous approaches. Finally, we show that diffusion tensor magnetic resonance imaging (DT-MRI) stream surfaces, which were previously used for the analysis of planar regions in diffusion tensor MRI data, are mathematically ill-defined. As an example application of our method, creases in a measure of planarity are presented as a viable substitute.

Smoke surfaces: an interactive flow visualization technique inspired by real-world flow experiments.

Smoke rendering is a standard technique for flow visualization. Most approaches are based on a volumetric, particle based, or image based representation of the smoke. This paper introduces an alternative representation of smoke structures: as semi-transparent streak surfaces. In order to make streak surface integration fast enough for interactive applications, we avoid expensive adaptive retriangulations by coupling the opacity of the triangles to their shapes. This way, the surface shows a smoke-like look even in rather turbulent areas. Furthermore, we show modifications of the approach to mimic smoke nozzles, wool tufts, and time surfaces. The technique is applied to a number of test data sets.

Estimating crossing fibers: a tensor decomposition approach.

Diffusion weighted magnetic resonance imaging is a unique tool for non-invasive investigation of major nerve fiber tracts. Since the popular diffusion tensor (DT-MRI) model is limited to voxels with a single fiber direction, a number of high angular resolution techniques have been proposed to provide information about more diverse fiber distributions. Two such approaches are Q-Ball imaging and spherical deconvolution, which produce orientation distribution functions (ODFs) on the sphere. For analysis and visualization, the maxima of these functions have been used as principal directions, even though the results are known to be biased in case of crossing fiber tracts. In this paper, we present a more reliable technique for extracting discrete orientations from continuous ODFs, which is based on decomposing their higher-order tensor representation into an isotropic component, several rank-1 terms, and a small residual. Comparing to ground truth in synthetic data shows that the novel method reduces bias and reliably reconstructs crossing fibers which are not resolved as individual maxima in the ODF. We present results on both Q-Ball and spherical deconvolution data and demonstrate that the estimated directions allow for plausible fiber tracking in a real data set.

Visualizing transport structures of time-dependent flow fields.

This article focuses on the transport characteristics of physical properties in fluids-in particular, visualizing the finite-time transport structure of property advection. Applied to a well-chosen set of property fields, the proposed approach yields structures giving insights into the underlying flow's dynamic processes.

Meshless Modeling of Deformable Shapes and their Motion.

We present a new framework for interactive shape deformation modeling and key frame interpolation based on a meshless finite element formulation. Starting from a coarse nodal sampling of an object's volume, we formulate rigidity and volume preservation constraints that are enforced to yield realistic shape deformations at interactive frame rates. Additionally, by specifying key frame poses of the deforming shape and optimizing the nodal displacements while targeting smooth interpolated motion, our algorithm extends to a motion planning framework for deformable objects. This allows reconstructing smooth and plausible deformable shape trajectories in the presence of possibly moving obstacles. The presented results illustrate that our framework can handle complex shapes at interactive rates and hence is a valuable tool for animators to realistically and efficiently model and interpolate deforming 3D shapes.

Topological visualization of brain diffusion MRI data.

Topological methods give concise and expressive visual representations of flow fields. The present work suggests a comparable method for the visualization of human brain diffusion MRI data. We explore existing techniques for the topological analysis of generic tensor fields, but find them inappropriate for diffusion MRI data. Thus, we propose a novel approach that considers the asymptotic behavior of a probabilistic fiber tracking method and define analogs of the basic concepts of flow topology, like critical points, basins, and faces, with interpretations in terms of brain anatomy. The resulting features are fuzzy, reflecting the uncertainty inherent in any connectivity estimate from diffusion imaging. We describe an algorithm to extract the new type of features, demonstrate its robustness under noise, and present results for two regions in a diffusion MRI dataset to illustrate that the method allows a meaningful visual analysis of probabilistic fiber tracking results.

Seeing people in different light--joint shape, motion, and reflectance capture.

By means of passive optical motion capture, real people can be authentically animated and photo-realistically textured. To import real-world characters into virtual environments, however, surface reflectance properties must also be known. We describe a video-based modeling approach that captures human shape and motion as well as reflectance characteristics from a handful of synchronized video recordings. The presented method is able to recover spatially varying surface reflectance properties of clothes from multiview video footage. The resulting model description enables us to realistically reproduce the appearance of animated virtual actors under different lighting conditions, as well as to interchange surface attributes among different people, e.g., for virtual dressing. Our contribution can be used to create 3D renditions of real-world people under arbitrary novel lighting conditions on standard graphics hardware.