A theorem proving approach for automatically synthesizing visualizations of flow cytometry data.

BACKGROUND: Polychromatic flow cytometry is a popular technique that has wide usage in the medical sciences, especially for studying phenotypic properties of cells. The high-dimensionality of data generated by flow cytometry usually makes it difficult to visualize. The naive solution of simply plotting two-dimensional graphs for every combination of observables becomes impractical as the number of dimensions increases. A natural solution is to project the data from the original high dimensional space to a lower dimensional space while approximately preserving the overall relationship between the data points. The expert can then easily visualize and analyze this low-dimensional embedding of the original dataset. RESULTS: This paper describes a new method, SANJAY, for visualizing high-dimensional flow cytometry datasets. This technique uses a decision procedure to automatically synthesize two-dimensional and three-dimensional projections of the original high-dimensional data while trying to minimize distortion. We compare SANJAY to the popular multidimensional scaling (MDS) approach for visualization of small data sets drawn from a representative set of benchmarks, and our experiments show that SANJAY produces distortions that are 1.44 to 4.15 times smaller than those caused due to MDS. Our experimental results show that SANJAY also outperforms the Random Projections technique in terms of the distortions in the projections. CONCLUSIONS: We describe a new algorithmic technique that uses a symbolic decision procedure to automatically synthesize low-dimensional projections of flow cytometry data that typically have a high number of dimensions. Our algorithm is the first application, to our knowledge, of using automated theorem proving for automatically generating highly-accurate, low-dimensional visualizations of high-dimensional data.

Enhancing the Sense of Presence in Virtual Reality.

The vision of virtual reality, since Ivan Sutherland's first head mounted device in 1968, has been the re-creation of reality, something indistinguishable from reality, akin to what was depicted in the 1999 film, The Matrix. However, researchers and developers have largely favored visual perception over other senses, leading to the design of virtual worlds that perhaps look real but do not feel real. This favoring of the visual sense and more recently visual and audio senses, overlooks theory in psychology and phenomenology that places embodied action at the center of perception. That is to say, it is the virtual environment's ability to support and enable user actions that impacts perception and perhaps by extension, the user's sense of presence, not just the visual fidelity. Starting with Gibson's approach to action-based perception, we proposed a 4-D framework for creating virtual reality experiences that seamlessly blend extrinsic elements such as the user's real world context with intrinsic elements such as the hardware specifications, application, and interactive content, with the goal to enable a higher sense of presence.

Designing Intelligent Tools for Creative People.

With artificial intelligence (AI) poised to transform the nature of creative media, it is especially important to design tools with the creative process in mind. While ample research demonstrates the importance of flow, playfulness, and exploration for creative tasks, these concepts are rarely considered when designing digital interfaces. My thesis develops principles for the design of intelligent and playful user interfaces through a series of concrete design tasks. I explore a number of approaches for establishing artist needs, develop digital representations that are amenable to both machine learning and user interaction, and design novel digital media that amplify, not stifle, creativity. I conclude with an informal design philosophy developed throughout this study and thoughts on how we can leverage AI to elevate human creativity.

Reliable Contact Simulation With IPC.

Simulating the contact of deformable and possibly thin solids in a robust, accurate, and efficient manner is challenging. Traditionally, the contact of solids is approximately modeled with linearized geometric information near the contacting regions. This approximation is prone to generating underconstrained or overconstrained subproblems that can produce interpenetrating or numerically unstable results, especially when large deformation of solids is also present. To avoid these issues, we propose incremental potential contact (IPC) by formulating a mathematically consistent and general noninterpenetration constraint based on precisely calculated unsigned distances between boundary elements. IPC applies a customized barrier potential to directly relate the distances to the contact forces, which can grow infinitely large as the distance approaches zero to guarantee noninterpenetration. Results show reliable contact simulation even with versatile materials, large timestep sizes, fast impact velocities, severe deformation, and varying boundary conditions-bringing the intricate and important dynamical details to computer graphics in a reliable way for the first time.

Differentiable Visual Computing: Challenges and Opportunities.

Classical algorithms typically contain domain-specific insights. This makes them often more robust, interpretable, and efficient. On the other hand, deep-learning models must learn domain-specific insight from scratch from a large amount of data using gradient-based optimization techniques. To have the best of both worlds, we should make classical visual computing algorithms differentiable to enable gradient-based optimization. Computing derivatives of classical visual computing algorithms is challenging: there can be discontinuities, and the computation pattern is often irregular compared to high-arithmetic intensity neural networks. In this article, we discuss the benefits and challenges of combining classical visual computing algorithms and modern data-driven methods, with particular emphasis to my thesis, which took one of the first steps toward addressing these challenges.

Temporal radiance caching.

We present a novel method for fast, high quality computation of glossy global illumination in animated environments. Building on the irradiance caching and radiance caching algorithms, our method leverages temporal coherence by sparse temporal sampling and interpolation of the indirect lighting. In our approach, part of the global illumination solution computed in previous frames is reused in the current frame. Our reusing scheme adapts to the change of incoming radiance by updating the indirect lighting only where there is a significant change. By reusing data in several frames, our method removes the flickering artifacts and yields a significant speedup compared to classical computation in which a new cache is computed for every frame. We also define temporal gradients for smooth temporal interpolation. A key aspect of our method is the absence of any additional complex data structure, making the implementation into any existing renderer based on irradiance and radiance caching straightforward. We describe the implementation of our method using graphics hardware for improved performance.

Caustics mapping: an image-space technique for real-time caustics.

In this paper, we present a simple and practical technique for real-time rendering of caustics from reflective and refractive objects. Our algorithm, conceptually similar to shadow mapping, consists of two main parts: creation of a caustic map texture, and utilization of the map to render caustics onto nonshiny surfaces. Our approach avoids performing any expensive geometric tests, such as ray-object intersection, and involves no precomputation; both of which are common features in previous work. The algorithm is well suited for the standard rasterization pipeline and runs entirely on the graphics hardware.

Radiance caching for efficient global illumination computation.

In this paper, we present a ray tracing-based method for accelerated global illumination computation in scenes with low-frequency glossy BRDFs. The method is based on sparse sampling, caching, and interpolating radiance on glossy surfaces. In particular, we extend the irradiance caching scheme proposed by Ward et al. to cache and interpolate directional incoming radiance instead of irradiance. The incoming radiance at a point is represented by a vector of coefficients with respect to a hemispherical or spherical basis. The surfaces suitable for interpolation are selected automatically according to the roughness of their BRDF. We also propose a novel method for computing translational radiance gradient at a point.

A parallel architecture for interactively rendering scattering and refraction effects.

A new method for interactive rendering of complex lighting effects combines two algorithms. The first performs accurate ray tracing in heterogeneous refractive media to compute high-frequency phenomena. The second applies lattice-Boltzmann lighting to account for low-frequency multiple-scattering effects. The two algorithms execute in parallel on modern graphics hardware. This article includes a video animation of the authors' real-time algorithm rendering a variety of scenes.

Rendering grass in real time with dynamic lighting.

The authors present a real-time grass rendering technique that works for large, arbitrary terrains with dynamic lighting, shadows, and a good parallax effect. A novel combination of geometry and lit volume slices provides accurate, per-pixel lighting. A fast grass-density management scheme allows the rendering of arbitrarily shaped patches of grass.

Image-space subsurface scattering for interactive rendering of deformable translucent objects.

The authors present an algorithm for real-time realistic rendering of translucent materials such as marble, wax, and milk. Their method captures subsurface scattering effects while maintaining interactive frame rates. The main idea of this work is that it employs the dipole diffusion model with a splatting approach to evaluate the integral over the surface area for computing illumination due to multiple scattering.