To Authenticity, and Beyond! Building Safe and Fair Generative AI Upon the Three Pillars of Provenance.

Provenance facts, such as who made an image and how, can provide valuable context for users to make trust decisions about visual content. Against a backdrop of inexorable progress in generative AI for computer graphics, over two billion people will vote in public elections this year. Emerging standards and provenance enhancing tools promise to play an important role in fighting fake news and the spread of misinformation. In this article, we contrast three provenance enhancing technologies-metadata, fingerprinting, and watermarking-and discuss how we can build upon the complementary strengths of these three pillars to provide robust trust signals to support stories told by real and generative images. Beyond authenticity, we describe how provenance can also underpin new models for value creation in the age of generative AI. In doing so, we address other risks arising with generative AI such as ensuring training consent, and the proper attribution of credit to creatives who contribute their work to train generative models. We show that provenance may be combined with distributed ledger technology to develop novel solutions for recognizing and rewarding creative endeavor in the age of generative AI.

A Workflow to Visually Assess Interobserver Variability in Medical Image Segmentation.

We introduce a workflow for the visual assessment of interobserver variability in medical image segmentation. Image segmentation is a crucial step in the diagnosis, prognosis, and treatment of many diseases. Despite the advancements in autosegmentation, clinical practice widely relies on manual delineations performed by radiologists. Our work focuses on designing a solution for understanding the radiologists' thought processes during segmentation and for unveiling reasons that lead to interobserver variability. To this end, we propose a visual analysis tool connecting multiple radiologists' delineation processes with their outcomes, and we demonstrate its potential in a case study.

Cuttlefish: Pushing the Limits of Graphical 3-D Printing.

This article presents Cuttlefish, a 3-D printer driver developed at the Fraunhofer Institute for Computer Graphics Research IGD. Cuttlefish maximizes the reproduction quality of shape and appearance in multimaterial 3-D printing systems. It controls various printing systems and material sets, including color, transparent, and flexible materials. The article provides an overview of Cuttlefish's graphical 3-D printing pipeline and focuses on its unique features: Displaced signed distance fields for robust voxelization of objects not designed for 3-D printing, joint color and translucency reproduction, accurate color and translucency characterization, shape dithering to eliminate quantization-based staircase artifacts, and support for displacement maps and constructive solid geometry operations. The article concludes by showcasing applications of Cuttlefish in printing replacement faces for stop-motion animation, large-scale figurine production, and prosthetic eye printing.

The Long and Winding Road: 25 Years of the National Advanced Driving Simulator.

The National Advanced Driving Simulator is a high-fidelity motion-base simulator owned by the National Highway Transportation Safety Administration and managed and operated by the University of Iowa. Its 25-year history has intersected with some of the most significant developments in automotive history, such as advanced driver assistance systems like stability control and collision warning systems, and highly automated vehicles. The simulator is an application of immersive virtual reality that uses multiprojection instead of head-mounted displays. A large-excursion motion system provides realistic acceleration and rotation cues to the driver. Due to its level of immersion and realism, drivers respond to events in the simulator the same way they would in their own vehicle. We document the history and technology behind this national facility.

Development of a Virtual Reality Tool for the Treatment of Pediatric Patients in the ICU.

Prolonged stays in the intensive care unit (ICU) cause difficulties in rehabilitation and other disorders for patients. This problem is exacerbated in the case of pediatric patients. The use of virtual reality can help with the lack of external stimuli and contribute as potential nonpharmacological therapies in some patient rehabilitation processes. To this end, we have developed a virtual reality application for use in the pediatric ICU as a tool for the treatment and rehabilitation of delirium. The tool consists of two applications: an immersive environment for a virtual reality headset used by the patient, and a web application managed by a therapist with which they can customize, control, adapt, and analyze in real time everything that happens in the patient's virtual world. Our application has been designed jointly with a university center and a hospital, and initial evaluations indicate the results to be promising.

Evaluation of an Immersive COVID-19 Data Visualization.

COVID-19 restrictions have detrimental effects on the population, both socially and economically. However, these restrictions are necessary as they help reduce the spread of the virus. For the public to comply, easily comprehensible communication between decision makers and the public is thus crucial. To address this, we propose a novel 3-D visualization of COVID-19 data, which could increase the awareness of COVID-19 trends in the general population. We conducted a user study and compared a conventional 2-D visualization with the proposed method in an immersive environment. Results showed that the our 3-D visualization approach facilitated understanding of the complexity of COVID-19. A majority of participants preferred to see the COVID-19 data with the 3-D method. Moreover, individual results revealed that our method increases the engagement of users with the data. We hope that our method will help governments to improve their communication with the public in the future.

The Making of a Newspaper Interview in Virtual Reality: Realistic Avatars, Philosophy, and Sushi.

VR United is a virtual reality application that we have developed to support multiple people simultaneously interacting in the same environment. Each person is represented with a virtual body that looks like themselves. Such immersive shared environments have existed and been the subject of research for the past 30 years. Here, we demonstrate how VR United meets criteria for successful interaction, where a journalist from the Financial Times in London interviewed a professor in New York for two hours. The virtual location of the interview was a restaurant, in line with the series of interviews published as "Lunch with the FT." We show how the interview was successful, as a substitute for a physically present one. The article based on the interview was published in the Financial Times as normal for the series. We finally consider the future development of such systems, including some implications for immersive journalism.

Virtual Reality Training for Post-Earthquake Rescue Operators.

Urban Search And Rescue (USAR) personnel need to be appropriately trained to effectively operate in post-disaster conditions, as they need to quickly identify locations where the presence of survivors is more likely. Currently, the training procedure for this kind of triage is based on showing static pictures presenting different types of building collapses accompanied by cards providing additional environmental information. This article presents VRescue, a simulator for training USAR operators using the paradigm of immersive virtual reality (VR). VRescue recreates a range of operation conditions (day/night, presence of people, dangerous locations, etc.) experienced by real rescuers, and enables learning the use of the equipment commonly used in such scenarios.

Podiy: A System for Design and Production of Pouches by Novices.

We have developed a system called "Podiy" that supports handicraft production of pouch-style bags using three-dimensional (3-D) computer graphics. We divide the making of a pouch into four steps: pouch design, fabric design, pattern design, and production, each supported by different panels. In pouch design mode, the user can design both the size and pattern. In fabric design mode, the user can design two types of checkered patterns. In cloth cutout mode, the user can choose from three ways of printing the pattern onto the fabric. In production mode, the system shows the user a 3-D illustration view using the user-designed fabrics. Using Podiy, even a handicraft novice can design an original pouch and produce it successfully.

Explainability of Text Clustering Visualizations-Twitter Disinformation Case Study.

While text clustering methods have been available for decades, there is a paucity of material that would help practitioners with the choice and configuration of suitable algorithms and visualizations. In this article, we present a case study analyzing two disinformation datasets composed of tweets from the era of the 2016 United States Presidential Election. We use this to demonstrate steps for selecting the best configuration of the clustering algorithm and consequently conduct a user experiment for evaluating the comprehensibility of three alternate visualizations. A supplementary GitHub repository contains source code with examples.

Evaluating Representation Learning and Graph Layout Methods for Visualization.

Graphs and other structured data have come to the forefront in machine learning over the past few years due to the efficacy of novel representation learning methods boosting the prediction performance in various tasks. Representation learning methods embed the nodes in a low-dimensional real-valued space, enabling the application of traditional machine learning methods on graphs. These representations have been widely premised to be also suited for graph visualization. However, no benchmarks or encompassing studies on this topic exist. We present an empirical study comparing several state-of-the-art representation learning methods with two recent graph layout algorithms, using readability and distance-based measures as well as the link prediction performance. Generally, no method consistently outperformed the others across quality measures. The graph layout methods provided qualitatively superior layouts when compared to representation learning methods. Embedding graphs in a higher dimensional space and applying t-distributed stochastic neighbor embedding for visualization improved the preservation of local neighborhoods, albeit at substantially higher computational cost.

BoidVR: An Agent Simulation Environment Based on Freehand and Virtual Reality.

Freehand-based interaction techniques (FITs) are an emerging technology of importance for effective virtual reality (VR). In this regard, most modern head-mounted displays, such as the Oculus Quest, are equipped with inside-out cameras for egocentric hand tracking and natural hand gestures recognition. Following this technological evolution, we have developed an agent-based modeling and simulation VR application, called BoidVR, focused on freehand-based interaction with simulated agents. Our system enables users to view and interact with both the agents and virtual environment using FIT-based tools. We illustrate a number of specific gestures that allow users to manipulate the virtual environment and affect agent behaviors. Finally, we conduct a user experiment in which we evaluate the usability and user sentiment of our BoidVR application.

Role of Intricate Pottery Visualization in Ceramic Manufacturing.

Layered manufacturing, the underlying technology of 3-D printing, has made rapid strides over the last 30 years. We discuss layered manufacturing from the artist's perspective, especially for intricate ceramic pottery. We contend that opportunities exist for applying visualization to the foremost problems plaguing layered manufacturing. Virtual pottery involves meeting two conflicting constraints: rapid visualization during modeling and accurate rapid prototyping during manufacturing. Artists simultaneously need both low polygon shape representation for interactive visualization and adequate representation for generating accurate printable models. Artists also face the additional complexities of adding surface details that cannot be achieved by hand and handling materials like clay used in the manufacturing of real pottery. Illustrated by a system we have developed that uses sound resonance patterns to create volumetric textures for virtual pottery, we show how visualization helps address both these problem areas.

Scaling the Virtual Fitness Buddy Ecosystem as a School-Based Physical Activity Intervention for Children.

Childhood obesity is a growing concern as it can lead to lifelong health problems that carry over into adulthood. A substantial contributing factor to obesity is the physical activity (PA) habits that are formed in early childhood, as these habits tend to sustain throughout adulthood. To aid children in forming healthy PA habits, we designed a mixed reality system called the Virtual Fitness Buddy ecosystem, in which children can interact with a virtual pet agent. As a child exercises, their pet becomes slimmer, faster, and able to play more games with them. Our initial deployment of this project showed promise but was only designed for a short-term intervention lasting three days. More recently, we have scaled it from a pilot grade study to a 9-month intervention comprised of 422 children. Ultimately, our goal is to scale this project to be a nationwide primary prevention program to encourage moderate to vigorous PA in children. This article explores the challenges and lessons learned during the design and deployment of this system at scale in the field.

Saliency Computation for Virtual Cinematography in 360° Videos.

Recent advances in virtual reality cameras have contributed to a phenomenal growth of 360$^{\circ }$∘ videos. Estimating regions likely to attract user attention is critical for efficiently streaming and rendering 360$^{\circ }$∘ videos. In this article, we present a simple, novel, GPU-driven pipeline for saliency computation and virtual cinematography in 360$^{\circ }$∘ videos using spherical harmonics (SH). We efficiently compute the 360$^{\circ }$∘ video saliency through the spectral residual of the SH coefficients between multiple bands at over 60FPS for 4K resolution videos. Further, our interactive computation of spherical saliency can be used for saliency-guided virtual cinematography in 360$^{\circ }$∘ videos.

Visualizing Multimodal Deep Learning for Lesion Prediction.

A U-Net is a type of convolutional neural network that has been shown to output impressive results in medical imaging segmentation tasks. Still, neural networks in general form a black box that is hard to interpret, especially by noncomputer scientists. This work provides a visual system that allows users to examine U-Nets that were trained to predict brain lesions caused by stroke using multimodal imaging. We provide several visualization views that allow users to load trained U-Nets, run them on different patient data, and examine the results while visually following the computation of the U-Net. With these visualizations, we can provide useful information for our medical collaborators showing how the training database can be improved and which features are best learned by the neural network.

Virtual Reality Application for Fostering Interest in Art.

There is concern within the cultural community about the decreasing number of people, especially young people, visiting museums and cultural institutions. To fight this trend, new ways of accessing art are needed. We believe that technologies like virtual reality can greatly foster the relationship between users and works of art. In this article, we present an elaborated virtual reality experience, ImmersArt, whose aim is to take users literally inside a work of art and the creative universe of its creator. The motivational impact of this tool was empirically analyzed and compared with other formats for experiencing art such as illustrated books and audiovisual presentations. The study suggests that young people are more motivated to study artwork when using virtual reality.

The Dreamcatcher: Interactive Storytelling of Dreams.

Sleep scientists have extensively validated the continuity hypothesis, according to which our dreams reflect what happens during our waking life. Yet, only a few attempts have been made to increase the general public's awareness about the benefits of dream analysis in better understanding and improving our daily life. We designed "The Dreamcatcher," an interactive visual tool that explores the link between dreams and waking life through a collection of dream reports. We conducted a user study with 154 participants and found a 25% increase in the number of people believing that dream analysis can improve our daily lives after interacting with our tool. The visualization informed people about the potential of the continuity hypothesis to a surprising extent, to the point that it increased their concerns about sharing their own dream reports, thus opening new questions on how to design privacy-aware tools for dream collection.

Antarctic Water Masses and Ice Shelves: Visualizing the Physics.

High-resolution simulation of global climate physics enables us to model how the climate may change under a variety of future scenarios. Such simulations produce vast amounts of information and dense datasets. If interrogated in tandem, these datasets can provide holistic, vital information on Earth's many integrated systems by revealing the manifold interrelated properties of the atmosphere, ocean, and polar ice, framed by real-world terrain in three-dimensional space as they vary over time. To accomplish this, climate scientists have joined with computer scientists and an artist to develop techniques enabling scientists to see these relationships. The impact of ocean water properties on Antarctic ice shelves illustrates the benefit of this analysis in understanding land ice melt rates and thus sea-level rise.

Measurement and Inspection of Photo-Realistic 3-D VR Models.

Recent advancements in virtual reality (VR) may help unlock the full potential offered by 3-D photorealistic models generated using state-of-the-art photogrammetric methods. Using VR to carry out analyses on photogrammetric models has the potential to assist the user in performing basic offline engineering inspection of digital twins-digitized representations of real-world objects and structures. However, for such benefits to materialize, it is necessary to create suitable interactive systems for working with photogrammetric models in VR. To this end, this article presents PhotoTwinVR-an immersive gesture-controlled system for manipulation and inspection of 3-D photogrammetric models of physical objects in VR. An observational study with three domain experts validates the feasibility of the system design for practical use-cases involving offline inspections of pipelines and other 3-D structures.