It Sounds Cool: Exploring Sonification of Mid-Air Haptic Textures Exploration on Texture Judgments, Body Perception, and Motor Behaviour.

Ultrasonic mid-air haptic technology allows for the perceptual rendering of textured surfaces onto the user's hand. Unlike real textured surfaces, however, mid-air haptic feedback lacks implicit multisensory cues needed to reliably infer a texture's attributes (e.g., its roughness). In this paper, we combined mid-air haptic textures with congruent sound feedback to investigate how sonification could influence people's (1) explicit judgment of the texture attributes, (2) explicit sensations of their own hand, and (3) implicit motor behavior during haptic exploration. Our results showed that audio cues (presented solely or combined with haptics) influenced participants' judgment of the texture attributes (roughness, hardness, moisture and viscosity), produced some hand sensations (the feeling of having a hand smoother, softer, looser, more flexible, colder, wetter and more natural), and changed participants' speed (moving faster or slower) while exploring the texture. We then conducted a principal component analysis to better understand and visualize the found results and conclude with a short discussion on how audio-haptic associations can be used to create embodied experiences in emerging application scenarios in the metaverse.

Scent-delivery devices as a digital healthcare tool for olfactory training: A pilot focus group study in Parkinson's disease patients.

Parkinson's disease (PD) patients display a combination of motor and non-motor symptoms. The most common non-motor symptom is scent (olfactory) impairment, occurring at least four years prior to motor symptom onset. Recent and growing interest in digital healthcare technology used in PD has resulted in more technologies developed for motor rather than non-motor symptoms. Human-computer interaction (HCI), which uses computer technology to explore human activity and work, could be combined with digital healthcare technologies to better understand and support olfaction via scent training - leading to the development of a scent-delivery device (SDD). In this pilot study, three PD patients were invited to an online focus group to explore the association between PD and olfaction, understand HCI and sensory technologies and were demonstrated a new multichannel SDD with an associated mobile app. Participants had a preconceived link, a result of personal experience, between olfactory impairment and PD. Participants felt that healthcare professionals did not take olfactory dysfunction concerns seriously prior to PD diagnosis. Two were not comfortable with sharing scent loss experiences with others. Participants expected the multichannel SDD to be small, portable and easy-to-use, with customisable cartridges to deliver chosen scents and the mobile app to create a sense of community. None of the participants regularly performed scent training but would consider doing so if some scent function could be regained. Standardised digital SDDs for regular healthcare check-ups may facilitate improvement in olfactory senses in PD patients and potential earlier PD diagnosis, allowing earlier therapeutic and symptomatic PD management.

The sense of agency in emerging technologies for human-computer integration: A review.

Human-computer integration is an emerging area in which the boundary between humans and technology is blurred as users and computers work collaboratively and share agency to execute tasks. The sense of agency (SoA) is an experience that arises by a combination of a voluntary motor action and sensory evidence whether the corresponding body movements have somehow influenced the course of external events. The SoA is not only a key part of our experiences in daily life but also in our interaction with technology as it gives us the feeling of "I did that" as opposed to "the system did that," thus supporting a feeling of being in control. This feeling becomes critical with human-computer integration, wherein emerging technology directly influences people's body, their actions, and the resulting outcomes. In this review, we analyse and classify current integration technologies based on what we currently know about agency in the literature, and propose a distinction between body augmentation, action augmentation, and outcome augmentation. For each category, we describe agency considerations and markers of differentiation that illustrate a relationship between assistance level (low, high), agency delegation (human, technology), and integration type (fusion, symbiosis). We conclude with a reflection on the opportunities and challenges of integrating humans with computers, and finalise with an expanded definition of human-computer integration including agency aspects which we consider to be particularly relevant. The aim this review is to provide researchers and practitioners with guidelines to situate their work within the integration research agenda and consider the implications of any technologies on SoA, and thus overall user experience when designing future technology.

Corrigendum: May I Smell Your Attention: Exploration of Smell and Sound for Visuospatial Attention in Virtual Reality.

[This corrects the article DOI: 10.3389/fpsyg.2021.671470.].

May I Smell Your Attention: Exploration of Smell and Sound for Visuospatial Attention in Virtual Reality.

When interacting with technology, attention is mainly driven by audiovisual and increasingly haptic stimulation. Olfactory stimuli are widely neglected, although the sense of smell influences many of our daily life choices, affects our behavior, and can catch and direct our attention. In this study, we investigated the effect of smell and sound on visuospatial attention in a virtual environment. We implemented the Bells Test, an established neuropsychological test to assess attentional and visuospatial disorders, in virtual reality (VR). We conducted an experiment with 24 participants comparing the performance of users under three experimental conditions (smell, sound, and smell and sound). The results show that multisensory stimuli play a key role in driving the attention of the participants and highlight asymmetries in directing spatial attention. We discuss the relevance of the results within and beyond human-computer interaction (HCI), particularly with regard to the opportunity of using VR for rehabilitation and assessment procedures for patients with spatial attention deficits.

Multisensory Integration as per Technological Advances: A Review.

Multisensory integration research has allowed us to better understand how humans integrate sensory information to produce a unitary experience of the external world. However, this field is often challenged by the limited ability to deliver and control sensory stimuli, especially when going beyond audio-visual events and outside laboratory settings. In this review, we examine the scope and challenges of new technology in the study of multisensory integration in a world that is increasingly characterized as a fusion of physical and digital/virtual events. We discuss multisensory integration research through the lens of novel multisensory technologies and, thus, bring research in human-computer interaction, experimental psychology, and neuroscience closer together. Today, for instance, displays have become volumetric so that visual content is no longer limited to 2D screens, new haptic devices enable tactile stimulation without physical contact, olfactory interfaces provide users with smells precisely synchronized with events in virtual environments, and novel gustatory interfaces enable taste perception through levitating stimuli. These technological advances offer new ways to control and deliver sensory stimulation for multisensory integration research beyond traditional laboratory settings and open up new experimentations in naturally occurring events in everyday life experiences. Our review then summarizes these multisensory technologies and discusses initial insights to introduce a bridge between the disciplines in order to advance the study of multisensory integration.

Spatial Soundscapes and Virtual Worlds: Challenges and Opportunities.

There is increasing effort to characterize the soundscapes around us so that we can design more compelling and immersive experiences. This review paper focuses on the challenges and opportunities around sound perception, with a particular focus on spatial sound perception in a virtual reality (VR) cityscape. We review how research on temporal aspects has recently been extended to evaluating spatial factors when designing soundscapes. In particular, we discuss key findings on the human capability of localizing and distinguishing spatial sound cues for different technical setups. We highlight studies carried out in both real-world and virtual reality settings to evaluate spatial sound perception. We conclude this review by highlighting the opportunities offered by VR technology and the remaining open questions for virtual soundscape designers, especially with the advances in spatial sound stimulation.

Ingested but not perceived: Response to satiety cues disrupted by perceptual load.

Selective attention research has shown that when perceptual demand is high, unattended sensory information is filtered out at early stages of processing. We investigated for the first time whether the sensory and nutrient cues associated with becoming full (satiety) would be filtered out in a similar manner. One-hundred and twenty participants consumed either a low-satiety (75 kcal) or high-satiety (272 kcal plus thicker texture) beverage, delivered via an intra-oral infusion device while participants simultaneously completed a task which was either low or high in perceptual demand. Among participants who performed the low perceptual load task, ingestion of the high-satiety beverage increased rated satiety and reduced consumption at a subsequent snack test. However, both effects were eliminated by the high perceptual load task. Therefore, the processing of satiety cues was dependent on the availability of attention, identifying a novel perceptual load mechanism of inattentive eating and supporting more recent cognitive models of appetite control.

Mid-Air Haptic Rendering of 2D Geometric Shapes With a Dynamic Tactile Pointer.

An important challenge that affects ultrasonic mid-air haptics, in contrast to physical touch, is that we lose certain exploratory procedures such as contour following. This makes the task of perceiving geometric properties and shape identification more difficult. Meanwhile, the growing interest in mid-air haptics and their application to various new areas requires an improved understanding of how we perceive specific haptic stimuli, such as icons and control dials in mid-air. We address this challenge by investigating static and dynamic methods of displaying 2D geometric shapes in mid-air. We display a circle, a square, and a triangle, in either a static or dynamic condition, using ultrasonic mid-air haptics. In the static condition, the shapes are presented as a full outline in mid-air, while in the dynamic condition, a tactile pointer is moved around the perimeter of the shapes. We measure participants' accuracy and confidence of identifying shapes in two controlled experiments ( n1 = 34, n2 = 25). Results reveal that in the dynamic condition people recognise shapes significantly more accurately, and with higher confidence. We also find that representing polygons as a set of individually drawn haptic strokes, with a short pause at the corners, drastically enhances shape recognition accuracy. Our research supports the design of mid-air haptic user interfaces in application scenarios such as in-car interactions or assistive technology in education.

Creating an Illusion of Movement between the Hands Using Mid-Air Touch.

Apparent tactile motion has been shown to occur on many contiguous parts of the body, such as the fingers, forearms, and back. More recently, the illusion has also been elicited on non-contiguous parts of the body, such as between one hand and the other, either when the hands are interconnected or not interconnected by an object (e.g., when holding a tablet or not). Here, we explore the reproducibility of the intermanual tactile illusion of movement between two free hands by employing mid-air tactile stimulation. We investigate the optimal parameters to generate a continuous and smooth motion using two arrays of ultrasound speakers and two stimulation techniques (i.e., static versus dynamic focal point). In the first experiment, we investigate the occurrence of the illusion when using a static focal point, and we define a perceptive model. In the second experiment, we examine the illusion using a dynamic focal point, defining a second perceptive model. Finally, we discuss the differences between the two techniques.

Emotional ratings and skin conductance response to visual, auditory and haptic stimuli.

The human emotional reactions to stimuli delivered by different sensory modalities is a topic of interest for many disciplines, from Human-Computer-Interaction to cognitive sciences. Different databases of stimuli eliciting emotional reaction are available, tested on a high number of participants. Interestingly, stimuli within one database are always of the same type. In other words, to date, no data was obtained and compared from distinct types of emotion-eliciting stimuli from the same participant. This makes it difficult to use different databases within the same experiment, limiting the complexity of experiments investigating emotional reactions. Moreover, whereas the stimuli and the participants' rating to the stimuli are available, physiological reactions of participants to the emotional stimuli are often recorded but not shared. Here, we test stimuli delivered either through a visual, auditory, or haptic modality in a within participant experimental design. We provide the results of our study in the form of a MATLAB structure including basic demographics on the participants, the participant's self-assessment of his/her emotional state, and his/her physiological reactions (i.e., skin conductance).

Sour Promotes Risk-Taking: An Investigation into the Effect of Taste on Risk-Taking Behaviour in Humans.

Taking risks is part of everyday life. Some people actively pursue risky activities (e.g., jumping out of a plane), while others avoid any risk (e.g., people with anxiety disorders). Paradoxically, risk-taking is a primitive behaviour that may lead to a happier life by offering a sense of excitement through self-actualization. Here, we demonstrate for the first time that sour - amongst the five basic tastes (sweet, bitter, sour, salty, and umami) - promotes risk-taking. Based on a series of three experiments, we show that sour has the potential to modulate risk-taking behaviour across two countries (UK and Vietnam), across individual differences in risk-taking personality and styles of thinking (analytic versus intuitive). Modulating risk-taking can improve everyday life for a wide range of people.

Multisensory Technology for Flavor Augmentation: A Mini Review.

There is growing interest in the development of new technologies that capitalize on our emerging understanding of the multisensory influences on flavor perception in order to enhance human-food interaction design. This review focuses on the role of (extrinsic) visual, auditory, and haptic/tactile elements in modulating flavor perception and more generally, our food and drink experiences. We review some of the most exciting examples of recent multisensory technologies for augmenting such experiences. Here, we discuss applications for these technologies, for example, in the field of food experience design, in the support of healthy eating, and in the rapidly growing world of sensory marketing. However, as the review makes clear, while there are many opportunities for novel human-food interaction design, there are also a number of challenges that will need to be tackled before new technologies can be meaningfully integrated into our everyday food and drink experiences.

Viewing experience of 3DTV: An exploration of the feeling of sickness and presence in a shopping mall.

The adoption and deployment of 3DTV can be seen as a major step in the history of television, comparable to the transition from analogue to digital and standard to high definition TV. Although 3D is expected to emerge from the cinema to peoples' home, there is still a lack of knowledge on how people (future end users) perceive 3DTV and how this influences their viewing experience as well as their acceptance of 3DTV. Within this paper, findings from a three-day field evaluation study on people's 3DTV experiences, focusing on the feeling of sickness and presence, are presented. Contrary to the traditional controlled laboratory setting, the study was conducted in the public setting of a shopping center and involved 700 participants. The study revealed initial insights on users' feeling of presence and sickness when watching 3DTV content. Results from this explorative study show that most of the participants reported symptoms of sickness after watching 3DTV with an effect of gender and age on the reported feeling of sickness. Our results further suggest that the users' previous experience with 3D content has an influence on how realistic people rate the viewing experience and how involved they feel. The particularities of the study environment, a shopping mall, are reflected in our findings and future research directions and action points for investigating people's viewing experiences of 3DTV are summarized.