Decoding the dynamic perception of risk and speed using naturalistic stimuli: A multivariate, whole-brain analysis.

Time-resolved decoding of speed and risk perception in car driving is important for understanding the perceptual processes related to driving safety. In this study, we used an fMRI-compatible trackball with naturalistic stimuli to record dynamic ratings of perceived risk and speed and investigated the degree to which different brain regions were able to decode these. We presented participants with first-person perspective videos of cars racing on the same course. These videos varied in terms of subjectively perceived speed and risk profiles, as determined during a behavioral pilot. During the fMRI experiment, participants used the trackball to dynamically rate subjective risk in a first and speed in a second session and assessed overall risk and speed after watching each video. A standard multivariate correlation analysis based on these ratings revealed sparse decodability in visual areas only for the risk ratings. In contrast, the dynamic rating-based correlation analysis uncovered frontal, visual, and temporal region activation for subjective risk and dorsal visual stream and temporal region activation for subjectively perceived speed. Interestingly, further analyses showed that the brain regions for decoding risk changed over time, whereas those for decoding speed remained constant. Overall, our results demonstrate the advantages of time-resolved decoding to help our understanding of the dynamic networks associated with decoding risk and speed perception in realistic driving scenarios.

Manipulating and decoding subjective gaming experience during active gameplay: a multivariate, whole-brain analysis.

A large number of perceptual and cognitive processes are instantiated during active gameplay, culminating in what is termed the overall "gaming experience", which encapsulates multiple, subjective dimensions of how one feels about the game. Although some research has been conducted into the neural mechanisms underlying the gaming experience, previous studies so far have relied on commercial games that provide little control over key aspects of gameplay and also have focused only on a few individual dimensions of the gaming experience. Here, we used a custom-made, immersive driving car game in four different gameplay versions (baseline, obstacle increase, goal decrease, speed increase) to assess and modulate the subjective gameplay experience while participants underwent a fMRI scan. A multivariate correlation analysis of whole-brain neural activity with behaviorally-identified subjective gaming experience uncovered brain networks associated with different experiences, including higher-level visual processing networks, the default network, and emotional areas. These regions were in addition able to decode the four different game conditions above chance. Our results for the first time describe the full range of cortical networks that become engaged to create the subjective experience during active gameplay.

A Survey of Viewpoint Selection Methods for Polygonal Models.

Viewpoint selection has been an emerging area in computer graphics for some years, and it is now getting maturity with applications in fields such as scene navigation, scientific visualization, object recognition, mesh simplification, and camera placement. In this survey, we review and compare twenty-two measures to select good views of a polygonal 3D model, classify them using an extension of the categories defined by Secord et al., and evaluate them against the Dutagaci et al. benchmark. Eleven of these measures have not been reviewed in previous surveys. Three out of the five short-listed best viewpoint measures are directly related to information. We also present in which fields the different viewpoint measures have been applied. Finally, we provide a publicly available framework where all the viewpoint selection measures are implemented and can be compared against each other.

"Can touch this": Cross-modal shape categorization performance is associated with microstructural characteristics of white matter association pathways.

Previous studies on visuo-haptic shape processing provide evidence that visually learned shape information can transfer to the haptic domain. In particular, recent neuroimaging studies have shown that visually learned novel objects that were haptically tested recruited parts of the ventral pathway from early visual cortex to the temporal lobe. Interestingly, in such tasks considerable individual variation in cross-modal transfer performance was observed. Here, we investigate whether this individual variation may be reflected in microstructural characteristics of white-matter (WM) pathways. We first trained participants on a fine-grained categorization task of novel shapes in the visual domain, followed by a haptic categorization test. We then correlated visual training-performance and haptic test-performance, as well as performance on a symbol-coding task requiring visuo-motor dexterity with microstructural properties of WM bundles potentially involved in visuo-haptic processing (the inferior longitudinal fasciculus [ILF], the fronto-temporal part of the superior longitudinal fasciculus [SLFft ] and the vertical occipital fasciculus [VOF]). Behavioral results showed that haptic categorization performance was good on average but exhibited large inter-individual variability. Haptic performance also was correlated with performance in the symbol-coding task. WM analyses showed that fast visual learners exhibited higher fractional anisotropy (FA) in left SLFft and left VOF. Importantly, haptic test-performance (and symbol-coding performance) correlated with FA in ILF and with axial diffusivity in SLFft . These findings provide clear evidence that individual variation in visuo-haptic performance can be linked to microstructural characteristics of WM pathways. Hum Brain Mapp 38:842-854, 2017. © 2016 Wiley Periodicals, Inc.

Visual and Haptic Shape Processing in the Human Brain: Unisensory Processing, Multisensory Convergence, and Top-Down Influences.

Humans are highly adept at multisensory processing of object shape in both vision and touch. Previous studies have mostly focused on where visually perceived object-shape information can be decoded, with haptic shape processing receiving less attention. Here, we investigate visuo-haptic shape processing in the human brain using multivoxel correlation analyses. Importantly, we use tangible, parametrically defined novel objects as stimuli. Two groups of participants first performed either a visual or haptic similarity-judgment task. The resulting perceptual object-shape spaces were highly similar and matched the physical parameter space. In a subsequent fMRI experiment, objects were first compared within the learned modality and then in the other modality in a one-back task. When correlating neural similarity spaces with perceptual spaces, visually perceived shape was decoded well in the occipital lobe along with the ventral pathway, whereas haptically perceived shape information was mainly found in the parietal lobe, including frontal cortex. Interestingly, ventrolateral occipito-temporal cortex decoded shape in both modalities, highlighting this as an area capable of detailed visuo-haptic shape processing. Finally, we found haptic shape representations in early visual cortex (in the absence of visual input), when participants switched from visual to haptic exploration, suggesting top-down involvement of visual imagery on haptic shape processing.

Abstract representations of associated emotions in the human brain.

Emotions can be aroused by various kinds of stimulus modalities. Recent neuroimaging studies indicate that several brain regions represent emotions at an abstract level, i.e., independently from the sensory cues from which they are perceived (e.g., face, body, or voice stimuli). If emotions are indeed represented at such an abstract level, then these abstract representations should also be activated by the memory of an emotional event. We tested this hypothesis by asking human participants to learn associations between emotional stimuli (videos of faces or bodies) and non-emotional stimuli (fractals). After successful learning, fMRI signals were recorded during the presentations of emotional stimuli and emotion-associated fractals. We tested whether emotions could be decoded from fMRI signals evoked by the fractal stimuli using a classifier trained on the responses to the emotional stimuli (and vice versa). This was implemented as a whole-brain searchlight, multivoxel activation pattern analysis, which revealed successful emotion decoding in four brain regions: posterior cingulate cortex (PCC), precuneus, MPFC, and angular gyrus. The same analysis run only on responses to emotional stimuli revealed clusters in PCC, precuneus, and MPFC. Multidimensional scaling analysis of the activation patterns revealed clear clustering of responses by emotion across stimulus types. Our results suggest that PCC, precuneus, and MPFC contain representations of emotions that can be evoked by stimuli that carry emotional information themselves or by stimuli that evoke memories of emotional stimuli, while angular gyrus is more likely to take part in emotional memory retrieval.

Adaptation of cortical activity to sustained pressure stimulation on the fingertip.

BACKGROUND: Tactile adaptation is a phenomenon of the sensory system that results in temporal desensitization after an exposure to sustained or repetitive tactile stimuli. Previous studies reported psychophysical and physiological adaptation where perceived intensity and mechanoreceptive afferent signals exponentially decreased during tactile adaptation. Along with these studies, we hypothesized that somatosensory cortical activity in the human brain also exponentially decreased during tactile adaptation. The present neuroimaging study specifically investigated temporal changes in the human cortical responses to sustained pressure stimuli mediated by slow-adapting type I afferents. METHODS: We applied pressure stimulation for up to 15 s to the right index fingertip in 21 healthy participants and acquired functional magnetic resonance imaging (fMRI) data using a 3T MRI system. We analyzed cortical responses in terms of the degrees of cortical activation and inter-regional connectivity during sustained pressure stimulation. RESULTS: Our results revealed that the degrees of activation in the contralateral primary and secondary somatosensory cortices exponentially decreased over time and that intra- and inter-hemispheric inter-regional functional connectivity over the regions associated with tactile perception also linearly decreased or increased over time, during pressure stimulation. CONCLUSION: These results indicate that cortical activity dynamically adapts to sustained pressure stimulation mediated by SA-I afferents, involving changes in the degrees of activation on the cortical regions for tactile perception as well as in inter-regional functional connectivity among them. We speculate that these adaptive cortical activity may represent an efficient cortical processing of tactile information.

Intra- and inter-hemispheric effective connectivity in the human somatosensory cortex during pressure stimulation.

BACKGROUND: Slow-adapting type I (SA-I) afferents deliver sensory signals to the somatosensory cortex during low-frequency (or static) mechanical stimulation. It has been reported that the somatosensory projection from SA-I afferents is effective and reliable for object grasping and manipulation. Despite a large number of neuroimaging studies on cortical activation responding to tactile stimuli mediated by SA-I afferents, how sensory information of such tactile stimuli flows over the somatosensory cortex remains poorly understood. In this study, we investigated tactile information processing of pressure stimuli between the primary (SI) and secondary (SII) somatosensory cortices by measuring effective connectivity using dynamic causal modeling (DCM). We applied pressure stimuli for 3 s to the right index fingertip of healthy participants and acquired functional magnetic resonance imaging (fMRI) data using a 3T MRI system. RESULTS: DCM analysis revealed intra-hemispheric effective connectivity between the contralateral SI (cSI) and SII (cSII) characterized by both parallel (signal inputs to both cSI and cSII) and serial (signal transmission from cSI to cSII) pathways during pressure stimulation. DCM analysis also revealed inter-hemispheric effective connectivity among cSI, cSII, and the ipsilateral SII (iSII) characterized by serial (from cSI to cSII) and SII-level (from cSII to iSII) pathways during pressure stimulation. CONCLUSIONS: Our results support a hierarchical somatosensory network that underlies processing of low-frequency tactile information. The network consists of parallel inputs to both cSI and cSII (intra-hemispheric), followed by serial pathways from cSI to cSII (intra-hemispheric) and from cSII to iSII (inter-hemispheric). Importantly, our results suggest that both serial and parallel processing take place in tactile information processing of static mechanical stimuli as well as highlighting the contribution of callosal transfer to bilateral neuronal interactions in SII.

Psychological distress and attentional bias toward acne lesions in patients with acne.

Acne vulgaris is a common inflammatory disease that manifests on the face and affects appearance. In general, facial acne has a wide-ranging negative impact on the psychosocial functioning of acne sufferers and leaves physical and emotional scars. In the present study, we investigated whether patients with acne vulgaris demonstrate enhanced psychological bias when assessing the attractiveness of faces with acne symptoms and whether they devote greater selective attention to acne lesions than to acne-free (control) individuals. Participants viewed images of faces under two different skin (acne vs. acne-free) and emotional facial expression (happy and neutral) conditions. They rated the attractiveness of the faces, and the time spent fixating on the acne lesions was recorded with an eye tracker. We found that the gap in perceived attractiveness between acne and acne-free faces was greater for acne sufferers. Furthermore, patients with acne fixated longer on facial regions exhibiting acne lesions than did control participants irrespective of the facial expression depicted. In summary, patients with acne have a stronger attentional bias for acne lesions and focus more on the skin lesions than do those without acne. Clinicians treating the skin problems of patients with acne should consider these psychological and emotional scars.

Decoding visual fatigue in a visual search task selectively manipulated via myopia-correcting lenses.

Introduction: Visual fatigue resulting from sustained, high-workload visual activities can significantly impact task performance and general wellbeing. So far, however, little is known about the underlying brain networks of visual fatigue. This study aimed to identify such potential networks using a unique paradigm involving myopia-correcting lenses known to directly modulate subjectively-perceived fatigue levels. Methods: A sample of N = 31 myopia participants [right eye-SE: -3.77D (SD: 2.46); left eye-SE: -3.75D (SD: 2.45)] performed a demanding visual search task with varying difficulty levels, both with and without the lenses, while undergoing fMRI scanning. There were a total of 20 trials, after each of which participants rated the perceived difficulty and their subjective visual fatigue level. We used representational similarity analysis to decode brain regions associated with fatigue and difficulty, analyzing their individual and joint decoding pattern. Results and discussion: Behavioral results showed correlations between fatigue and difficulty ratings and above all a significant reduction in fatigue levels when wearing the lenses. Imaging results implicated the cuneus, lingual gyrus, middle occipital gyrus (MOG), and declive for joint fatigue and difficulty decoding. Parts of the lingual gyrus were able to selectively decode perceived difficulty. Importantly, a broader network of visual and higher-level association areas showed exclusive decodability of fatigue (culmen, middle temporal gyrus (MTG), parahippocampal gyrus, precentral gyrus, and precuneus). Our findings enhance our understanding of processing within the context of visual search, attention, and mental workload and for the first time demonstrate that it is possible to decode subjectively-perceived visual fatigue during a challenging task from imaging data. Furthermore, the study underscores the potential of myopia-correcting lenses in investigating and modulating fatigue.

An EEG Dataset of Neural Signatures in a Competitive Two-Player Game Encouraging Deceptive Behavior.

Studying deception is vital for understanding decision-making and social dynamics. Recent EEG research has deepened insights into the brain mechanisms behind deception. Standard methods in this field often rely on memory, are vulnerable to countermeasures, yield false positives, and lack real-world relevance. Here, we present a comprehensive dataset from an EEG-monitored competitive, two-player card game designed to elicit authentic deception behavior. Our extensive dataset contains EEG data from 12 pairs (N = 24 participants with role switching), controlled for age, gender, and risk-taking, with detailed labels and annotations. The dataset combines standard event-related potential and microstate analyses with state-of-the-art decoding approaches of four scenarios: spontaneous/instructed truth-telling and lying. This demonstrates game-based methods' efficacy in studying deception and sets a benchmark for future research. Overall, our dataset represents a unique resource with applications in cognitive neuroscience and related fields for studying deception, competitive behavior, decision-making, inter-brain synchrony, and benchmarking of decoding frameworks in a difficult, high-level cognitive task.

Exploiting object constancy: effects of active exploration and shape morphing on similarity judgments of novel objects.

Humans are experts at shape processing. This expertise has been learned and fine tuned by actively manipulating and perceiving thousands of objects during development. Therefore, shape processing possesses an active component and a perceptual component. Here, we investigate both components in six experiments in which participants view and/or interact with novel, parametrically defined 3D objects using a touch-screen interface. For probing shape processing, we use a similarity rating task. In Experiments 1-3, we show that active manipulation leads to a better perceptual reconstruction of the physical parameter space than judging rotating objects, or passively viewing someone else's exploration pattern. In Experiment 4, we exploit object constancy-the fact that the visual system assumes that objects do not change their identity during manipulation. We show that slow morphing of an object during active manipulation systematically biases similarity ratings-despite the participants being unaware of the morphing. Experiments 5 and 6 investigate the time course of integrating shape information by restricting the morphing to the first and second half of the trial only. Interestingly, the results indicate that participants do not seem to integrate shape information beyond 5 s of exploration time. Finally, Experiment 7 uses a secondary task that suggests that the previous results are not simply due to lack of attention during the later parts of the trial. In summary, our results demonstrate the advantage of active manipulation for shape processing and indicate a continued, perceptual integration of complex shape information within a time window of a few seconds during object interactions.

Learning to recognize face shapes through serial exploration.

Human observers are experts at visual face recognition due to specialized visual mechanisms for face processing that evolve with perceptual expertize. Such expertize has long been attributed to the use of configural processing, enabled by fast, parallel information encoding of the visual information in the face. Here we tested whether participants can learn to efficiently recognize faces that are serially encoded-that is, when only partial visual information about the face is available at any given time. For this, ten participants were trained in gaze-restricted face recognition in which face masks were viewed through a small aperture controlled by the participant. Tests comparing trained with untrained performance revealed (1) a marked improvement in terms of speed and accuracy, (2) a gradual development of configural processing strategies, and (3) participants' ability to rapidly learn and accurately recognize novel exemplars. This performance pattern demonstrates that participants were able to learn new strategies to compensate for the serial nature of information encoding. The results are discussed in terms of expertize acquisition and relevance for other sensory modalities relying on serial encoding.

Relationship between virtual reality balloon analogue risk task and risky decision-making.

The balloon analogue risk task (BART) is widely used to assess risk-taking tendencies on behavioral tests. However, biases or unstable results are sometimes reported, and there are concerns about whether the BART can predict risk behavior in the real world. To address this problem, the present study developed a virtual reality (VR) BART to enhance the reality of the task and narrow the gap between performance on the BART and risk behavior in the real world. We evaluated the usability of our VR BART through assessments of the relationships between BART scores and psychological metrics and additionally implemented an emergency decision-making VR driving task to investigate further whether the VR BART can predict risk-related decision-making in emergency situations. Notably, we found that the BART score significantly correlated with both sensation-seeking and risky driving behavior. Additionally, when we split participants into groups with high and low BART scores and compared their psychological metrics, we found that the high-score BART group included more male participants and exhibited higher sensation-seeking and more risky decision-making in an emergency situation. Overall, our study shows the potential of our new VR BART paradigm to predict risky decision-making in the real world.

Dynamic measurements of speed and risk perception during driving: Evidence of speed misestimation from continuous ratings and video analysis.

Investigating the factors underlying perceived speed and risk is crucial to ensure safe driving. However, existing studies on this topic usually measure speed and risk perception indirectly after a driving session, which makes it difficult to trace dynamic effects and time points of potential misestimates. To address this problem, we developed and validated a novel continuous method for dynamically measuring risk and speed perceptions. To study the factors affecting risk and speed perception, we presented participants with videos captured on the same racing track from the same point of view but with different drivers who varied in their speed and risk profiles. During the experiment, participants used a joystick to continuously rate the subjectively perceived risk of driving in the first block and the perceived speed in the second block. Our analysis of these dynamic ratings indicates that risk and speed estimates were decoupled, with curves resulting in decreased speeds but increased risk ratings. However, a close distance to the car in front increased both speed and risk. Based on actual and estimated speed data, we found that overtaking cars on curves resulted in participants overestimating their own speed, whereas an increase in the distance to the car in front on a straight course led to underestimations of their own speed. Our results showcase the usefulness of dynamic rating profiles for in-depth investigations into situations that could result in drivers misjudging speed or risk and will thus help the development of more intelligent, human-centered driving assistance systems.

Putting Vision and Touch into Conflict: Results From a Multimodal Mixed Reality Setup.

What happens if we put vision and touch into conflict? Which modality "wins"? Although several previous studies have addressed this topic, they have solely focused on integration of vision and touch for low-level object properties (such as curvature, slant, or depth). In the present study, we introduce a multimodal mixed-reality setup based on real-time hand-tracking, which was used to display real-world, haptic exploration of objects in a virtual environment through a head-mounted-display (HMD). With this setup we studied multimodal conflict situations of objects varying along higher-level, parametrically-controlled global shape properties. Participants explored these objects in both unimodal and multimodal settings with the latter including congruent and incongruent conditions and differing instructions for weighting the input modalities. Results demonstrated a surprisingly clear touch dominance throughout all experiments, which in addition was only marginally influenceable through instructions to bias their modality weighting. We also present an initial analysis of the hand-tracking patterns that illustrates the potential for our setup to investigate exploration behavior in more detail.

Visuo-haptic processing of unfamiliar shapes: Comparing children and adults.

The question of how our sensory perception abilities develop has been an active area of research, establishing trajectories of development from infancy that last well into late childhood and even adolescence. In this context, several studies have established changes in sensory processing of vision and touch around the age of 8 to 9 years. In this experiment, we explored the visual and haptic perceptual development of elementary school children of ages 6-11 in similarity-rating tasks of unfamiliar objects and compared their performance to adults. The participants were presented with parametrically-defined objects to be explored haptically and visually in separate groups for both children and adults. Our results showed that the raw similarity ratings of the children had more variability compared to adults. A detailed multidimensional scaling analysis revealed that the reconstructed perceptual space of the adult haptic group was significantly closer to the parameter space compared to the children group, whereas both groups' visual perceptual space was similarly well reconstructed. Beyond this, however, we found no clear evidence for an age effect in either modality within the children group. These results suggest that haptic processing of unfamiliar, abstract shapes may continue to develop beyond the age of 11 years later into adolescence.

Categorizing natural objects: a comparison of the visual and the haptic modalities.

Although the hands are the most important tool for humans to manipulate objects, only little is known about haptic processing of natural objects. Here, we selected a unique set of natural objects, namely seashells, which vary along a variety of object features, while others are shared across all stimuli. To correctly interact with objects, they have to be identified or categorized. For both processes, measuring similarities between objects is crucial. Our goal is to better understand the haptic similarity percept by comparing it to the visual similarity percept. First, direct similarity measures were analyzed using multidimensional scaling techniques to visualize the perceptual spaces of both modalities. We find that the visual and the haptic modality form almost identical perceptual spaces. Next, we performed three different categorization tasks. All tasks exhibit a highly accurate processing of complex shapes of the haptic modality. Moreover, we find that objects grouped into the same category form regions within the perceptual space. Hence, in both modalities, perceived similarity constitutes the basis for categorizing objects. Moreover, both modalities focus on shape to form categories. Taken together, our results lead to the assumption that the same cognitive processes link haptic and visual similarity perception and the resulting categorization behavior.

Serial exploration of faces: comparing vision and touch.

Even though we can recognize faces by touch surprisingly well, haptic face recognition performance is still worse than for visual exploration. One possibility for this performance difference might be due to different encoding strategies in the two modalities, namely, holistic encoding in vision versus serial encoding in haptics. Here, we tested this hypothesis by promoting serial encoding in vision, using a novel, gaze-restricted display that limited the effective field of view in vision to resemble that of haptic exploration. First, we compared haptic with gaze-restricted and unrestricted visual face recognition. Second, we used the face inversion paradigm to assess how encoding differences might affect processing strategies (featural vs. holistic). By promoting serial encoding in vision, we found equal face recognition performance in vision and haptics with a clear switch from holistic to featural processing, suggesting that performance differences in visual and haptic face recognition are due to modality-specific encoding strategies.