Silent Expectations: Dynamic Causal Modeling of Cortical Prediction and Attention to Sounds That Weren't.

UNLABELLED: There is increasing evidence that human perception is realized by a hierarchy of neural processes in which predictions sent backward from higher levels result in prediction errors that are fed forward from lower levels, to update the current model of the environment. Moreover, the precision of prediction errors is thought to be modulated by attention. Much of this evidence comes from paradigms in which a stimulus differs from that predicted by the recent history of other stimuli (generating a so-called "mismatch response"). There is less evidence from situations where a prediction is not fulfilled by any sensory input (an "omission" response). This situation arguably provides a more direct measure of "top-down" predictions in the absence of confounding "bottom-up" input. We applied Dynamic Causal Modeling of evoked electromagnetic responses recorded by EEG and MEG to an auditory paradigm in which we factorially crossed the presence versus absence of "bottom-up" stimuli with the presence versus absence of "top-down" attention. Model comparison revealed that both mismatch and omission responses were mediated by increased forward and backward connections, differing primarily in the driving input. In both responses, modeling results suggested that the presence of attention selectively modulated backward "prediction" connections. Our results provide new model-driven evidence of the pure top-down prediction signal posited in theories of hierarchical perception, and highlight the role of attentional precision in strengthening this prediction. SIGNIFICANCE STATEMENT: Human auditory perception is thought to be realized by a network of neurons that maintain a model of and predict future stimuli. Much of the evidence for this comes from experiments where a stimulus unexpectedly differs from previous ones, which generates a well-known "mismatch response." But what happens when a stimulus is unexpectedly omitted altogether? By measuring the brain's electromagnetic activity, we show that it also generates an "omission response" that is contingent on the presence of attention. We model these responses computationally, revealing that mismatch and omission responses only differ in the location of inputs into the same underlying neuronal network. In both cases, we show that attention selectively strengthens the brain's prediction of the future.

Expectation and attention in hierarchical auditory prediction.

Hierarchical predictive coding suggests that attention in humans emerges from increased precision in probabilistic inference, whereas expectation biases attention in favor of contextually anticipated stimuli. We test these notions within auditory perception by independently manipulating top-down expectation and attentional precision alongside bottom-up stimulus predictability. Our findings support an integrative interpretation of commonly observed electrophysiological signatures of neurodynamics, namely mismatch negativity (MMN), P300, and contingent negative variation (CNV), as manifestations along successive levels of predictive complexity. Early first-level processing indexed by the MMN was sensitive to stimulus predictability: here, attentional precision enhanced early responses, but explicit top-down expectation diminished it. This pattern was in contrast to later, second-level processing indexed by the P300: although sensitive to the degree of predictability, responses at this level were contingent on attentional engagement and in fact sharpened by top-down expectation. At the highest level, the drift of the CNV was a fine-grained marker of top-down expectation itself. Source reconstruction of high-density EEG, supported by intracranial recordings, implicated temporal and frontal regions differentially active at early and late levels. The cortical generators of the CNV suggested that it might be involved in facilitating the consolidation of context-salient stimuli into conscious perception. These results provide convergent empirical support to promising recent accounts of attention and expectation in predictive coding.

Investigating the Haptic Perception of Directional Information Within a Handle.

This paper studies the perception of 2-dimensional directional cues presented on a hand-held tangible interface that resembles a cylindrical handle. The tangible interface is designed to be comfortably held with one hand and houses five custom electromagnetic actuators composed of coils as stators and magnets as movers. We carried out a human subjects experiment enrolling 24 participants, analysing the recognition rate of directional cues using the actuators either to vibrate or tap in sequence across the user's palm. Results show an impact of the positioning/holding of the handle, the mode of stimulation, and the directional indication sent via the handle. There was also a correlation between the score and the confidence of the participants, showing that participants are more confident when recognising vibration patterns. Overall, results supported the potential of the haptic handle to provide accurate guidance, with recognition rates higher than 70 % in all conditions and higher than 75 % in the precane and power wheelchair configurations.

Normal and tangential forces combine to convey contact pressure during dynamic tactile stimulation.

Humans need to accurately process the contact forces that arise as they perform everyday haptic interactions such as sliding the fingers along a surface to feel for bumps, sticky regions, or other irregularities. Several different mechanisms are possible for how the forces on the skin could be represented and integrated in such interactions. In this study, we used a force-controlled robotic platform and simultaneous ultrasonic modulation of the finger-surface friction to independently manipulate the normal and tangential forces during passive haptic stimulation by a flat surface. To assess whether the contact pressure on their finger had briefly increased or decreased during individual trials in this broad stimulus set, participants did not rely solely on either the normal force or the tangential force. Instead, they integrated tactile cues induced by both components. Support-vector-machine analysis classified physical trial data with up to 75% accuracy and suggested a linear perceptual mechanism. In addition, the change in the amplitude of the force vector predicted participants' responses better than the change of the coefficient of dynamic friction, suggesting that intensive tactile cues are meaningful in this task. These results provide novel insights about how normal and tangential forces shape the perception of tactile contact.

Phononic Crystals Applied to Localised Surface Haptics.

Metamaterials are solid lattices with periodicities commensurate with desired wavelengths. Their geometric features can endow the bulk material with unusual properties, such as inter alia, negative indices of refraction or unique absorbing qualities. Mesoscale metamaterials and phononic crystals can be designed to cause the occurence of band gaps in the ultrasonic domain. These localised phenomena induce fixed boundary conditions that correspond to acoustic mirrors which, in turn, can be used to establish waveguides in thin plates. Ultrasonic lubrication has been successfully applied to create haptic interfaces that operate by modulating the apparent friction of a surface. In this article, we demonstrate that phononic crystals can be designed to localise the modulation of friction in specific portions of the surface of a thin plate, opening novel possibilities for the design of surface haptic interfaces.

Physical Variables Underlying Tactile Stickiness During Fingerpad Detachment.

One may notice a relatively wide range of tactile sensations even when touching the same hard, flat surface in similar ways. Little is known about the reasons for this variability, so we decided to investigate how the perceptual intensity of light stickiness relates to the physical interaction between the skin and the surface. We conducted a psychophysical experiment in which nine participants actively pressed their finger on a flat glass plate with a normal force close to 1.5 N and detached it after a few seconds. A custom-designed apparatus recorded the contact force vector and the finger contact area during each interaction as well as pre- and post-trial finger moisture. After detaching their finger, participants judged the stickiness of the glass using a nine-point scale. We explored how sixteen physical variables derived from the recorded data correlate with each other and with the stickiness judgments of each participant. These analyses indicate that stickiness perception mainly depends on the pre-detachment pressing duration, the time taken for the finger to detach, and the impulse in the normal direction after the normal force changes sign; finger-surface adhesion seems to build with pressing time, causing a larger normal impulse during detachment and thus a more intense stickiness sensation. We additionally found a strong between-subjects correlation between maximum real contact area and peak pull-off force, as well as between finger moisture and impulse.

Compensating for Fingertip Size to Render Tactile Cues More Accurately.

Fingertip haptic feedback offers advantages in many applications, including robotic teleoperation, gaming, and training. However, fingertip size and shape vary significantly across humans, making it difficult to design fingertip interfaces and rendering techniques suitable for everyone. This article starts with an existing data-driven haptic rendering algorithm that ignores fingertip size, and it then develops two software-based approaches to personalize this algorithm for fingertips of different sizes using either additional data or geometry. We evaluate our algorithms in the rendering of pre-recorded tactile sensations onto rubber casts of six different fingertips as well as onto the real fingertips of 13 human participants. Results on the casts show that both approaches significantly improve performance, reducing force error magnitudes by an average of 78% with respect to the standard non-personalized rendering technique. Congruent results were obtained for real fingertips, with subjects rating each of the two personalized rendering techniques significantly better than the standard non-personalized method.

The Perception of Ultrasonic Square Reductions of Friction With Variable Sharpness and Duration.

The human perception of square ultrasonic modulation of the finger-surface friction was investigated during active tactile exploration by using short frictional cues of varying duration and sharpness. In a first experiment, we asked participants to discriminate the transition time and duration of short square ultrasonic reductions of friction. They proved very sensitive to discriminate millisecond differences in these two parameters with the average psychophysical thresholds being 2.3-2.4 ms for both parameters. A second experiment focused on the perception of square friction reductions with variable transition times and durations. We found that for durations of the stimulation larger than 90 ms, participants often perceived three or four edges when only two stimulations were presented while they consistently felt two edges for signals shorter than 50 ms. A subsequent analysis of the contact forces induced by these ultrasonic stimulations during slow and fast active exploration showed that two identical consecutive ultrasonic pulses can induce significantly different frictional dynamics especially during fast motion of the finger. These results confirm the human sensitivity to transient frictional cues and suggest that the human perception of square reductions of friction can depend on their sharpness and duration as well as on the speed of exploration.

The tactile perception of transient changes in friction.

When we touch an object or explore a texture, frictional strains are induced by the tactile interactions with the surface of the object. Little is known about how these interactions are perceived, although it becomes crucial for the nascent industry of interactive displays with haptic feedback (e.g. smartphones and tablets) where tactile feedback based on friction modulation is particularly relevant. To investigate the human perception of frictional strains, we mounted a high-fidelity friction modulating ultrasonic device on a robotic platform performing controlled rubbing of the fingertip and asked participants to detect induced decreases of friction during a forced-choice task. The ability to perceive the changes in friction was found to follow Weber's Law of just noticeable differences, as it consistently depended on the ratio between the reduction in tangential force and the pre-stimulation tangential force. The Weber fraction was 0.11 in all conditions demonstrating a very high sensitivity to transient changes in friction. Humid fingers experienced less friction reduction than drier ones for the same intensity of ultrasonic vibration but the Weber fraction for detecting changes in friction was not influenced by the humidity of the skin.

Touch uses frictional cues to discriminate flat materials.

In a forced-choice task, we asked human participants to discriminate by touch alone glass plates from transparent polymethyl methacrylate (PMMA) plastic plates. While the surfaces were flat and did not exhibit geometric features beyond a few tens of nanometres, the materials differed by their molecular structures. They produced similar coefficients of friction and thermal effects were controlled. Most participants performed well above chance and participants with dry fingers discriminated the materials especially well. Current models of tactile surface perception appeal to surface topography and cannot explain our results. A correlation analysis between detailed measurements of the interfacial forces and discrimination performance suggested that the perceptual task depended on the transitory contact phase leading to full slip. This result demonstrates that differences in interfacial mechanics between the finger and a material can be sensed by touch and that the evanescent mechanics that take place before the onset of steady slip have perceptual value.