ß-Bursts over Frontal Cortex Track the Surprise of Unexpected Events in Auditory, Visual, and Tactile Modalities.

One of the fundamental ways in which the brain regulates and monitors behavior is by making predictions about the sensory environment and adjusting behavior when those expectations are violated. As such, surprise is one of the fundamental computations performed by the human brain. In recent years, it has been well established that one key aspect by which behavior is adjusted during surprise is inhibitory control of the motor system. Moreover, because surprise automatically triggers inhibitory control without much proactive influence, it can provide unique insights into largely reactive control processes. Recent years have seen tremendous interest in burst-like ß frequency events in the human (and nonhuman) local field potential-especially over (p)FC-as a potential signature of inhibitory control. To date, ß-bursts have only been studied in paradigms involving a substantial amount of proactive control (such as the stop-signal task). Here, we used two cross-modal oddball tasks to investigate whether surprise processing is accompanied by increases in scalp-recorded ß-bursts. Indeed, we found that unexpected events in all tested sensory domains (haptic, auditory, visual) were followed by low-latency increases in ß-bursting over frontal cortex. Across experiments, ß-burst rates were positively correlated with estimates of surprise derived from Shannon's information theory, a type of surprise that represents the degree to which a given stimulus violates prior expectations. As such, the current work clearly implicates frontal ß-bursts as a signature of surprise processing. We discuss these findings in the context of common frameworks of inhibitory and cognitive control after unexpected events.

Common and Unique Inhibitory Control Signatures of Action-Stopping and Attentional Capture Suggest That Actions Are Stopped in Two Stages.

The ability to stop an already initiated action is paramount to adaptive behavior. Much scientific debate in the field of human action-stopping currently focuses on two interrelated questions. (1) Which cognitive and neural processes uniquely underpin the implementation of inhibitory control when actions are stopped after explicit stop signals, and which processes are instead commonly evoked by all salient signals, even those that do not require stopping? (2) Why do purported (neuro)physiological signatures of inhibition occur at two different latencies after stop signals? Here, we address both questions via two preregistered experiments that combined measurements of corticospinal excitability, EMG, and whole-scalp EEG. Adult human subjects performed a stop signal task that also contained "ignore" signals: equally salient signals that did not require stopping but rather completion of the Go response. We found that both stop- and ignore signals produced equal amounts of early-latency inhibition of corticospinal excitability and EMG, which took place ∼150 ms following either signal. Multivariate pattern analysis of the whole-scalp EEG data further corroborated that this early processing stage was shared between stop- and ignore signals, as neural activity following the two signals could not be decoded from each other until a later time period. In this later period, unique activity related to stop signals emerged at frontocentral scalp sites, reflecting an increased stop signal P3. These findings suggest a two-step model of action-stopping, according to which an initial, universal inhibitory response to the saliency of the stop signal is followed by a slower process that is unique to outright stopping.SIGNIFICANCE STATEMENT Humans often have to stop their ongoing actions when indicated by environmental stimuli (stop signals). Successful action-stopping requires both the ability to detect these salient stop signals and to subsequently inhibit ongoing motor programs. Because of this tight entanglement of attentional control and motor inhibition, identifying unique neurophysiological signatures of action-stopping is difficult. Indeed, we report that recently proposed early-latency signatures of motor inhibition during action-stopping are also found after salient signals that do not require stopping. However, using multivariate pattern analysis of scalp-recorded neural data, we also identified subsequent neural activity that uniquely distinguished action-stopping from saliency detection. These results suggest that actions are stopped in two stages: the first common to all salient events and the second unique to action-stopping.

Judgments of Learning in Context: Backgrounds Can Both Reduce and Produce Metamemory Illusions.

Varying item-specific features such as size (Rhodes & Castel, 2008) or blur (Yue, Castel, & Bjork, 2013) often produces metamemory illusions in which one type of item receives higher judgments of learning (JOLs) without being recalled better. In this study, we explored how similar manipulations to context would influence JOLs. When to-be-recalled words varying in size (or blur) were accompanied by backgrounds also varying in size (or blur), the traditional JOL illusions were reduced (Experiments 1, 2, 4, and 5) compared to when there were no backgrounds (Experiments 3a, 3b, and 4). Thus, the item-specific and contextual cues were used interactively. Further, the background manipulations also sometimes themselves led to metamemory illusions regarding JOLs for the to-be-remembered items. In general, there were robust individual differences in how participants used the cues, including how they incorporated the contextual cues into their JOL decisions. In part, this may explain why interactive cue utilization did not always emerge at the group level. In sum, we showed that context may affect JOLs both directly and indirectly by influencing participants' use of item-specific cues. These findings broaden our understanding of how cues may be utilized (e.g., Koriat, 1997) and integrated (e.g., Undorf, Söllner, and Bröder, 2018) in JOLs.

Intensifying the intensity illusion in judgments of learning: Modality and cue combinations.

We showed that judgments of learning (JOLs) were not affected by presentation modality in a list-learning task, although the typical font-size and loudness illusions emerged in that large-font visual presentations and loud auditory presentations elicited higher JOLs than their less intense counterparts. Further, when items were presented in both modalities simultaneously, large-font/quiet and small-font/loud items received similar JOLs (and were recalled similarly). Most importantly, when the intensity manipulation was compounded across modalities, the magnitude of the illusion increased beyond that observed in a single modality, showing the influence of combining cues. Whereas recall was still the same, large-font/loud items received higher JOLs than either small-font/loud items or large-font/quiet items, and not-intense items received very low JOLs. These differences emerged only when all conditions were presented within a single list and not in a between-subjects design, underscoring the importance of comparative judgments.

Examining motor evidence for the pause-then-cancel model of action-stopping: Insights from motor system physiology.

Stopping initiated actions is fundamental to adaptive behavior. Longstanding, single-process accounts of action-stopping have been challenged by recent, two-process, 'pause-then-cancel' models. These models propose that action-stopping involves two inhibitory processes: 1) a fast Pause process, which broadly suppresses the motor system as the result of detecting any salient event, and 2) a slower Cancel process, which involves motor suppression specific to the cancelled action. A purported signature of the Pause process is global suppression, or the reduced corticospinal excitability (CSE) of task-unrelated effectors early on in action-stopping. However, unlike the Pause process, few (if any) motor system signatures of a Cancel process have been identified. Here, we used single- and paired-pulse TMS methods to comprehensively measure the local physiological excitation and inhibition of both responding and task-unrelated motor effector systems during action-stopping. Specifically, we measured CSE, short-interval intracortical inhibition (SICI), and the duration of the cortical silent period (CSP). Consistent with key predictions from the pause-then-cancel model, CSE measurements at the responding effector indicated that additional suppression was necessary to counteract Go-related increases in CSE during-action-stopping, particularly at later timepoints. Increases in SICI on Stop-signal trials did not differ across responding and non-responding effectors, or across timepoints. This suggests SICI as a potential source of global suppression. Increases in CSP duration on Stop-signal trials were more prominent at later timepoints. SICI and CSP duration therefore appeared most consistent with the Pause and Cancel processes, respectively. Our study provides further evidence from motor system physiology that multiple inhibitory processes influence action-stopping.

Cross-Modal Contributions to Episodic Memory for Voices.

Multisensory context often facilitates perception and memory. In fact, encoding items within a multisensory context can improve memory even on strictly unisensory tests (i.e., when the multisensory context is absent). Prior studies that have consistently found these multisensory facilitation effects have largely employed multisensory contexts in which the stimuli were meaningfully related to the items targeting for remembering (e.g., pairing canonical sounds and images). Other studies have used unrelated stimuli as multisensory context. A third possible type of multisensory context is one that is environmentally related simply because the stimuli are often encountered together in the real world. We predicted that encountering such a multisensory context would also enhance memory through cross-modal associations, or representations relating to one's prior multisensory experience with that sort of stimuli in general. In two memory experiments, we used faces and voices of unfamiliar people as everyday stimuli individuals have substantial experience integrating the perceptual features of. We assigned participants to face- or voice-recognition groups and ensured that, during the study phase, half of the face or voice targets were encountered also with information in the other modality. Voices initially encoded along with faces were consistently remembered better, providing evidence that cross-modal associations could explain the observed multisensory facilitation.

Effect of impoverished information on multisensory integration in judgments of learning.

According to the principle of inverse effectiveness (PIE), weaker responses to information in one modality (i.e., unisensory) benefit more from additional information in a second modality (i.e., multisensory; Meredith & Stein, 1986). We suggest that the PIE may also inform whether perceptual fluency affects judgments of learning (JOLs). If JOLs follow the PIE, the differences in JOLs for multisensory and unisensory items should increase as the unisensory study items become harder to perceive. That is, an influence of perceptual fluency should prompt a similar, interactive pattern across perceptual responses and JOLs. In 3 experiments, we systematically varied the signal intensity or noise in 1 modality to examine how responses might change with the inclusion of information in a second modality. In Experiment 1, written words in several font sizes were sometimes accompanied by spoken equivalents. In Experiments 2 and 3, spoken words in various background noise levels were sometimes accompanied by visual speech articulations. Consistent with the PIE, the multisensory benefits in response time and/or correct identification increased as responses to unisensory information decreased. Also, the multisensory formats received higher JOLs than the unisensory formats; however, unlike the predictions from PIE, this difference did not increase as study items became harder to perceive. Experiment 3 extended this finding to participants' explicit beliefs. In multisensory settings, JOLs may rely more on theory- than data-driven processes. We suggest that broadly defined processing fluency may always contribute to JOLs, but, regarding perceptual information, JOLs appear to track perceptual attributes rather than perceptual fluency. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Face-voice space: Integrating visual and auditory cues in judgments of person distinctiveness.

Faces and voices each convey multiple cues enabling us to tell people apart. Research on face and voice distinctiveness commonly utilizes multidimensional space to represent these complex, perceptual abilities. We extend this framework to examine how a combined face-voice space would relate to its constituent face and voice spaces. Participants rated videos of speakers for their dissimilarity in face only, voice only, and face-voice together conditions. Multiple dimensional scaling (MDS) and regression analyses showed that whereas face-voice space more closely resembled face space, indicating visual dominance, face-voice distinctiveness was best characterized by a multiplicative integration of face-only and voice-only distinctiveness, indicating that auditory and visual cues are used interactively in person-distinctiveness judgments. Further, the multiplicative integration could not be explained by the small correlation found between face-only and voice-only distinctiveness. As an exploratory analysis, we next identified auditory and visual features that correlated with the dimensions in the MDS solutions. Features pertaining to facial width, lip movement, spectral centroid, fundamental frequency, and loudness variation were identified as important features in face-voice space. We discuss the implications of our findings in terms of person perception, recognition, and face-voice matching abilities.

Influence of Music Style and Rate on Repetitive Finger Tapping.

Auditory cues, including music, are commonly used in the treatment of persons with Parkinson's disease. Yet, how music style and movement rate modulate movement performance in persons with Parkinson's disease have been neglected and remain limited in healthy young populations. The purpose of this study was to determine how music style and movement rate influence movement performance in healthy young adults. Healthy participants were asked to perform repetitive finger movements at two pacing rates (70 and 140 beats per minute) for the following conditions: (a) a tone only, (b) activating music, and (c) relaxing music. Electromyography, movement kinematics, and variability were collected. Results revealed that the provision of music, regardless of style, reduced amplitude variability at both pacing rates. Intermovement interval was longer, and acceleration variability was reduced during both music conditions at the lower pacing rate only. These results may prove beneficial for designing therapeutic interventions for persons with Parkinson's disease.

McGurk Effect in Gender Identification: Vision Trumps Audition in Voice Judgments.

Demonstrations of non-speech McGurk effects are rare, mostly limited to emotion identification, and sometimes not considered true analogues. We presented videos of males and females singing a single syllable on the same pitch and asked participants to indicate the true range of the voice-soprano, alto, tenor, or bass. For one group of participants, the gender shown on the video matched the gender of the voice heard, and for the other group they were mismatched. Soprano or alto responses were interpreted as "female voice" decisions and tenor or bass responses as "male voice" decisions. Identification of the voice gender was 100% correct in the preceding audio-only condition. However, whereas performance was also 100% correct in the matched video/audio condition, it was only 31% correct in the mismatched video/audio condition. Thus, the visual gender information overrode the voice gender identification, showing a robust non-speech McGurk effect.