Movement trajectories as a window into the dynamics of emerging neural representations.

The rapid transformation of sensory inputs into meaningful neural representations is critical to adaptive human behaviour. While non-invasive neuroimaging methods are the de-facto method for investigating neural representations, they remain expensive, not widely available, time-consuming, and restrictive. Here we show that movement trajectories can be used to measure emerging neural representations with fine temporal resolution. By combining online computer mouse-tracking and publicly available neuroimaging data via representational similarity analysis (RSA), we show that movement trajectories track the unfolding of stimulus- and category-wise neural representations along key dimensions of the human visual system. We demonstrate that time-resolved representational structures derived from movement trajectories overlap with those derived from M/EEG (albeit delayed) and those derived from fMRI in functionally-relevant brain areas. Our findings highlight the richness of movement trajectories and the power of the RSA framework to reveal and compare their information content, opening new avenues to better understand human perception.

Behavioral dynamics of conversation, (mis)communication and coordination in noisy environments.

During conversations people coordinate simultaneous channels of verbal and nonverbal information to hear and be heard. But the presence of background noise levels such as those found in cafes and restaurants can be a barrier to conversational success. Here, we used speech and motion-tracking to reveal the reciprocal processes people use to communicate in noisy environments. Conversations between twenty-two pairs of typical-hearing adults were elicited under different conditions of background noise, while standing or sitting around a table. With the onset of background noise, pairs rapidly adjusted their interpersonal distance and speech level, with the degree of initial change dependent on noise level and talker configuration. Following this transient phase, pairs settled into a sustaining phase in which reciprocal speech and movement-based coordination processes synergistically maintained effective communication, again with the magnitude of stability of these coordination processes covarying with noise level and talker configuration. Finally, as communication breakdowns increased at high noise levels, pairs exhibited resetting behaviors to help restore communication-decreasing interpersonal distance and/or increasing speech levels in response to communication breakdowns. Approximately 78 dB SPL defined a threshold where behavioral processes were no longer sufficient for maintaining effective conversation and communication breakdowns rapidly increased.

An online browser-based attentional blink replication using visual objects.

The complex relationship between attention and visual perception can be exemplified and investigated through the Attentional Blink. The attentional blink is characterised by impaired attention to the second of two target stimuli, when both occur within 200 - 500ms. The attentional blink has been well studied in experimental lab settings. However, despite the rise of online methods for behavioural research, their suitability for studying the attentional blink has not been fully addressed yet, the main concern being the lack of control and timing variability for stimulus presentation. Here, we investigated the suitability of online testing for studying the attentional blink with visual objects. Our results show a clear attentional blink effect between 200 to 400ms following the distractor including a Lag 1 sparing effect in line with previous research despite significant inter-subject and timing variability. This work demonstrates the suitability of online methods for studying the attentional blink with visual objects, opening new avenues to explore its underlying processes.

Neural tracking of visual periodic motion.

Periodicity is a fundamental property of biological systems, including human movement systems. Periodic movements support displacements of the body in the environment as well as interactions and communication between individuals. Here, we use electroencephalography (EEG) to investigate the neural tracking of visual periodic motion, and more specifically, the relevance of spatiotemporal information contained at and between their turning points. We compared EEG responses to visual sinusoidal oscillations versus nonlinear Rayleigh oscillations, which are both typical of human movements. These oscillations contain the same spatiotemporal information at their turning points but differ between turning points, with Rayleigh oscillations having an earlier peak velocity, shown to increase an individual's capacity to produce accurately synchronized movements. EEG analyses highlighted the relevance of spatiotemporal information between the turning points by showing that the brain precisely tracks subtle differences in velocity profiles, as indicated by earlier EEG responses for Rayleigh oscillations. The results suggest that the brain is particularly responsive to velocity peaks in visual periodic motion, supporting their role in conveying behaviorally relevant timing information at a neurophysiological level. The results also suggest key functions of neural oscillations in the Alpha and Beta frequency bands, particularly in the right hemisphere. Together, these findings provide insights into the neural mechanisms underpinning the processing of visual periodic motion and the critical role of velocity peaks in enabling proficient visuomotor synchronization.

Spatial and temporal (non)binding of audiovisual rhythms in sensorimotor synchronisation.

Human movement synchronisation with moving objects strongly relies on visual input. However, auditory information also plays an important role, since real environments are intrinsically multimodal. We used electroencephalography (EEG) frequency tagging to investigate the selective neural processing and integration of visual and auditory information during motor tracking and tested the effects of spatial and temporal congruency between audiovisual modalities. EEG was recorded while participants tracked with their index finger a red flickering (rate fV = 15 Hz) dot oscillating horizontally on a screen. The simultaneous auditory stimulus was modulated in pitch (rate fA = 32 Hz) and lateralised between left and right audio channels to induce perception of a periodic displacement of the sound source. Audiovisual congruency was manipulated in terms of space in Experiment 1 (no motion, same direction or opposite direction), and timing in Experiment 2 (no delay, medium delay or large delay). For both experiments, significant EEG responses were elicited at fV and fA tagging frequencies. It was also hypothesised that intermodulation products corresponding to the nonlinear integration of visual and auditory stimuli at frequencies fV ± fA would be elicited, due to audiovisual integration, especially in Congruent conditions. However, these components were not observed. Moreover, synchronisation and EEG results were not influenced by congruency manipulations, which invites further exploration of the conditions which may modulate audiovisual processing and the motor tracking of moving objects.

Unique contributions of perceptual and conceptual humanness to object representations in the human brain.

The human brain is able to quickly and accurately identify objects in a dynamic visual world. Objects evoke different patterns of neural activity in the visual system, which reflect object category memberships. However, the underlying dimensions of object representations in the brain remain unclear. Recent research suggests that objects similarity to humans is one of the main dimensions used by the brain to organise objects, but the nature of the human-similarity features driving this organisation are still unknown. Here, we investigate the relative contributions of perceptual and conceptual features of humanness to the representational organisation of objects in the human visual system. We collected behavioural judgements of human-similarity of various objects, which were compared with time-resolved neuroimaging responses to the same objects. The behavioural judgement tasks targeted either perceptual or conceptual humanness features to determine their respective contribution to perceived human-similarity. Behavioural and neuroimaging data revealed significant and unique contributions of both perceptual and conceptual features of humanness, each explaining unique variance in neuroimaging data. Furthermore, our results showed distinct spatio-temporal dynamics in the processing of conceptual and perceptual humanness features, with later and more lateralised brain responses to conceptual features. This study highlights the critical importance of social requirements in information processing and organisation in the human brain.

Lateralised dynamic modulations of corticomuscular coherence associated with bimanual learning of rhythmic patterns.

Human movements are spontaneously attracted to auditory rhythms, triggering an automatic activation of the motor system, a central phenomenon to music perception and production. Cortico-muscular coherence (CMC) in the theta, alpha, beta and gamma frequencies has been used as an index of the synchronisation between cortical motor regions and the muscles. Here we investigated how learning to produce a bimanual rhythmic pattern composed of low- and high-pitch sounds affects CMC in the beta frequency band. Electroencephalography (EEG) and electromyography (EMG) from the left and right First Dorsal Interosseus and Flexor Digitorum Superficialis muscles were concurrently recorded during constant pressure on a force sensor held between the thumb and index finger while listening to the rhythmic pattern before and after a bimanual training session. During the training, participants learnt to produce the rhythmic pattern guided by visual cues by pressing the force sensors with their left or right hand to produce the low- and high-pitch sounds, respectively. Results revealed no changes after training in overall beta CMC or beta oscillation amplitude, nor in the correlation between the left and right sides for EEG and EMG separately. However, correlation analyses indicated that left- and right-hand beta EEG-EMG coherence were positively correlated over time before training but became uncorrelated after training. This suggests that learning to bimanually produce a rhythmic musical pattern reinforces lateralised and segregated cortico-muscular communication.

Mapping between sound, brain and behaviour: four-level framework for understanding rhythm processing in humans and non-human primates.

Humans perceive and spontaneously move to one or several levels of periodic pulses (a meter, for short) when listening to musical rhythm, even when the sensory input does not provide prominent periodic cues to their temporal location. Here, we review a multi-levelled framework to understanding how external rhythmic inputs are mapped onto internally represented metric pulses. This mapping is studied using an approach to quantify and directly compare representations of metric pulses in signals corresponding to sensory inputs, neural activity and behaviour (typically body movement). Based on this approach, recent empirical evidence can be drawn together into a conceptual framework that unpacks the phenomenon of meter into four levels. Each level highlights specific functional processes that critically enable and shape the mapping from sensory input to internal meter. We discuss the nature, constraints and neural substrates of these processes, starting with fundamental mechanisms investigated in macaque monkeys that enable basic forms of mapping between simple rhythmic stimuli and internally represented metric pulse. We propose that human evolution has gradually built a robust and flexible system upon these fundamental processes, allowing more complex levels of mapping to emerge in musical behaviours. This approach opens promising avenues to understand the many facets of rhythmic behaviours across individuals and species. This article is part of the theme issue 'Synchrony and rhythm interaction: from the brain to behavioural ecology'.

Dynamic modulation of cortico-muscular coupling during real and imagined sensorimotor synchronisation.

People have a natural and intrinsic ability to coordinate body movements with rhythms surrounding them, known as sensorimotor synchronisation. This can be observed in daily environments, when dancing or singing along with music, or spontaneously walking, talking or applauding in synchrony with one another. However, the neurophysiological mechanisms underlying accurately synchronised movement with selected rhythms in the environment remain unclear. Here we studied real and imagined sensorimotor synchronisation with interleaved auditory and visual rhythms using cortico-muscular coherence (CMC) to better understand the processes underlying the preparation and execution of synchronised movement. Electroencephalography (EEG), electromyography (EMG) from the finger flexors, and continuous force signals were recorded in 20 participants during tapping and imagined tapping with discrete stimulus sequences consisting of alternating auditory beeps and visual flashes. The results show that the synchronisation between cortical and muscular activity in the beta (14-38 Hz) frequency band becomes time-locked to the taps executed in synchrony with the visual and auditory stimuli. Dynamic modulation in CMC also occurred when participants imagined tapping with the visual stimuli, but with lower amplitude and a different temporal profile compared to real tapping. These results suggest that CMC does not only reflect changes related to the production of the synchronised movement, but also to its preparation, which appears heightened under higher attentional demands imposed when synchronising with the visual stimuli. These findings highlight a critical role of beta band neural oscillations in the cortical-muscular coupling underlying sensorimotor synchronisation.

Vocal interaction during rhythmic joint action stabilizes interpersonal coordination and individual movement timing.

Because work songs are ubiquitous around the world, singing while working and performing a task with a coactor is presumably beneficial for both joint action and individual task performance. The present study investigated the impact of interpersonal rhythmic vocal interaction on interpersonal phase relations and on individual motor timing performance, which was evaluated by a synchronization-continuation paradigm requiring whole-body movement with or without visual contact. Participants repeated the syllable "tah" or remained silent in a manipulation of vocal interaction, and they were oriented toward or away from their partner to manipulate visual interaction. Results indicated the occurrence of spontaneous interpersonal coordination, evidenced by interpersonal phase relations that were closer to 0° and less variable when participants interacted both visually and vocally. At the individual level, visual interaction increased the variability of synchronization with the metronome but did not modulate the variability of continuation movements, whereas vocal interaction helped to decrease the variability of synchronization and continuation movements. Visual interaction therefore degraded individual movement timing while vocal interaction improved it. Communication via the auditory modality may play a compensatory role in naturalistic contexts where visual contact has potential destabilizing effects. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

The influence of a conductor and co-performer on auditory-motor synchronisation, temporal prediction, and ancillary entrainment in a musical drumming task.

Interpersonal coordination is exemplified in ensemble musicians, who coordinate their actions deliberately in order to achieve temporal synchronisation in their performances. However, musicians also move parts of their bodies unintentionally or spontaneously, sometimes in ways that do not directly produce sound from their instruments. Musicians' movements-intentional or otherwise-provide visual signals to co-performers, which might facilitate temporal synchronisation. In large ensembles, a conductor also provides a visual cue, which has been shown to enhance synchronisation. In the present study, we tested how visual cues from a co-performer and a conductor affect processes of temporal anticipation, synchronisation, and ancillary movements in a sample of primarily non-musicians. We used a dyadic synchronisation drumming task, in which paired participants drummed to the beat of tempo-changing music. We manipulated visual access between partners and a virtual conductor. Results showed that the conductor improved synchronisation with the music, but synchrony with the music did not improve when partners could see each other. Temporal prediction was improved when partners saw the conductor, but not each other. Ancillary movements of the head were more synchronised between partners when they could see each other, and greater ancillary synchrony at beat-related frequencies of movement was associated with greater drumming synchrony. These results suggest that compatible audio-visual cues can improve intentional synchronisation, that ancillary movements are affected by seeing a partner, and that attended vs. incidental visual cues thus have partially dissociable effects on temporal coordination during joint action.

Accent-induced Modulation of Neural and Movement Patterns during Spontaneous Synchronization to Auditory Rhythms.

Human rhythmic movements spontaneously synchronize with auditory rhythms at various frequency ratios. The emergence of more complex relationships-for instance, frequency ratios of 1:2 and 1:3-is enhanced by adding a congruent accentuation pattern (binary for 1:2 and ternary for 1:3), resulting in a 1:1 movement-accentuation relationship. However, this benefit of accentuation on movement synchronization appears to be stronger for the ternary pattern than for the binary pattern. Here, we investigated whether this difference in accent-induced movement synchronization may be related to a difference in the neural tracking of these accentuation profiles. Accented and control unaccented auditory sequences were presented to participants who concurrently produced finger taps at their preferred frequency, and spontaneous movement synchronization was measured. EEG was recorded during passive listening to each auditory sequence. The results revealed that enhanced movement synchronization with ternary accentuation was accompanied by enhanced neural tracking of this pattern. Larger EEG responses at the accentuation frequency were found for the ternary pattern compared with the binary pattern. Moreover, the amplitude of accent-induced EEG responses was positively correlated with the magnitude of accent-induced movement synchronization across participants. Altogether, these findings show that the dynamics of spontaneous auditory-motor synchronization is strongly driven by the multi-time-scale sensory processing of auditory rhythms, highlighting the importance of considering neural responses to rhythmic sequences for understanding and enhancing synchronization performance.

Neural multimodal integration underlying synchronization with a co-performer in music: Influences of motor expertise and visual information.

Sensorimotor synchronization is a general skill that musicians have developed to the highest levels of performance, including synchronization in timing and articulation. This study investigated neurocognitive processes that enable such high levels of performance, specifically testing the relevance of 1) motor resonance and sharing high levels of motor expertise with the co-performer, and 2) the role of visual information in addition to auditory information. Musicians with varying levels of piano expertise (including non-pianists) performed on a single piano key with their right hand along with recordings of a pianist who performed simple melodies with the left hand, synchronizing timing and articulation. The prerecorded performances were presented as audio-only, audio-video, or audio-animation stimuli. Double pulse Transcranial Magnetic Stimulation (dTMS) was applied to test the contribution of the right dorsal premotor cortex (dPMC), an area implicated in motor resonance with observed (left-hand) actions, and the contribution of the right intraparietal sulcus (IPS), an area known for multisensory binding. Results showed effects of dTMS in the conditions that included visual information. IPS stimulation improved synchronization, although this effect was found to reverse in the video condition with higher levels of piano expertise. dPMC stimulation improved or worsened synchronization ability. Level of piano expertise was found to influence this direction in the video condition. These results indicate that high levels of relevant motor expertise are required to beneficially employ visual and motor information of a co-performer for sensorimotor synchronization, which may qualify the effects of dPMC and IPS involvement.

Neural and Behavioral Evidence for Frequency-Selective Context Effects in Rhythm Processing in Humans.

When listening to music, people often perceive and move along with a periodic meter. However, the dynamics of mapping between meter perception and the acoustic cues to meter periodicities in the sensory input remain largely unknown. To capture these dynamics, we recorded the electroencephalography while nonmusician and musician participants listened to nonrepeating rhythmic sequences, where acoustic cues to meter frequencies either gradually decreased (from regular to degraded) or increased (from degraded to regular). The results revealed greater neural activity selectively elicited at meter frequencies when the sequence gradually changed from regular to degraded compared with the opposite. Importantly, this effect was unlikely to arise from overall gain, or low-level auditory processing, as revealed by physiological modeling. Moreover, the context effect was more pronounced in nonmusicians, who also demonstrated facilitated sensory-motor synchronization with the meter for sequences that started as regular. In contrast, musicians showed weaker effects of recent context in their neural responses and robust ability to move along with the meter irrespective of stimulus degradation. Together, our results demonstrate that brain activity elicited by rhythm does not only reflect passive tracking of stimulus features, but represents continuous integration of sensory input with recent context.

Dynamic Modulation of Beta Band Cortico-Muscular Coupling Induced by Audio-Visual Rhythms.

Human movements often spontaneously fall into synchrony with auditory and visual environmental rhythms. Related behavioral studies have shown that motor responses are automatically and unintentionally coupled with external rhythmic stimuli. However, the neurophysiological processes underlying such motor entrainment remain largely unknown. Here, we investigated with electroencephalography (EEG) and electromyography (EMG) the modulation of neural and muscular activity induced by periodic audio and/or visual sequences. The sequences were presented at either 1 or 2 Hz, while participants maintained constant finger pressure on a force sensor. The results revealed that although there was no change of amplitude in participants' EMG in response to the sequences, the synchronization between EMG and EEG recorded over motor areas in the beta (12-40 Hz) frequency band was dynamically modulated, with maximal coherence occurring about 100 ms before each stimulus. These modulations in beta EEG-EMG motor coherence were found for the 2-Hz audio-visual sequences, confirming at a neurophysiological level the enhancement of motor entrainment with multimodal rhythms that fall within preferred perceptual and movement frequency ranges. Our findings identify beta band cortico-muscular coupling as a potential underlying mechanism of motor entrainment, further elucidating the nature of the link between sensory and motor systems in humans.

Accent-induced stabilization of spontaneous auditory-motor synchronization.

Humans spontaneously synchronize their movements with external auditory rhythms such as a metronome or music. Although such synchronization preferentially occurs toward a simple 1:1 movement-sound frequency ratio, the parameters facilitating spontaneous synchronization to more complex frequency ratios remain largely unclear. The present study investigates the dynamics of spontaneous auditory-motor synchronization at a range of frequency ratios between movement and sound, and examines the benefit of simple accentuation pattern on synchronization emergence and stability. Participants performed index finger oscillations at their preferred tempo while listening to a metronome presented at either their preferred tempo, or twice or three times faster (frequency ratios of 1:1, 1:2 or 1:3) with different patterns of accentuation (unaccented, binary or ternary accented), and no instruction to synchronize. Participants' movements were spontaneously entrained to the auditory stimuli in the three different frequency ratio conditions. Moreover, the emergence and stability of the modes of coordination were influenced by the interaction between frequency ratio and pattern of accentuation. Coherent patterns, such as a 1:3 frequency ratio supported by a ternary accentuation, facilitated the emergence and stability of the corresponding mode of coordination. Furthermore, ternary accentuation induced a greater gain in stability for the corresponding mode of coordination than was observed with binary accentuation. Together, these findings demonstrate the importance of matching accentuation pattern and movement tempo for enhanced synchronization, opening new perspectives for stabilizing complex rhythmic motor behaviors, such as running.

Effects of pitch and tempo of auditory rhythms on spontaneous movement entrainment and stabilisation.

Human movements spontaneously entrain to auditory rhythms, which can help to stabilise movements in time and space. The properties of auditory rhythms supporting the occurrence of this phenomenon, however, remain largely unclear. Here, we investigate in two experiments the effects of pitch and tempo on spontaneous movement entrainment and stabilisation. We examined spontaneous entrainment of hand-held pendulum swinging in time with low-pitched (100 Hz) and high-pitched (1600 Hz) metronomes to test whether low pitch favours movement entrainment and stabilisation. To investigate whether stimulation and movement tempi moderate these effects of pitch, we manipulated (1) participants' preferred movement tempo by varying pendulum mechanical constraints (Experiment 1) and (2) stimulation tempo, which was either equal to, or slightly slower or faster (± 10%) than the participant's preferred movement tempo (Experiment 2). The results showed that participants' movements spontaneously entrained to auditory rhythms, and that this effect was stronger with low-pitched rhythms independently of stimulation and movement tempi. Results also indicated that auditory rhythms can lead to increased movement amplitude and stabilisation of movement tempo and amplitude, particularly when low-pitched. However, stabilisation effects were found to depend on intrinsic movement variability. Auditory rhythms decreased movement variability of individuals with higher intrinsic variability but increased movement variability of individuals with lower intrinsic variability. These findings provide new insights into factors that influence auditory-motor entrainment and how they may be optimised to enhance movement efficiency.

Neural tracking and integration of 'self' and 'other' in improvised interpersonal coordination.

Humans coordinate their movements with one another in a range of everyday activities and skill domains. Optimal joint performance requires the continuous anticipation of and adaptation to each other's movements, especially when actions are spontaneous rather than pre-planned. Here we employ dual-EEG and frequency-tagging techniques to investigate how the neural tracking of self- and other-generated movements supports interpersonal coordination during improvised motion. LEDs flickering at 5.7 and 7.7 Hz were attached to participants' index fingers in 28 dyads as they produced novel patterns of synchronous horizontal forearm movements. EEG responses at these frequencies revealed enhanced neural tracking of self-generated movement when leading and of other-generated movements when following. A marker of self-other integration at 13.4 Hz (inter-modulation frequency of 5.7 and 7.7 Hz) peaked when no leader was designated, and mutual adaptation and movement synchrony were maximal. Furthermore, the amplitude of EEG responses reflected differences in the capacity of dyads to synchronize their movements, offering a neurophysiologically grounded perspective for understanding perceptual-motor mechanisms underlying joint action.

The influence of pacer-movement continuity and pattern matching on auditory-motor synchronisation.

People commonly move along with auditory rhythms in the environment. Although the processes underlying such sensorimotor synchronisation have been extensively investigated in the previous research, the properties of auditory rhythms that facilitate the synchronisation remain largely unclear. This study explored the possible benefits of a continuity matching between auditory pacers and the movement produced as well as of a spatial pattern matching that has been previously demonstrated with visual pacers. Participants synchronised either finger tapping or forearm oscillations with either discrete or continuous pacers. The pacers had either a spatial pattern (left-right panning) that matched the movement pattern produced or no spatial pattern. The accuracy and variability of synchronisation were assessed by the mean and standard deviation of the asynchronies, respectively, between participant's movement and the pacers. Results indicated that synchronisation was more accurate and less variable for discrete pacers and continuous movement (i.e., forearm oscillations). The interaction between those two factors involved a more complex relationship than a simple continuity match benefit. Although synchronisation variability increased with continuous pacers for both types of movement, this increase was smaller for continuous movement than discrete movement, suggesting that continuous movement is more beneficial only for continuous pacers. Moreover, the results revealed limited benefits of spatial pattern matching on auditory-motor synchronisation variability, which might be due to lower spatial resolution of the auditory sensory modality. Together, these findings confirm that sensorimotor synchronisation is modulated by complex relations between pacer and movement properties.

Preferred frequency ratios for spontaneous auditory-motor synchronization: Dynamical stability and hysteresis.

Humans spontaneously synchronize their movements with external auditory rhythms such as a metronome or music. Although such synchronization preferentially occurs toward simple 1:1 movement-stimulus frequency ratio, the extent to which spontaneous synchronization can also occur toward more complex frequency ratios remains largely unclear. The present study investigates the occurrence and dynamical stability of spontaneous auditory-motor synchronization at multiple frequency ratios. Participants performed index finger oscillations at their preferred tempo while listening to auditory metronomes with frequency progressively increasing or decreasing between 1 Hz and 6 Hz. The results demonstrated that participants' movements were not only entrained toward the 1:1 frequency ratio but also toward the 1:2 ratio. The occurrence and stability of these ratios differed as a function of the direction of frequency change. Participants synchronized to the 1:2 ratio and transitioned to a 1:1 ratio in the descending condition. In the ascending condition only the 1:1 ratio was sustained, for a longer extent than in the descending condition. These results show that the initial coordination pattern influenced pattern transition, demonstrating the occurrence of a hysteresis effect that is typical of complex system dynamics. These findings provide new insight into the mechanisms underlying the occurrence and stability of spontaneous movement synchronization to auditory rhythms.