Designing Ecological Auditory Feedback on Lower Limb Kinematics for Hemiparetic Gait Training.

Auditory feedback has earlier been explored as a tool to enhance patient awareness of gait kinematics during rehabilitation. In this study, we devised and tested a novel set of concurrent feedback paradigms on swing phase kinematics in hemiparetic gait training. We adopted a user-centered design approach, where kinematic data recorded from 15 hemiparetic patients was used to design three feedback algorithms (wading sounds, abstract, musical) based on filtered gyroscopic data from four inexpensive wireless inertial units. The algorithms were tested (hands-on) by a focus group of five physiotherapists. They recommended that the abstract and musical algorithms be discarded due to sound quality and informational ambiguity. After modifying the wading algorithm (as per their feedback), we conducted a feasibility test involving nine hemiparetic patients and seven physiotherapists, where variants of the algorithm were applied to a conventional overground training session. Most patients found the feedback meaningful, enjoyable to use, natural-sounding, and tolerable for the typical training duration. Three patients exhibited immediate improvements in gait quality when the feedback was applied. However, minor gait asymmetries were found to be difficult to perceive in the feedback, and there was variability in receptiveness and motor change among the patients. We believe that our findings can advance current research in inertial sensor-based auditory feedback for motor learning enhancement during neurorehabilitation.

An Embodied Sonification Model for Sit-to-Stand Transfers.

Interactive sonification of biomechanical quantities is gaining relevance as a motor learning aid in movement rehabilitation, as well as a monitoring tool. However, existing gaps in sonification research (issues related to meaning, aesthetics, and clinical effects) have prevented its widespread recognition and adoption in such applications. The incorporation of embodied principles and musical structures in sonification design has gradually become popular, particularly in applications related to human movement. In this study, we propose a general sonification model for the sit-to-stand (STS) transfer, an important activity of daily living. The model contains a fixed component independent of the use-case, which represents the rising motion of the body as an ascending melody using the physical model of a flute. In addition, a flexible component concurrently sonifies STS features of clinical interest in a particular rehabilitative/monitoring situation. Here, we chose to represent shank angular jerk and movement stoppages (freezes), through perceptually salient pitch modulations and bell sounds. We outline the details of our technical implementation of the model. We evaluated the model by means of a listening test experiment with 25 healthy participants, who were asked to identify six normal and simulated impaired STS patterns from sonified versions containing various combinations of the constituent mappings of the model. Overall, we found that the participants were able to classify the patterns accurately (86.67 ± 14.69% correct responses with the full model, 71.56% overall), confidently (64.95 ± 16.52% self-reported rating), and in a timely manner (response time: 4.28 ± 1.52 s). The amount of sonified kinematic information significantly impacted classification accuracy. The six STS patterns were also classified with significantly different accuracy depending on their kinematic characteristics. Learning effects were seen in the form of increased accuracy and confidence with repeated exposure to the sound sequences. We found no significant accuracy differences based on the participants' level of music training. Overall, we see our model as a concrete conceptual and technical starting point for STS sonification design catering to rehabilitative and clinical monitoring applications.

Action expertise reduces brain activity for audiovisual matching actions: an fMRI study with expert drummers.

When we observe someone perform a familiar action, we can usually predict what kind of sound that action will produce. Musical actions are over-experienced by musicians and not by non-musicians, and thus offer a unique way to examine how action expertise affects brain processes when the predictability of the produced sound is manipulated. We used functional magnetic resonance imaging to scan 11 drummers and 11 age- and gender-matched novices who made judgments on point-light drumming movements presented with sound. In Experiment 1, sound was synchronized or desynchronized with drumming strikes, while in Experiment 2 sound was always synchronized, but the natural covariation between sound intensity and velocity of the drumming strike was maintained or eliminated. Prior to MRI scanning, each participant completed psychophysical testing to identify personal levels of synchronous and asynchronous timing to be used in the two fMRI activation tasks. In both experiments, the drummers' brain activation was reduced in motor and action representation brain regions when sound matched the observed movements, and was similar to that of novices when sound was mismatched. This reduction in neural activity occurred bilaterally in the cerebellum and left parahippocampal gyrus in Experiment 1, and in the right inferior parietal lobule, inferior temporal gyrus, middle frontal gyrus and precentral gyrus in Experiment 2. Our results indicate that brain functions in action-sound representation areas are modulated by multimodal action expertise.

Multisensory integration of drumming actions: musical expertise affects perceived audiovisual asynchrony.

We investigated the effect of musical expertise on sensitivity to asynchrony for drumming point-light displays, which varied in their physical characteristics (Experiment 1) or in their degree of audiovisual congruency (Experiment 2). In Experiment 1, 21 repetitions of three tempos x three accents x nine audiovisual delays were presented to four jazz drummers and four novices. In Experiment 2, ten repetitions of two audiovisual incongruency conditions x nine audiovisual delays were presented to 13 drummers and 13 novices. Participants gave forced-choice judgments of audiovisual synchrony. The results of Experiment 1 show an enhancement in experts' ability to detect asynchrony, especially for slower drumming tempos. In Experiment 2 an increase in sensitivity to asynchrony was found for incongruent stimuli; this increase, however, is attributable only to the novice group. Altogether the results indicated that through musical practice we learn to ignore variations in stimulus characteristics that otherwise would affect our multisensory integration processes.

Characterizing Movement Fluency in Musical Performance: Toward a Generic Measure for Technology Enhanced Learning.

Virtuosity in music performance is often associated with fast, precise, and efficient sound-producing movements. The generation of such highly skilled movements involves complex joint and muscle control by the central nervous system, and depends on the ability to anticipate, segment, and coarticulate motor elements, all within the biomechanical constraints of the human body. When successful, such motor skill should lead to what we characterize as fluency in musical performance. Detecting typical features of fluency could be very useful for technology-enhanced learning systems, assisting and supporting students during their individual practice sessions by giving feedback and helping them to adopt sustainable movement patterns. In this study, we propose to assess fluency in musical performance as the ability to smoothly and efficiently coordinate while accurately performing slow, transitionary, and rapid movements. To this end, the movements of three cello players and three drummers at different levels of skill were recorded with an optical motion capture system, while a wireless electromyography (EMG) system recorded the corresponding muscle activity from relevant landmarks. We analyzed the kinematic and coarticulation characteristics of these recordings separately and then propose a combined model of fluency in musical performance predicting music sophistication. Results suggest that expert performers' movements are characterized by consistently smooth strokes and scaling of muscle phasic coactivation. The explored model of fluency as a function of movement smoothness and coarticulation patterns was shown to be limited by the sample size, but it serves as a proof of concept. Results from this study show the potential of a technology-enhanced objective measure of fluency in musical performance, which could lead to improved practices for aspiring musicians, instructors, and researchers.

Estimating internal drift and just noticeable difference in the perception of continuous tempo drift. A new method.

Is there such a thing as an internal representation of a "steady tempo" and is this representation itself free from tempo drift? To investigate this question, we propose a new method for studying detection of continuous tempo drift.