Remote Assessment for ALS using Multimodal Dialog Agents: Data Quality, Feasibility and Task Compliance.

We investigate the feasibility, task compliance and audiovisual data quality of a multimodal dialog-based solution for remote assessment of Amyotrophic Lateral Sclerosis (ALS). 53 people with ALS and 52 healthy controls interacted with Tina, a cloud-based conversational agent, in performing speech tasks designed to probe various aspects of motor speech function while their audio and video was recorded. We rated a total of 250 recordings for audio/video quality and participant task compliance, along with the relative frequency of different issues observed. We observed excellent compliance (98%) and audio (95.2%) and visual quality rates (84.8%), resulting in an overall yield of 80.8% recordings that were both compliant and of high quality. Furthermore, recording quality and compliance were not affected by level of speech severity and did not differ significantly across end devices. These findings support the utility of dialog systems for remote monitoring of speech in ALS.

Impaired Speaking-Induced Suppression in Alzheimer's Disease.

Alzheimer's disease (AD) is a neurodegenerative disease involving cognitive impairment and abnormalities in speech and language. Here, we examine how AD affects the fidelity of auditory feedback predictions during speaking. We focus on the phenomenon of speaking-induced suppression (SIS), the auditory cortical responses' suppression during auditory feedback processing. SIS is determined by subtracting the magnitude of auditory cortical responses during speaking from listening to playback of the same speech. Our state feedback control (SFC) model of speech motor control explains SIS as arising from the onset of auditory feedback matching a prediction of that feedback onset during speaking, a prediction that is absent during passive listening to playback of the auditory feedback. Our model hypothesizes that the auditory cortical response to auditory feedback reflects the mismatch with the prediction: small during speaking, large during listening, with the difference being SIS. Normally, during speaking, auditory feedback matches its predictions, then SIS will be large. Any reductions in SIS will indicate inaccuracy in auditory feedback prediction not matching the actual feedback. We investigated SIS in AD patients [n = 20; mean (SD) age, 60.77 (10.04); female (%), 55.00] and healthy controls [n = 12; mean (SD) age, 63.68 (6.07); female (%), 83.33] through magnetoencephalography (MEG)-based functional imaging. We found a significant reduction in SIS at ∼100 ms in AD patients compared with healthy controls (linear mixed effects model, F (1,57.5) = 6.849, p = 0.011). The results suggest that AD patients generate inaccurate auditory feedback predictions, contributing to abnormalities in AD speech.

Speech, Facial and Fine Motor Features for Conversation-Based Remote Assessment and Monitoring of Parkinson's Disease.

We present a cloud-based multimodal dialogue platform for the remote assessment and monitoring of speech, facial and fine motor function in Parkinson's Disease (PD) at scale, along with a preliminary investigation of the efficacy of the various metrics automatically extracted by the platform. 22 healthy controls and 38 people with Parkinson's Disease (pPD) were instructed to complete four interactive sessions, spaced a week apart, on the platform. Each session involved a battery of tasks designed to elicit speech, facial movements and finger movements. We find that speech, facial kinematic and finger movement dexterity metrics show statistically significant differences between controls and pPD. We further investigate the sensitivity, specificity, reliability and generalisability of these metrics. Our results offer encouraging evidence for the utility of automatically-extracted audiovisual analytics in remote mon-itoring of PD and other movement disorders.

Temporal specificity of abnormal neural oscillations during phonatory events in laryngeal dystonia.

Laryngeal dystonia is a debilitating disorder of voicing in which the laryngeal muscles are intermittently in spasm resulting in involuntary interruptions during speech. The central pathophysiology of laryngeal dystonia, underlying computational impairments in vocal motor control, remains poorly understood. Although prior imaging studies have found aberrant activity in the CNS during phonation in patients with laryngeal dystonia, it is not known at what timepoints during phonation these abnormalities emerge and what function may be impaired. To investigate this question, we recruited 22 adductor laryngeal dystonia patients (15 female, age range = 28.83-72.46 years) and 18 controls (eight female, age range = 27.40-71.34 years). We leveraged the fine temporal resolution of magnetoencephalography to monitor neural activity around glottal movement onset, subsequent voice onset and after the onset of pitch feedback perturbations. We examined event-related beta-band (12-30 Hz) and high-gamma-band (65-150 Hz) neural oscillations. Prior to glottal movement onset, we observed abnormal frontoparietal motor preparatory activity. After glottal movement onset, we observed abnormal activity in the somatosensory cortex persisting through voice onset. Prior to voice onset and continuing after, we also observed abnormal activity in the auditory cortex and the cerebellum. After pitch feedback perturbation onset, we observed no differences between controls and patients in their behavioural responses to the perturbation. But in patients, we did find abnormal activity in brain regions thought to be involved in the auditory feedback control of vocal pitch (premotor, motor, somatosensory and auditory cortices). Our study results confirm the abnormal processing of somatosensory feedback that has been seen in other studies. However, there were several remarkable findings in our study. First, patients have impaired vocal motor activity even before glottal movement onset, suggesting abnormal movement preparation. These results are significant because (i) they occur before movement onset, abnormalities in patients cannot be ascribed to deficits in vocal performance and (ii) they show that neural abnormalities in laryngeal dystonia are more than just abnormal responses to sensory feedback during phonation as has been hypothesized in some previous studies. Second, abnormal auditory cortical activity in patients begins even before voice onset, suggesting abnormalities in setting up auditory predictions before the arrival of auditory feedback at voice onset. Generally, activation abnormalities identified in key brain regions within the speech motor network around various phonation events not only provide temporal specificity to neuroimaging phenotypes in laryngeal dystonia but also may serve as potential therapeutic targets for neuromodulation.

Speech compensation responses and sensorimotor adaptation to formant feedback perturbations.

Control of speech formants is important for the production of distinguishable speech sounds and is achieved with both feedback and learned feedforward control. However, it is unclear whether the learning of feedforward control involves the mechanisms of feedback control. Speakers have been shown to compensate for unpredictable transient mid-utterance perturbations of pitch and loudness feedback, demonstrating online feedback control of these speech features. To determine whether similar feedback control mechanisms exist in the production of formants, responses to unpredictable vowel formant feedback perturbations were examined. Results showed similar within-trial compensatory responses to formant perturbations that were presented at utterance onset and mid-utterance. The relationship between online feedback compensation to unpredictable formant perturbations and sensorimotor adaptation to consistent formant perturbations was further examined. Within-trial online compensation responses were not correlated with across-trial sensorimotor adaptation. A detailed analysis of within-trial time course dynamics across trials during sensorimotor adaptation revealed that across-trial sensorimotor adaptation responses did not result from an incorporation of within-trial compensation response. These findings suggest that online feedback compensation and sensorimotor adaptation are governed by distinct neural mechanisms. These findings have important implications for models of speech motor control in terms of how feedback and feedforward control mechanisms are implemented.

Sensorimotor adaptation of speech depends on the direction of auditory feedback alteration.

A hallmark feature of speech motor control is its ability to learn to anticipate and compensate for persistent feedback alterations, a process referred to as sensorimotor adaptation. Because this process involves adjusting articulation to counter the perceived effects of altering acoustic feedback, there are a number of factors that affect it, including the complex relationship between acoustics and articulation and non-uniformities of speech perception. As a consequence, sensorimotor adaptation is hypothesised to vary as a function of the direction of the applied auditory feedback alteration in vowel formant space. This hypothesis was tested in two experiments where auditory feedback was altered in real time, shifting the frequency values of the first and second formants (F1 and F2) of participants' speech. Shifts were designed on a subject-by-subject basis and sensorimotor adaptation was quantified with respect to the direction of applied shift, normalised for individual speakers. Adaptation was indeed found to depend on the direction of the applied shift in vowel formant space, independent of shift magnitude. These findings have implications for models of sensorimotor adaptation of speech.

Establishing a Causal Role for Medial Prefrontal Cortex in Reality Monitoring.

Reality monitoring is defined as the ability to distinguish internally self-generated information from externally-derived information. Functional imaging studies have consistently found that the medial prefrontal cortex (mPFC) is a key brain region subserving reality monitoring. This study aimed to determine a causal role for mPFC in reality monitoring using navigated repetitive transcranial magnetic stimulation (nrTMS). In a subject-blinded sham-controlled crossover design, healthy individuals received either active or sham nrTMS targeting mPFC. Active modulation of mPFC using nrTMS at a frequency of 10 Hz, significantly improved identification of both self-generated and externally-derived information during reality monitoring, when compared to sham or baseline. Targeted excitatory modulation of mPFC also improved positive mood, reduced negative mood, and increased overall alertness/arousal. These results establish optimal nrTMS dosing parameters that maximized tolerability/comfort and induced significant neuromodulatory effects in the mPFC target. Importantly, this is a proof-of-concept study that establishes the mPFC as a novel brain target that can be stimulated with nrTMS to causally impact both higher-order reality monitoring and mood.

Abnormally increased vocal responses to pitch feedback perturbations in patients with cerebellar degeneration.

Cerebellar degeneration (CD) has deleterious effects on speech motor behavior. Recently, a dissociation between feedback and feedforward control of speaking was observed in CD: Whereas CD patients exhibited reduced adaptation across trials to consistent formant feedback alterations, they showed enhanced within-trial compensation for unpredictable formant feedback perturbations. In this study, it was found that CD patients exhibit abnormally increased within-trial vocal compensation responses to unpredictable pitch feedback perturbations. Taken together with recent findings, the results indicate that CD is associated with a general hypersensitivity to auditory feedback during speaking.

Beta-band activity in medial prefrontal cortex predicts source memory encoding and retrieval accuracy.

Reality monitoring is defined as the ability to distinguish internally self-generated information from externally-derived information. The medial prefrontal cortex (mPFC) is a key brain region subserving reality monitoring and has been shown to be activated specifically during the retrieval of self-generated information. However, it is unclear if mPFC is activated during the encoding of self-generated information into memory. If so, it is important to understand whether successful retrieval of self-generated information critically depends on enhanced neural activity within mPFC during initial encoding of this self-generated information. We used magnetoencephalographic imaging (MEGI) to determine the timing and location of cortical activity during a reality-monitoring task involving self generated contextual source memory encoding and retrieval. We found both during encoding and retrieval of self-generated information, when compared to externally-derived information, mPFC showed significant task induced oscillatory power modulation in the beta-band. During initial encoding of self-generated information, greater mPFC beta-band power reductions occurred within a time window of -700 ms to -500 ms prior to vocalization. This increased activity in mPFC was not observed during encoding of externally-derived information. Additionally, increased mPFC activity during encoding of self-generated information predicted subsequent retrieval accuracy of this self-generated information. Beta-band activity in mPFC was also observed during the initial retrieval of self-generated information within a time window of 300 to 500 ms following stimulus onset and correlated with accurate retrieval performance of self-generated information. Together, these results further highlight the importance of mPFC in mediating the initial generation and awareness of participants' internal thoughts.

Neural correlates of abnormal auditory feedback processing during speech production in Alzheimer's disease.

Accurate integration of sensory inputs and motor commands is essential to achieve successful behavioral goals. A robust model of sensorimotor integration is the pitch perturbation response, in which speakers respond rapidly to shifts of the pitch in their auditory feedback. In a previous study, we demonstrated abnormal sensorimotor integration in patients with Alzheimer's disease (AD) with an abnormally enhanced behavioral response to pitch perturbation. Here we examine the neural correlates of the abnormal pitch perturbation response in AD patients, using magnetoencephalographic imaging. The participants phonated the vowel /α/ while a real-time signal processor briefly perturbed the pitch (100 cents, 400 ms) of their auditory feedback. We examined the high-gamma band (65-150 Hz) responses during this task. AD patients showed significantly reduced left prefrontal activity during the early phase of perturbation and increased right middle temporal activity during the later phase of perturbation, compared to controls. Activity in these brain regions significantly correlated with the behavioral response. These results demonstrate that impaired prefrontal modulation of speech-motor-control network and additional recruitment of right temporal regions are significant mediators of aberrant sensorimotor integration in patients with AD. The abnormal neural integration mechanisms signify the contribution of cortical network dysfunction to cognitive and behavioral deficits in AD.

Vocal motor control and central auditory impairments in unilateral vocal fold paralysis.

OBJECTIVES: To evaluate differences in vocal motor control and central auditory processing between treated unilateral vocal fold paralysis (UVFP) and healthy control cohorts. STUDY DESIGN: Cross-sectional. METHODS: Ten UVFP study patients treated by type I thyroplasty with stable voices were compared to 12 control subjects for vocal motor control using a pitch perturbation response task and central auditory processing performance using a battery of complex sound intelligibility assays that included adverse temporal and noise conditions. Standard clinical evaluations of voice production and peripheral audiometric sensitivity were performed. RESULTS: Vocal motor control was impaired in treated UVFP. The UVFP cohort exhibited a 32.5% reduction in the instantaneous, subconscious compensatory response to pitch feedback perturbation in the interval between 150 ms and 550 ms following onset (P < 0.0001, linear mixed effects model). This impairment cannot simply be ascribed to vocal motor capacity insufficiency in the UVFP cohort because both cohorts demonstrated comparable functional capacity to perform the vocal motor task. The UVFP cohort also showed greater propensity for central auditory processing impairment (P < 0.05), notably for temporal compression and added noise challenges. CONCLUSION: Combined central vocal motor control and auditory processing impairments in treated UVFP highlight reciprocal interdependency of sensory and motor systems. This pilot study suggests that peripheral motor impairment of the larynx can degrade central auditory processing, which in turn may contribute to vocal motor control impairment. A more complete restoration communicative function in UVFP will require deeper understanding of sensory, motor, and sensorimotor aspects of the human communication loop. LEVEL OF EVIDENCE: 3b Laryngoscope, 129:2112-2117, 2019.

Cortical networks for speech motor control in unilateral vocal fold paralysis.

OBJECTIVE: To evaluate brain networks for motor control of voice production in patients with treated unilateral vocal fold paralysis (UVFP). STUDY DESIGN: Cross-sectional comparison. METHODS: Nine UVFP patients treated by type I thyroplasty, and 11 control subjects were compared using magnetoencephalographic imaging to measure beta band (12-30 Hz) neural oscillations during voice production with perturbation of pitch feedback. Differences in beta band power relative to baseline were analyzed to identify cortical areas with abnormal activity within the 400 ms perturbation period and 125 ms beyond, for a total of 525 ms. RESULTS: Whole-brain task-induced beta band activation patterns were qualitatively similar in both treated UVFP patients and healthy controls. Central vocal motor control plasticity in UVFP was expressed within constitutive components of central human communication networks identified in healthy controls. Treated UVFP patients exhibited statistically significant enhancement (P < 0.05) in beta band activity following pitch perturbation onset in left auditory cortex to 525 ms, left premotor cortex to 225 ms, and left and right frontal cortex to 525 ms. CONCLUSION: This study further corroborates that a peripheral motor impairment of the larynx can affect central cortical networks engaged in auditory feedback processing, vocal motor control, and judgment of voice-as-self. Future research to dissect functional relationships among constitutive cortical networks could reveal neurophysiological bases of central contributions to voice production impairment in UVFP. Those novel insights would motivate innovative treatments to improve voice production and reduce misalignment of voice-quality judgment between clinicians and patients. LEVEL OF EVIDENCE: 3b Laryngoscope, 129:2125-2130, 2019.

Reality Monitoring and Feedback Control of Speech Production Are Related Through Self-Agency.

Self-agency is the experience of being the agent of one's own thoughts and motor actions. The intact experience of self-agency is necessary for successful interactions with the outside world (i.e., reality monitoring) and for responding to sensory feedback of our motor actions (e.g., speech feedback control). Reality monitoring is the ability to distinguish internally self-generated information from outside reality (externally-derived information). In the present study, we examined the relationship of self-agency between lower-level speech feedback monitoring (i.e., monitoring what we hear ourselves say) and a higher-level cognitive reality monitoring task. In particular, we examined whether speech feedback monitoring and reality monitoring were driven by the capacity to experience self-agency-the ability to make reliable predictions about the outcomes of self-generated actions. During the reality monitoring task, subjects made judgments as to whether information was previously self-generated (self-agency judgments) or externally derived (external-agency judgments). During speech feedback monitoring, we assessed self-agency by altering environmental auditory feedback so that subjects listened to a perturbed version of their own speech. When subjects heard minimal perturbations in their auditory feedback while speaking, they made corrective responses, indicating that they judged the perturbations as errors in their speech output. We found that self-agency judgments in the reality-monitoring task were higher in people who had smaller corrective responses (p = 0.05) and smaller inter-trial variability (p = 0.03) during minimal pitch perturbations of their auditory feedback. These results provide support for a unitary process for the experience of self-agency governing low-level speech control and higher level reality monitoring.

Abnormal Speech Motor Control in Individuals with 16p11.2 Deletions.

Speech and motor deficits are highly prevalent (>70%) in individuals with the 600 kb BP4-BP5 16p11.2 deletion; however, the mechanisms that drive these deficits are unclear, limiting our ability to target interventions and advance treatment. This study examined fundamental aspects of speech motor control in participants with the 16p11.2 deletion. To assess capacity for control of voice, we examined how accurately and quickly subjects changed the pitch of their voice within a trial to correct for a transient perturbation of the pitch of their auditory feedback. When compared to controls, 16p11.2 deletion carriers show an over-exaggerated pitch compensation response to unpredictable mid-vocalization pitch perturbations. We also examined sensorimotor adaptation of speech by assessing how subjects learned to adapt their sustained productions of formants (speech spectral peak frequencies important for vowel identity), in response to consistent changes in their auditory feedback during vowel production. Deletion carriers show reduced sensorimotor adaptation to sustained vowel identity changes in auditory feedback. These results together suggest that 16p11.2 deletion carriers have fundamental impairments in the basic mechanisms of speech motor control and these impairments may partially explain the deficits in speech and language in these individuals.

Abnormal vocal behavior predicts executive and memory deficits in Alzheimer's disease.

Speakers respond automatically and rapidly to compensate for brief perturbations of pitch in their auditory feedback. The specific adjustments in vocal output require integration of brain regions involved in speech-motor-control in order to detect the sensory-feedback error and implement the motor correction. Cortical regions involved in the pitch reflex phenomenon are highly vulnerable targets of network disruption in Alzheimer's disease (AD). We examined the pitch reflex in AD patients (n = 19) compared to an age-matched control group (n = 16). We measured the degree of behavioral compensation (peak compensation) and the extent of the adaptive response (pitch-response persistence). Healthy-controls reached a peak compensation of 18.7 ± 0.8 cents, and demonstrated a sustained compensation at 8.9 ± 0.69 cents. AD patients, in contrast, demonstrated a significantly elevated peak compensation (22.4 ± 1.2 cents, p < 0.05), and a reduced sustained response (pitch-response persistence, 4.5 ± 0.88 cents, p < 0.001). The degree of increased peak compensation predicted executive dysfunction, while the degree of impaired pitch-response persistence predicted memory dysfunction, in AD patients. The current study demonstrates pitch reflex as a sensitive behavioral index of impaired prefrontal modulation of sensorimotor integration, and compromised plasticity mechanisms of memory, in AD.