A compressive sensing approach for inferring cognitive representations with reverse correlation.

Uncovering cognitive representations is an elusive goal that is increasingly pursued using the reverse correlation method, wherein human subjects make judgments about ambiguous stimuli. Employing reverse correlation often entails collecting thousands of stimulus-response pairs, which severely limits the breadth of studies that are feasible using the method. Current techniques to improve efficiency bias the outcome. Here we show that this methodological barrier can be diminished using compressive sensing, an advanced signal processing technique designed to improve sampling efficiency. Simulations are performed to demonstrate that compressive sensing can improve the accuracy of reconstructed cognitive representations and dramatically reduce the required number of stimulus-response pairs. Additionally, compressive sensing is used on human subject data from a previous reverse correlation study, demonstrating a dramatic improvement in reconstruction quality. This work concludes by outlining the potential of compressive sensing to improve representation reconstruction throughout the fields of psychology, neuroscience, and beyond.

Reverse Correlation Uncovers More Complete Tinnitus Spectra.

Goal: This study validates an approach to characterizing the sounds experienced by tinnitus patients via reverse correlation, with potential for characterizing a wider range of sounds than currently possible. Methods: Ten normal-hearing subjects assessed the subjective similarity of random auditory stimuli and target tinnitus-like sounds ("buzzing" and "roaring"). Reconstructions of the targets were obtained by regressing subject responses on the stimuli, and were compared for accuracy to the frequency spectra of the targets using Pearson's [Formula: see text]. Results: Reconstruction accuracy was significantly higher than chance across subjects: buzzing: [Formula: see text] (mean [Formula: see text] s.d.), [Formula: see text], [Formula: see text]; roaring: [Formula: see text], [Formula: see text], [Formula: see text]; combined: [Formula: see text], [Formula: see text], [Formula: see text]. Conclusion: Reverse correlation can accurately reconstruct non-tonal tinnitus-like sounds in normal-hearing subjects, indicating its potential for characterizing the sounds experienced by patients with non-tonal tinnitus.

Stimulus whitening improves the efficiency of reverse correlation.

Human perception depends upon internal representations of the environment that help to organize the raw information available from the senses by acting as reference patterns. Internal representations are widely characterized using reverse correlation, a method capable of producing unconstrained estimates of the representation itself, all on the basis of simple responses to random stimuli. Despite its advantages, reverse correlation is often infeasible to apply because of its inefficiency-a very large number of stimulus-response trials are required in order to obtain an accurate estimate. Here, we show that an important source of this inefficiency is small, yet nontrivial, correlations that occur by chance between randomly generated stimuli. We demonstrate in simulation that whitening stimuli to remove such correlations before eliciting responses provides greater than 85% improvement in efficiency for a given estimation quality, as well as a two- to fivefold increase in quality for a given sample size. Moreover, unlike conventional approaches, whitening improves the efficiency of reverse correlation without introducing bias into the estimate, or requiring prior knowledge of the target internal representation. Improving the efficiency of reverse correlation with whitening may enable a broader scope of investigations into the individual variability and potential universality of perceptual mechanisms.

Compliance With In-Home Self-Managed Rehabilitation Post-Stroke is Largely Independent of Scheduling Approach.

OBJECTIVE: To investigate how participants self-schedule their engagement with domestic rehabilitation gaming platform, and how their scheduling behavior in turn influence overall compliance. DESIGN: Cohort of individuals randomized to receive in-home rehabilitation gaming during a multi-site randomized controlled trial. SETTING: In-home self-managed rehabilitation. PARTICIPANTS: Eighty community-dwelling participants who were >6 months post-stroke and had mild to moderate upper extremity impairment (N=80). INTERVENTIONS: Participants were prescribed 15 hours of independent in-home self-scheduled game play for upper extremity mobility over 3 weeks. MAIN OUTCOME MEASURES: Total number of hours of active game play was objectively measured by the rehabilitation gaming system. Cluster analysis identified scheduling patterns from the following scheduling characteristics: total number of sessions, average session length, and consistency of play schedule. RESULTS: Four distinct scheduling profiles were revealed, 3 of which were associated with complete or near-complete compliance, while a fourth (inconsistent schedule of short, infrequent sessions) was associated with very poor compliance. Poor compliance could be predicted within the first 7 days of the program with 78% accuracy based on the same play pattern metrics used to identify player profiles. CONCLUSIONS: Our findings support client autonomy in selecting the home practice schedule that works best for them, as compliance can successfully be achieved through a variety of different scheduling patterns. The objective measurements of compliance provided through rehabilitation gaming can assist therapists to identify individuals early on who exhibit scheduling behavior that is predictive of poor compliance.

Dynamic Stability of Passive Dynamic Walking Following Unexpected Perturbations.

Mitigating the risk of falling is an area of significant interest among clinicians due to the often profound health-related consequences of falls. Consequently, there is acute interest in characterizing the biomechanical conditions associated with increased fall risk, and in methods for quantifying gait stability under those conditions toward predicting and ultimately preventing falls. Considerable insights into the biomechanics of fall risk have been provided by examining the passive dynamic walking (PDW) model under nominal and perturbed conditions. This work aims to expand upon prior efforts and develop the PDW model as a model of tripping and slipping by simulating and analyzing the behavior of the model during transient perturbations. We show that fall risk increases with increasing perturbation magnitude, yet stable walking may be found even with fairly large perturbations. In cases where transient perturbations result in a fall, a nontrivial portion exhibit a substantial period of stumbling before the fall, indicating an opportunity for developing early fall-risk detection and intervention techniques. In such cases, we show that widely used kinematic metrics are able to predict whether or not a fall will occur with up to 82% balanced accuracy, even when a variety of gait kinematics are considered.

The efficacy of acoustic-based articulatory phenotyping for characterizing and classifying four divergent neurodegenerative diseases using sequential motion rates.

Despite the impacts of neurodegeneration on speech function, little is known about how to comprehensively characterize the resulting speech abnormalities using a set of objective measures. Quantitative phenotyping of speech motor impairments may have important implications for identifying clinical syndromes and their underlying etiologies, monitoring disease progression over time, and improving treatment efficacy. The goal of this research was to investigate the validity and classification accuracy of comprehensive acoustic-based articulatory phenotypes in speakers with distinct neurodegenerative diseases. Articulatory phenotypes were characterized based on acoustic features that were selected to represent five components of motor performance: Coordination, Consistency, Speed, Precision, and Rate. The phenotypes were first used to characterize the articulatory abnormalities across four progressive neurologic diseases known to have divergent speech motor deficits: amyotrophic lateral sclerosis (ALS), progressive ataxia (PA), Parkinson's disease (PD), and the nonfluent variant of primary progressive aphasia and progressive apraxia of speech (nfPPA + PAOS). We then examined the efficacy of articulatory phenotyping for disease classification. Acoustic analyses were conducted on audio recordings of 217 participants (i.e., 46 ALS, 52 PA, 60 PD, 20 nfPPA + PAOS, and 39 controls) during a sequential speech task. Results revealed evidence of distinct articulatory phenotypes for the four clinical groups and that the phenotypes demonstrated strong classification accuracy for all groups except ALS. Our results highlight the phenotypic variability present across neurodegenerative diseases, which, in turn, may inform (1) the differential diagnosis of neurological diseases and (2) the development of sensitive outcome measures for monitoring disease progression or assessing treatment efficacy.

Continuous TBI Monitoring From Spontaneous Speech Using Parametrized Sinc Filters and a Cascading GRU.

Traumatic Brain Injury (TBI) is caused by a head injury that affects the brain, impairing cognitive and communication function and resulting in speech and language disorders. Over 80,000 individuals in the US suffer from long-term TBI disabilities and continuous monitoring after TBI is essential to facilitate rehabilitation and prevent regression. Prior work has demonstrated the feasibility of TBI monitoring from speech by leveraging advancements in Artificial Intelligence (AI) and speech processing technology. However, much of prior work explored TBI detection using scripted speech tasks such as diadochokinesis tests or reading a passage. Such scripted approaches require active user involvement that significantly burdens participants. Moreover, they are episodic, are not realistic, and do not provide a longitudinal picture of the user's TBI condition. This study proposes a continuous TBI monitoring from changes in acoustic features of spontaneous speech collected passively using the smartphone. Low-level acoustic features are extracted using parametrized Sinc filters (pSinc) that are then classified TBI (yes/no) using a cascading Gated Recurrent Unit (cGRU). The cGRU model utilizes a cell gate unit in the GRU to store and incorporate each individual's prediction history as prior knowledge into the model. In rigorous evaluation, our proposed method outperformed prior TBI classification methods on conversational speech recorded during patient-therapist discourses following TBI, achieving 83.87% balanced accuracy. Furthermore, unique words that are important in TBI prediction were identified using SHapley Additive exPlanations (SHAP). A correlation was also found between features acquired by the proposed method and coordination deficits following TBI.

Validation of an Acoustic-Based Framework of Speech Motor Control: Assessing Criterion and Construct Validity Using Kinematic and Perceptual Measures.

PURPOSE: This study investigated the criterion (analytical and clinical) and construct (divergent) validity of a novel, acoustic-based framework composed of five key components of motor control: Coordination, Consistency, Speed, Precision, and Rate. METHOD: Acoustic and kinematic analyses were performed on audio recordings from 22 subjects with amyotrophic lateral sclerosis during a sequential motion rate task. Perceptual analyses were completed by two licensed speech-language pathologists, who rated each subject's speech on the five framework components and their overall severity. Analytical and clinical validity were assessed by comparing performance on the acoustic features to their kinematic correlates and to clinician ratings of the five components, respectively. Divergent validity of the acoustic-based framework was then assessed by comparing performance on each pair of acoustic features to determine whether the features represent distinct articulatory constructs. Bivariate correlations and partial correlations with severity as a covariate were conducted for each comparison. RESULTS: Results revealed moderate-to-strong analytical validity for every acoustic feature, both with and without controlling for severity, and moderate-to-strong clinical validity for all acoustic features except Coordination, without controlling for severity. When severity was included as a covariate, the strong associations for Speed and Precision became weak. Divergent validity was supported by weak-to-moderate pairwise associations between all acoustic features except Speed (second-formant [F2] slope of consonant transition) and Precision (between-consonant variability in F2 slope). CONCLUSIONS: This study demonstrated that the acoustic-based framework has potential as an objective, valid, and clinically useful tool for profiling articulatory deficits in individuals with speech motor disorders. The findings also suggest that compared to clinician ratings, instrumental measures are more sensitive to subtle differences in articulatory function. With further research, this framework could provide more accurate and reliable characterizations of articulatory impairment, which may eventually increase clinical confidence in the diagnosis and treatment of patients with different articulatory phenotypes.

Dark tone quality and vocal tract shaping in soprano song production: Insights from real-time MRI.

Tone quality termed "dark" is an aesthetically important property of Western classical voice performance and has been associated with lowered formant frequencies, lowered larynx, and widened pharynx. The present study uses real-time magnetic resonance imaging with synchronous audio recordings to investigate dark tone quality in four professionally trained sopranos with enhanced ecological validity and a relatively complete view of the vocal tract. Findings differ from traditional accounts, indicating that labial narrowing may be the primary driver of dark tone quality across performers, while many other aspects of vocal tract shaping are shown to differ significantly in a performer-specific way.

Sensorimotor conflict tests in an immersive virtual environment reveal subclinical impairments in mild traumatic brain injury.

Current clinical tests lack the sensitivity needed for detecting subtle balance impairments associated with mild traumatic brain injury (mTBI). Patient-reported symptoms can be significant and have a huge impact on daily life, but impairments may remain undetected or poorly quantified using clinical measures. Our central hypothesis was that provocative sensorimotor perturbations, delivered in a highly instrumented, immersive virtual environment, would challenge sensory subsystems recruited for balance through conflicting multi-sensory evidence, and therefore reveal that not all subsystems are performing optimally. The results show that, as compared to standard clinical tests, the provocative perturbations illuminate balance impairments in subjects who have had mild traumatic brain injuries. Perturbations delivered while subjects were walking provided greater discriminability (average accuracy ≈ 0.90) than those delivered during standing (average accuracy ≈ 0.65) between mTBI subjects and healthy controls. Of the categories of features extracted to characterize balance, the lower limb accelerometry-based metrics proved to be most informative. Further, in response to perturbations, subjects with an mTBI utilized hip strategies more than ankle strategies to prevent loss of balance and also showed less variability in gait patterns. We have shown that sensorimotor conflicts illuminate otherwise-hidden balance impairments, which can be used to increase the sensitivity of current clinical procedures. This augmentation is vital in order to robustly detect the presence of balance impairments after mTBI and potentially define a phenotype of balance dysfunction that enhances risk of injury.

Predicting changes in performance due to cognitive fatigue: A multimodal approach based on speech motor coordination and electrodermal activity.

OBJECTIVE: Military job and training activities place significant demands on service members' (SMs') cognitive resources, increasing risk of injury and degrading performance. Early detection of cognitive fatigue is essential to reduce risk and support optimal function. This paper describes a multimodal approach, based on changes in measures of speech motor coordination and electrodermal activity (EDA), for predicting changes in performance following sustained cognitive effort. METHODS: Twenty-nine active duty SMs completed computer-based cognitive tasks for 2 h (load period). Measures of speech derived from audio were acquired, along with concurrent measures of EDA, before and after the load period. Cognitive performance was assessed before and during the load period using the Automated Neuropsychological Assessment Metrics Military Battery (ANAM MIL). Subjective assessments of cognitive effort and alertness were obtained intermittently. RESULTS: Across the load period, participants' ratings of cognitive workload increased, while alertness ratings declined. Cognitive performance declined significantly during the first half of the load period. Three speech and arousal features predicted cognitive performance changes during this period with statistically significant accuracy: EDA (r = 0.43, p = 0.01), articulator velocity coordination (r = 0.50, p = 0.00), and vocal creak (r = 0.35, p = 0.03). Fusing predictions from these features predicted performance changes with r = 0.68 (p = 0.00). CONCLUSIONS: Results suggest that speech and arousal measures may be used to predict changes in performance associated with cognitive fatigue. This work supports ongoing efforts to develop reliable, unobtrusive measures for cognitive state assessment aimed at reducing injury risk, informing return to work decisions, and supporting diverse mobile healthcare applications in civilian and military settings.

Analysis of Phonetic Balance in Standard English Passages.

Purpose A common way of eliciting speech from individuals is by using passages of written language that are intended to be read aloud. Read passages afford the opportunity for increased control over the phonetic properties of elicited speech, of which phonetic balance is an often-noted example. No comprehensive analysis of the phonetic balance of read passages has been reported in the literature. The present article provides a quantitative comparison of the phonetic balance of widely used passages in English. Method Assessment of phonetic balance is carried out by comparing the distribution of phonemes in several passages to distributions consistent with typical spoken English. Data regarding the distribution of phonemes in spoken American English are aggregated from the published literature and large speech corpora. Phoneme distributions are compared using Spearman rank order correlation coefficient to quantify similarities of phoneme counts in those sources. Results Correlations between phoneme distributions in read passages and aggregated material representative of spoken American English ranged from .70 to .89. Correlations between phoneme counts from all passages, literature sources, and corpus sources ranged from .55 to .99. All correlations were statistically significant at the Bonferroni-adjusted level. Conclusions Passages considered in the present work provide high, but not ideal, phonetic balance. Space exists for the creation of new passages that more closely match the phoneme distributions observed in spoken American English. The Caterpillar provided the best phonetic balance, but phoneme distributions in all considered materials were highly similar to each other.

Bridging Dynamical Systems and Optimal Trajectory Approaches to Speech Motor Control With Dynamic Movement Primitives.

Current models of speech motor control rely on either trajectory-based control (DIVA, GEPPETO, ACT) or a dynamical systems approach based on feedback control (Task Dynamics, FACTS). While both approaches have provided insights into the speech motor system, it is difficult to connect these findings across models given the distinct theoretical and computational bases of the two approaches. We propose a new extension of the most widely used dynamical systems approach, Task Dynamics, that incorporates many of the strengths of trajectory-based approaches, providing a way to bridge the theoretical divide between what have been two separate approaches to understanding speech motor control. The Task Dynamics (TD) model posits that speech gestures are governed by point attractor dynamics consistent with a critically damped harmonic oscillator. Kinematic trajectories associated with such gestures should therefore be consistent with a second-order dynamical system, possibly modified by blending with temporally overlapping gestures or altering oscillator parameters. This account of observed kinematics is powerful and theoretically appealing, but may be insufficient to account for deviations from predicted kinematics-i.e., changes produced in response to some external perturbations to the jaw, changes in control during acquisition and development, or effects of word/syllable frequency. Optimization, such as would be needed to minimize articulatory effort, is also incompatible with the current TD model, though the idea that the speech production systems economizes effort has a long history and, importantly, also plays a critical role in current theories of domain-general human motor control. To address these issues, we use Dynamic Movement Primitives (DMPs) to expand a dynamical systems framework for speech motor control to allow modification of kinematic trajectories by incorporating a simple, learnable forcing term into existing point attractor dynamics. We show that integration of DMPs with task-based point-attractor dynamics enhances the potential explanatory power of TD in a number of critical ways, including the ability to account for external forces in planning and optimizing both kinematic and dynamic movement costs. At the same time, this approach preserves the successes of Task Dynamics in handling multi-gesture planning and coordination.

Current models of speech motor control: A control-theoretic overview of architectures and properties.

This paper reviews the current state of several formal models of speech motor control, with particular focus on the low-level control of the speech articulators. Further development of speech motor control models may be aided by a comparison of model attributes. The review builds an understanding of existing models from first principles, before moving into a discussion of several models, showing how each is constructed out of the same basic domain-general ideas and components-e.g., generalized feedforward, feedback, and model predictive components. This approach allows for direct comparisons to be made in terms of where the models differ, and their points of agreement. Substantial differences among models can be observed in their use of feedforward control, process of estimating system state, and method of incorporating feedback signals into control. However, many commonalities exist among the models in terms of their reliance on higher-level motor planning, use of feedback signals, lack of time-variant adaptation, and focus on kinematic aspects of control and biomechanics. Ongoing research bridging hybrid feedforward/feedback pathways with forward dynamic control, as well as feedback/internal model-based state estimation, is discussed.

Speed-accuracy tradeoffs in human speech production.

Speech motor actions are performed quickly, while simultaneously maintaining a high degree of accuracy. Are speed and accuracy in conflict during speech production? Speed-accuracy tradeoffs have been shown in many domains of human motor action, but have not been directly examined in the domain of speech production. The present work seeks evidence for Fitts' law, a rigorous formulation of this fundamental tradeoff, in speech articulation kinematics by analyzing USC-TIMIT, a real-time magnetic resonance imaging data set of speech production. A theoretical framework for considering Fitts' law with respect to models of speech motor control is elucidated. Methodological challenges in seeking relationships consistent with Fitts' law are addressed, including the operational definitions and measurement of key variables in real-time MRI data. Results suggest the presence of speed-accuracy tradeoffs for certain types of speech production actions, with wide variability across syllable position, and substantial variability also across subjects. Coda consonant targets immediately following the syllabic nucleus show the strongest evidence of this tradeoff, with correlations as high as 0.72 between speed and accuracy. A discussion is provided concerning the potentially limited applicability of Fitts' law in the context of speech production, as well as the theoretical context for interpreting the results.

On Short-Time Estimation of Vocal Tract Length from Formant Frequencies.

Vocal tract length is highly variable across speakers and determines many aspects of the acoustic speech signal, making it an essential parameter to consider for explaining behavioral variability. A method for accurate estimation of vocal tract length from formant frequencies would afford normalization of interspeaker variability and facilitate acoustic comparisons across speakers. A framework for considering estimation methods is developed from the basic principles of vocal tract acoustics, and an estimation method is proposed that follows naturally from this framework. The proposed method is evaluated using acoustic characteristics of simulated vocal tracts ranging from 14 to 19 cm in length, as well as real-time magnetic resonance imaging data with synchronous audio from five speakers whose vocal tracts range from 14.5 to 18.0 cm in length. Evaluations show improvements in accuracy over previously proposed methods, with 0.631 and 1.277 cm root mean square error on simulated and human speech data, respectively. Empirical results show that the effectiveness of the proposed method is based on emphasizing higher formant frequencies, which seem less affected by speech articulation. Theoretical predictions of formant sensitivity reinforce this empirical finding. Moreover, theoretical insights are explained regarding the reason for differences in formant sensitivity.

Co-registration of speech production datasets from electromagnetic articulography and real-time magnetic resonance imaging.

This paper describes a spatio-temporal registration approach for speech articulation data obtained from electromagnetic articulography (EMA) and real-time Magnetic Resonance Imaging (rtMRI). This is motivated by the potential for combining the complementary advantages of both types of data. The registration method is validated on EMA and rtMRI datasets obtained at different times, but using the same stimuli. The aligned corpus offers the advantages of high temporal resolution (from EMA) and a complete mid-sagittal view (from rtMRI). The co-registration also yields optimum placement of EMA sensors as articulatory landmarks on the magnetic resonance images, thus providing richer spatio-temporal information about articulatory dynamics.