Cross-Modal Sensory Boosting to Improve High-Frequency Hearing Loss: Device Development and Validation.

Background: High-frequency hearing loss is one of the most common problems in the aging population and with those who have a history of exposure to loud noises. This type of hearing loss can be frustrating and disabling, making it difficult to understand speech communication and interact effectively with the world. Objective: This study aimed to examine the impact of spatially unique haptic vibrations representing high-frequency phonemes on the self-perceived ability to understand conversations in everyday situations. Methods: To address high-frequency hearing loss, a multi-motor wristband was developed that uses machine learning to listen for specific high-frequency phonemes. The wristband vibrates in spatially unique locations to represent which phoneme was present in real time. A total of 16 participants with high-frequency hearing loss were recruited and asked to wear the wristband for 6 weeks. The degree of disability associated with hearing loss was measured weekly using the Abbreviated Profile of Hearing Aid Benefit (APHAB). Results: By the end of the 6-week study, the average APHAB benefit score across all participants reached 12.39 points, from a baseline of 40.32 to a final score of 27.93 (SD 13.11; N=16; P=.002, 2-tailed dependent t test). Those without hearing aids showed a 10.78-point larger improvement in average APHAB benefit score at 6 weeks than those with hearing aids (t14=2.14; P=.10, 2-tailed independent t test). The average benefit score across all participants for ease of communication was 15.44 (SD 13.88; N=16; P<.001, 2-tailed dependent t test). The average benefit score across all participants for background noise was 10.88 (SD 17.54; N=16; P=.03, 2-tailed dependent t test). The average benefit score across all participants for reverberation was 10.84 (SD 16.95; N=16; P=.02, 2-tailed dependent t test). Conclusions: These findings show that vibrotactile sensory substitution delivered by a wristband that produces spatially distinguishable vibrations in correspondence with high-frequency phonemes helps individuals with high-frequency hearing loss improve their perceived understanding of verbal communication. Vibrotactile feedback provides benefits whether or not a person wears hearing aids, albeit in slightly different ways. Finally, individuals with the greatest perceived difficulty understanding speech experienced the greatest amount of perceived benefit from vibrotactile feedback.

Bimodal Stimulation for the Reduction of Tinnitus Using Vibration on the Skin.

Tinnitus (ringing in the ears) affects 1 in 10 adults in the United States, often with damaging psychological consequences. Currently, there exists no cure for most forms of tinnitus. Recently, bimodal stimulation - the pairing of sounds with haptic stimulation - has shown efficacy in reducing the symptoms of tinnitus. Previous bimodal stimulation approaches have used electrical shocks on the tongue, a technique that requires daily in-person sessions at an audiologist's office. We here show that excellent results can be achieved wearing a wristband with multiple vibratory motors. Tones are played and the wristband correspondingly vibrates the wrist of the user at different spatial locations depending on the frequency of the tone. We compared the experimental group with a control group who listened to the tones but did not wear the wristband. The tone frequencies were centered on each user's tinnitus frequency and the tones were randomized both in frequency and duration. 45 participants with Tinnitus Functional Index (TFI) scores of 25 and above were tested. Results show a significantly greater reduction in TFI scores for the experimental group compared to the control. Importantly, with higher baseline severity we find larger differences between the experimental and control groups, revealing greater symptom improvement for those with severe tinnitus. The therapeutic approach of combining sounds with spatiallyand temporally-correlated vibrations on the wrist is found to be a simple, time-efficient, and effective procedure to lessen the symptoms of tinnitus.

A call for comparing theories of consciousness and data sharing.

Merker, Williford, and Rudrauf make several arguments against the integrated information theory of consciousness; whereas some have merit, their conclusion that the theory should be discarded is premature. Coming years promise advances in the empirical study of consciousness, and only after theories are independently tested with shared data can they be ruled in or out. We propose future research directions.

Larger visual changes compress time: The inverted effect of asemantic visual features on interval time perception.

Time perception is fluid and affected by manipulations to visual inputs. Previous literature shows that changes to low-level visual properties alter time judgments at the millisecond-level. At longer intervals, in the span of seconds and minutes, high-level cognitive effects (e.g., emotions, memories) elicited by visual inputs affect time perception, but these effects are confounded with semantic information in these inputs, and are therefore challenging to measure and control. In this work, we investigate the effect of asemantic visual properties (pure visual features devoid of emotional or semantic value) on interval time perception. Our experiments were conducted with binary and production tasks in both conventional and head-mounted displays, testing the effects of four different visual features (spatial luminance contrast, temporal frequency, field of view, and visual complexity). Our results reveal a consistent pattern: larger visual changes all shorten perceived time in intervals of up to 3min, remarkably contrary to their effect on millisecond-level perception. Our findings may help alter participants' time perception, which can have broad real-world implications.

The future of sensory substitution, addition, and expansion via haptic devices.

Haptic devices use the sense of touch to transmit information to the nervous system. As an example, a sound-to-touch device processes auditory information and sends it to the brain via patterns of vibration on the skin for people who have lost hearing. We here summarize the current directions of such research and draw upon examples in industry and academia. Such devices can be used for sensory substitution (replacing a lost sense, such as hearing or vision), sensory expansion (widening an existing sensory experience, such as detecting electromagnetic radiation outside the visible light spectrum), and sensory addition (providing a novel sense, such as magnetoreception). We review the relevant literature, the current status, and possible directions for the future of sensory manipulation using non-invasive haptic devices.

The Defensive Activation Theory: REM Sleep as a Mechanism to Prevent Takeover of the Visual Cortex.

Regions of the brain maintain their territory with continuous activity: if activity slows or stops (e.g., because of blindness), the territory tends to be taken over by its neighbors. A surprise in recent years has been the speed of takeover, which is measurable within an hour. These findings lead us to a new hypothesis on the origin of REM sleep. We hypothesize that the circuitry underlying REM sleep serves to amplify the visual system's activity periodically throughout the night, allowing it to defend its territory against takeover from other senses. We find that measures of plasticity across 25 species of primates correlate positively with the proportion of rapid eye movement (REM) sleep. We further find that plasticity and REM sleep increase in lockstep with evolutionary recency to humans. Finally, our hypothesis is consistent with the decrease in REM sleep and parallel decrease in neuroplasticity with aging.

Deciphering Sounds Through Patterns of Vibration on the Skin.

Sensory substitution refers to the concept of feeding information to the brain via an atypical sensory pathway. We here examined the degree to which participants (deaf and hard of hearing) can learn to identify sounds that are algorithmically translated into spatiotemporal patterns of vibration on the skin of the wrist. In a three-alternative forced choice task, participants could determine the identity of up to 95% and on average 70% of the stimuli simply by the spatial pattern of vibrations on the skin. Performance improved significantly over the course of 1 month. Younger participants tended to score better, possibly because of higher brain plasticity, more sensitive skin, or better skills at playing digital games. Similar results were obtained with pattern discrimination, in which a pattern representing the sound of one word was presented to the skin, followed by that of a second word. Participants answered whether the word was the same or different. With minimal difference pairs (distinguished by only one phoneme, such as "house" and "mouse"), the best performance was 83% (average of 62%), while with non-minimal pairs (such as "house" and "zip") the best performance was 100% (average of 70%). Collectively, these results demonstrate that participants are capable of using the channel of the skin to interpret auditory stimuli, opening the way for low-cost, wearable sensory substitution for the deaf and hard of hearing communities.

A Rapid, Mobile Neurocognitive Screening Test to Aid in Identifying Cognitive Impairment and Dementia (BrainCheck): Cohort Study.

BACKGROUND: The US population over the age of 65 is expected to double by the year 2050. Concordantly, the incidence of dementia is projected to increase. The subclinical stage of dementia begins years before signs and symptoms appear. Early detection of cognitive impairment and/or cognitive decline may allow for interventions to slow its progression. Furthermore, early detection may allow for implementation of care plans that may affect the quality of life of those affected and their caregivers. OBJECTIVE: We sought to determine the accuracy and validity of BrainCheck Memory as a diagnostic aid for age-related cognitive impairment, as compared against physician diagnosis and other commonly used neurocognitive screening tests, including the Saint Louis University Mental Status (SLUMS) exam, the Mini-Mental State Examination (MMSE), and the Montreal Cognitive Assessment (MoCA). METHODS: We tested 583 volunteers over the age of 49 from various community centers and living facilities in Houston, Texas. The volunteers were divided into five cohorts: a normative population and four comparison groups for the SLUMS exam, the MMSE, the MoCA, and physician diagnosis. Each comparison group completed their respective assessment and BrainCheck Memory. RESULTS: A total of 398 subjects were included in the normative population. A total of 84 participants were in the SLUMS exam cohort, 51 in the MMSE cohort, 35 in the MoCA cohort, and 18 in the physician cohort. BrainCheck Memory assessments were significantly correlated to the SLUMS exam, with coefficients ranging from .5 to .7. Correlation coefficients for the MMSE and BrainCheck and the MoCA and BrainCheck were also significant. Of the 18 subjects evaluated by a physician, 9 (50%) were healthy, 6 (33%) were moderately impaired, and 3 (17%) were severely impaired. A significant difference was found between the severely and moderately impaired subjects and the healthy subjects (P=.02). We derived a BrainCheck Memory composite score that showed stronger correlations with the standard assessments as compared to the individual BrainCheck assessments. Receiver operating characteristic (ROC) curve analysis of this composite score found a sensitivity of 81% and a specificity of 94%. CONCLUSIONS: BrainCheck Memory provides a sensitive and specific metric for age-related cognitive impairment in older adults, with the advantages of a mobile, digital, and easy-to-use test. TRIAL REGISTRATION: ClinicalTrials.gov NCT03608722; https://clinicaltrials.gov/ct2/show/NCT03608722 (Archived by WebCite at http://www.webcitation.org/76JLoYUGf).

Assessing Risk Among Correctional Community Probation Populations: Predicting Reoffense With Mobile Neurocognitive Assessment Software.

We seek to address current limitations of forensic risk assessments by introducing the first mobile, self-scoring, risk assessment software that relies on neurocognitive testing to predict reoffense. This assessment, run entirely on a tablet, measures decision-making via a suite of neurocognitive tests in less than 30 minutes. The software measures several cognitive and decision-making traits of the user, including impulsivity, empathy, aggression, and several other traits linked to reoffending. Our analysis measured whether this assessment successfully predicted recidivism by testing probationers in a large urban city (Houston, TX, United States) from 2017 to 2019. To determine predictive validity, we used machine learning to yield cross-validated receiver-operator characteristics. Results gave a recidivism prediction value of 0.70, making it comparable to commonly used risk assessments. This novel approach diverges from traditional self-reporting, interview-based, and criminal-records-based approaches, and can also add a protective layer against bias, while strengthening model accuracy in predicting reoffense. In addition, subjectivity is eliminated and time-consuming administrative efforts are reduced. With continued data collection, this approach opens the possibility of identifying different levels of recidivism risk, by crime type, for any age, or gender, and seeks to steer individuals appropriately toward rehabilitative programs. Suggestions for future research directions are provided.

Empathic Neural Responses Predict Group Allegiance.

Watching another person in pain activates brain areas involved in the sensation of our own pain. Importantly, this neural mirroring is not constant; rather, it is modulated by our beliefs about their intentions, circumstances, and group allegiances. We investigated if the neural empathic response is modulated by minimally-differentiating information (e.g., a simple text label indicating another's religious belief), and if neural activity changes predict ingroups and outgroups across independent paradigms. We found that the empathic response was larger when participants viewed a painful event occurring to a hand labeled with their own religion (ingroup) than to a hand labeled with a different religion (outgroup). Counterintuitively, the magnitude of this bias correlated positively with the magnitude of participants' self-reported empathy. A multivariate classifier, using mean activity in empathy-related brain regions as features, discriminated ingroup from outgroup with 72% accuracy; the classifier's confidence correlated with belief certainty. This classifier generalized successfully to validation experiments in which the ingroup condition was based on an arbitrary group assignment. Empathy networks thus allow for the classification of long-held, newly-modified and arbitrarily-formed ingroups and outgroups. This is the first report of a single machine learning model on neural activation that generalizes to multiple representations of ingroup and outgroup. The current findings may prove useful as an objective diagnostic tool to measure the magnitude of one's group affiliations, and the effectiveness of interventions to reduce ingroup biases.

Diagnostic accuracy of tablet-based software for the detection of concussion.

Despite the high prevalence of traumatic brain injuries (TBI), there are few rapid and straightforward tests to improve its assessment. To this end, we developed a tablet-based software battery ("BrainCheck") for concussion detection that is well suited to sports, emergency department, and clinical settings. This article is a study of the diagnostic accuracy of BrainCheck. We administered BrainCheck to 30 TBI patients and 30 pain-matched controls at a hospital Emergency Department (ED), and 538 healthy individuals at 10 control test sites. We compared the results of the tablet-based assessment against physician diagnoses derived from brain scans, clinical examination, and the SCAT3 test, a traditional measure of TBI. We found consistent distributions of normative data and high test-retest reliability. Based on these assessments, we defined a composite score that distinguishes TBI from non-TBI individuals with high sensitivity (83%) and specificity (87%). We conclude that our testing application provides a rapid, portable testing method for TBI.

The prevalence of synaesthesia depends on early language learning.

According to one theory, synaesthesia develops, or is preserved, because it helps children learn. If so, it should be more common among adults who faced greater childhood learning challenges. In the largest survey of synaesthesia to date, the incidence of synaesthesia was compared among native speakers of languages with transparent (easier) and opaque (more difficult) orthographies. Contrary to our prediction, native speakers of Czech (transparent) were more likely to be synaesthetes than native speakers of English (opaque). However, exploratory analyses suggested that this was because more Czechs learned non-native second languages, which was strongly associated with synaesthesia, consistent with the learning hypothesis. Furthermore, the incidence of synaesthesia among speakers of opaque languages was double that among speakers of transparent languages other than Czech, also consistent with the learning hypothesis. These findings contribute to an emerging understanding of synaesthetic development as a complex and lengthy process with multiple causal influences.

Real time functional MRI training to decrease motion in imaging studies: Lack of significant improvement.

Functional magnetic resonance imaging (fMRI) is widely used to study brain circuitry in healthy controls and in psychiatry. A major problem of fMRI studies is motion, which affects the quality of images, is a major source of noise, and can confound data if, for example, the experimental groups move differently. Despite continual reminders to experimental subjects about keeping still, however, movement in the scanner remains a problem. The authors hypothesized that showing head movement during a scanning session may help subjects learn how to keep their head still. The authors scanned subjects and displayed in real time a plot of head movement that had three regions. The authors found, in a limited sample, that the improvements were marginal and inconsistent. Thus, they concluded that this strategy, even if likely to work for some people, is probably not sufficiently successful to be implemented at this time.

Effects of Auditory Rhythm and Music on Gait Disturbances in Parkinson's Disease.

Gait abnormalities, such as shuffling steps, start hesitation, and freezing, are common and often incapacitating symptoms of Parkinson's disease (PD) and other parkinsonian disorders. Pharmacological and surgical approaches have only limited efficacy in treating these gait disorders. Rhythmic auditory stimulation (RAS), such as playing marching music and dance therapy, has been shown to be a safe, inexpensive, and an effective method in improving gait in PD patients. However, RAS that adapts to patients' movements may be more effective than rigid, fixed-tempo RAS used in most studies. In addition to auditory cueing, immersive virtual reality technologies that utilize interactive computer-generated systems through wearable devices are increasingly used for improving brain-body interaction and sensory-motor integration. Using multisensory cues, these therapies may be particularly suitable for the treatment of parkinsonian freezing and other gait disorders. In this review, we examine the affected neurological circuits underlying gait and temporal processing in PD patients and summarize the current studies demonstrating the effects of RAS on improving these gait deficits.

Duration estimates within a modality are integrated sub-optimally.

Perceived duration can be influenced by various properties of sensory stimuli. For example, visual stimuli of higher temporal frequency are perceived to last longer than those of lower temporal frequency. How does the brain form a representation of duration when each of two simultaneously presented stimuli influences perceived duration in different way? To answer this question, we investigated the perceived duration of a pair of dynamic visual stimuli of different temporal frequencies in comparison to that of a single visual stimulus of either low or high temporal frequency. We found that the duration representation of simultaneously occurring visual stimuli is best described by weighting the estimates of duration based on each individual stimulus. However, the weighting performance deviates from the prediction of statistically optimal integration. In addition, we provided a Bayesian account to explain a difference in the apparent sensitivity of the psychometric curves introduced by the order in which the two stimuli are displayed in a two-alternative forced-choice task.

Perceived duration is reduced by repetition but not by high-level expectation.

A repeated stimulus is judged as briefer than a novel one. It has been suggested that this duration illusion is an example of a more general phenomenon-namely that a more expected stimulus is judged as briefer than a less expected one. To test this hypothesis, we manipulated high-level expectation through the probability of a stimulus sequence, through the regularity of the preceding stimuli in a sequence, or through whether a stimulus violates an overlearned sequence. We found that perceived duration is not reduced by these types of expectation. Repetition of stimuli, on the other hand, consistently reduces perceived duration across our experiments. In addition, the effect of stimulus repetition is constrained to the location of the repeated stimulus. Our findings suggest that estimates of subsecond duration are largely the result of low-level sensory processing.

Using space and time to encode vibrotactile information: toward an estimate of the skin's achievable throughput.

Touch receptors in the skin can relay various forms of abstract information, such as words (Braille), haptic feedback (cell phones, game controllers, feedback for prosthetic control), and basic visual information such as edges and shape (sensory substitution devices). The skin can support such applications with ease: They are all low bandwidth and do not require a fine temporal acuity. But what of high-throughput applications? We use sound-to-touch conversion as a motivating example, though others abound (e.g., vision, stock market data). In the past, vibrotactile hearing aids have demonstrated improvement in speech perceptions in the deaf. However, a sound-to-touch sensory substitution device that works with high efficacy and without the aid of lipreading has yet to be developed. Is this because skin simply does not have the capacity to effectively relay high-throughput streams such as sound? Or is this because the spatial and temporal properties of skin have not been leveraged to full advantage? Here, we begin to address these questions with two experiments. First, we seek to determine the best method of relaying information through the skin using an identification task on the lower back. We find that vibrotactile patterns encoding information in both space and time yield the best overall information transfer estimate. Patterns encoded in space and time or "intensity" (the coupled coding of vibration frequency and force) both far exceed performance of only spatially encoded patterns. Next, we determine the vibrotactile two-tacton resolution on the lower back-the distance necessary for resolving two vibrotactile patterns. We find that our vibratory motors conservatively require at least 6 cm of separation to resolve two independent tactile patterns (>80 % correct), regardless of stimulus type (e.g., spatiotemporal "sweeps" versus single vibratory pulses). Six centimeter is a greater distance than the inter-motor distances used in Experiment 1 (2.5 cm), which explains the poor identification performance of spatially encoded patterns. Hence, when using an array of vibrational motors, spatiotemporal sweeps can overcome the limitations of vibrotactile two-tacton resolution. The results provide the first steps toward obtaining a realistic estimate of the skin's achievable throughput, illustrating the best ways to encode data to the skin (using as many dimensions as possible) and how far such interfaces would need to be separated if using multiple arrays in parallel.

Prevalence of learned grapheme-color pairings in a large online sample of synesthetes.

In this paper we estimate the minimum prevalence of grapheme-color synesthetes with letter-color matches learned from an external stimulus, by analyzing a large sample of English-speaking grapheme-color synesthetes. We find that at least 6% (400/6588 participants) of the total sample learned many of their matches from a widely available colored letter toy. Among those born in the decade after the toy began to be manufactured, the proportion of synesthetes with learned letter-color pairings approaches 15% for some 5-year periods. Among those born 5 years or more before it was manufactured, none have colors learned from the toy. Analysis of the letter-color matching data suggests the only difference between synesthetes with matches to the toy and those without is exposure to the stimulus. These data indicate learning of letter-color pairings from external contingencies can occur in a substantial fraction of synesthetes, and are consistent with the hypothesis that grapheme-color synesthesia is a kind of conditioned mental imagery.

Synesthesia in twins: incomplete concordance in monozygotes suggests extragenic factors.

Colored-sequence synesthesia (CSS) is a neurological condition in which sequential stimuli such as letters, numbers, or days of the week trigger simultaneous, involuntary color perception. Although the condition appears to run in families and several studies have sought a genetic link, the genetic contribution to synesthesia remains unclear. We conducted the first comparative twin study of CSS and found that CSS has a pairwise concordance of 73.9% in monozygotic twins, and a pairwise concordance of 36.4% in dizygotic twins. In line with previous studies, our results suggest a heritable element of synesthesia. However, consonant with the findings of previous single-pair case studies, our large sample size verifies that synesthesia is not completely conferred by genetics; if it were, monozygotic twins should have 100% concordance. These findings implicate a genetic mechanism of CSS that may work differently than previously thought: collectively, our data suggest that synesthesia is a heritable condition with incomplete penetrance that is substantially influenced by epigenetic and environmental factors.