Simulated operant reflex conditioning environment reveals effects of feedback parameters.

Operant conditioning of neural activation has been researched for decades in humans and animals. Many theories suggest two parallel learning processes, implicit and explicit. The degree to which feedback affects these processes individually remains to be fully understood and may contribute to a large percentage of non-learners. Our goal is to determine the explicit decision-making processes in response to feedback representing an operant conditioning environment. We developed a simulated operant conditioning environment based on a feedback model of spinal reflex excitability, one of the simplest forms of neural operant conditioning. We isolated the perception of the feedback signal from self-regulation of an explicit unskilled visuomotor task, enabling us to quantitatively examine feedback strategy. Our hypothesis was that feedback type, biological variability, and reward threshold affect operant conditioning performance and operant strategy. Healthy individuals (N = 41) were instructed to play a web application game using keyboard inputs to rotate a virtual knob representative of an operant strategy. The goal was to align the knob with a hidden target. Participants were asked to "down-condition" the amplitude of the virtual feedback signal, which was achieved by placing the knob as close as possible to the hidden target. We varied feedback type (knowledge of performance, knowledge of results), biological variability (low, high), and reward threshold (easy, moderate, difficult) in a factorial design. Parameters were extracted from real operant conditioning data. Our main outcomes were the feedback signal amplitude (performance) and the mean change in dial position (operant strategy). We observed that performance was modulated by variability, while operant strategy was modulated by feedback type. These results show complex relations between fundamental feedback parameters and provide the principles for optimizing neural operant conditioning for non-responders.

OPERANT REFLEX CONDITIONING SIMULATION ENVIRONMENT REVEALS EFFECTS OF FEEDBACK PARAMETERS.

Operant conditioning of neural activation has been researched for decades in humans and animals. Many theories suggest two parallel learning processes, implicit and explicit. The degree to which feedback affects these processes individually remains to be fully understood and may contribute to a large percentage of non-learners. Our goal is to determine the explicit decision-making processes in response to feedback representing an operant conditioning environment. We developed a simulated operant conditioning environment based on a feedback model of spinal reflex excitability, one of the simplest forms of neural operant conditioning. We isolated the perception of the feedback signal from self-regulation of an explicit unskilled visuomotor task, enabling us to quantitatively examine feedback strategy. Our hypothesis was that feedback type, signal quality and success threshold affect operant conditioning performance and operant strategy. Healthy individuals (N = 41) were instructed to play a web application game using keyboard inputs to rotate a virtual knob representative of an operant strategy. The goal was to align the knob with a hidden target. Participants were asked to "down-condition" the amplitude of the virtual feedback signal, which was achieved by placing the knob as close as possible to the hidden target. We varied feedback type (knowledge of performance, knowledge of results), success threshold (easy, moderate, difficult), and biological variability (low, high) in a factorial design. Parameters were extracted from real operant conditioning data. Our main outcomes were the feedback signal amplitude (performance) and the mean change in dial position (operant strategy). We observed that performance was modulated by variability, while operant strategy was modulated by feedback type. These results show complex relations between fundamental feedback parameters and provide the principles for optimizing neural operant conditioning for non-responders.

Differential neural plasticity of individual fingers revealed by fMRI neurofeedback.

Previous work has shown that functional magnetic resonance imaging (fMRI) activity patterns associated with individual fingers can be shifted by temporary impairment of the hand. Here, we investigated whether these neural activity patterns could be modulated endogenously and whether any behavioral changes result from this modulation. We used decoded neurofeedback in healthy individuals to encourage participants to shift the neural activity pattern in sensorimotor cortex of the middle finger toward the index finger, and the ring finger toward the little finger. We first mapped the neural activity patterns for all fingers of the right hand in an fMRI pattern localizer session. Then, in three subsequent neurofeedback sessions, participants were rewarded after middle/ring finger presses according to their activity pattern overlap during each trial. A force-sensitive keyboard was used to ensure that participants were not altering their physical finger coordination patterns. We found evidence that participants could learn to shift the activity pattern of the ring finger but not of the middle finger. Increased variability of these activity patterns during the localizer session was associated with the ability of participants to modulate them using neurofeedback. Participants also showed an increased preference for the ring finger but not for the middle finger in a postneurofeedback motor task. Our results show that neural activity and behaviors associated with the ring finger are more readily modulated than those associated with the middle finger. These results have broader implications for rehabilitation of individual finger movements, which may be limited or enhanced by individual finger plasticity after neurological injury.NEW & NOTEWORTHY It may be possible to remobilize fingers after neurological injury by altering neural activity patterns. Toward this end, we examined whether finger-related neural activity patterns could be modified in healthy individuals without physical intervention, using fMRI neurofeedback. Our findings show that greater variability of neural patterns at baseline predicted a participant's ability to successfully shift these patterns. Because neural variability is common in individuals poststroke, this illustrates a potential clinical benefit of this procedure.

Towards a common template for neural reinforcement of finger individuation.

The inability to individuate finger movements is a common impairment following stroke. Conventional physical therapy ignores underlying neural changes with recovery, leaving it unclear why sensorimotor function often remains impaired. Functional MRI neurofeedback can monitor neural activity and reinforce it towards a healthy template to restore function. However, identifying an individualized training template may not be possible depending on the severity of impairment. In this study, we investigated the use of functional alignment of brain data across healthy participants to create an idealized neural template to be used as a training target for new participants. We employed multi-voxel pattern analyses to assess the prediction accuracy and robustness to missing data of pre-trained functional templates corresponding to individual finger presses. We found a significant improvement in classification accuracy (p < 0.001) of individual finger presses when group data was aligned based on function (88%) rather than anatomy (46%). Importantly, we found no significant drop in performance when aligning a new participant to a pre-established template as compared to including this new participant in the creation of a new template. These results indicate that functionally aligned templates could provide an effective surrogate training target for patients following neurological injury.

A simulation-based approach to improve decoded neurofeedback performance.

The neural correlates of specific brain functions such as visual orientation tuning and individual finger movements can be revealed using multivoxel pattern analysis (MVPA) of fMRI data. Neurofeedback based on these distributed patterns of brain activity presents a unique ability for precise neuromodulation. Recent applications of this technique, known as decoded neurofeedback, have manipulated fear conditioning, visual perception, confidence judgements and facial preference. However, there has yet to be an empirical justification of the timing and data processing parameters of these experiments. Suboptimal parameter settings could impact the efficacy of neurofeedback learning and contribute to the 'non-responder' effect. The goal of this study was to investigate how design parameters of decoded neurofeedback experiments affect decoding accuracy and neurofeedback performance. Subjects participated in three fMRI sessions: two 'finger localizer' sessions to identify the fMRI patterns associated with each of the four fingers of the right hand, and one 'finger finding' neurofeedback session to assess neurofeedback performance. Using only the localizer data, we show that real-time decoding can be degraded by poor experiment timing or ROI selection. To set key parameters for the neurofeedback session, we used offline simulations of decoded neurofeedback using data from the localizer sessions to predict neurofeedback performance. We show that these predictions align with real neurofeedback performance at the group level and can also explain individual differences in neurofeedback success. Overall, this work demonstrates the usefulness of offline simulation to improve the success of real-time decoded neurofeedback experiments.

Author Correction: Closed-loop brain training: the science of neurofeedback.

In this article, the affiliation for Mohit Rana was incorrectly listed as the Institute for Biological and Medical Engineering, Department of Psychiatry, and Section of Neuroscience, Pontificia Universidad Católica de Chile, Vicuña Mackenna 4860 Hernán Briones, piso 2, Macul 782-0436, Santiago, Chile. The listed affiliation should have been the following: Departamento de Psiquiatría, Escuela de Medicina, Centro Interdisciplinario de Neurociencias, Pontificia Universidad Católica de Chile, Santiago, Chile; and the Laboratory for Brain-Machine Interfaces and Neuromodulation, Pontificia Universidad Católica de Chile, Santiago, Chile. An acknowledgement to Mohit Rana's funding source was also missing. The following sentence should have been included in the acknowledgments section: M.R. is supported by a Fondecyt postdoctoral fellowship (project no. 3100648).

An MRI-compatible force sensor for measuring differential isometric precision grip force.

Investigating neural correlates of fine motor control in a magnetically sensitive environment requires special considerations in sensor design. Our application requires measurement of forefinger and thumb forces during precision grip in a relatively low (<; 20 N) force range. This work describes the design, characterization and performance evaluation of an MR-compatible precision grip sensor that independently measures forefinger and thumb forces. We selective laser sintered Nylon 12 into a flexure, measuring deformation using optic fibers which matches our finite element model simulation. We found that the device was capable of measuring forces within the desired range, with some hysteresis at higher frequencies as expected. We conclude that the device performs well compared to specifications.

Self-regulation strategy, feedback timing and hemodynamic properties modulate learning in a simulated fMRI neurofeedback environment.

Direct manipulation of brain activity can be used to investigate causal brain-behavior relationships. Current noninvasive neural stimulation techniques are too coarse to manipulate behaviors that correlate with fine-grained spatial patterns recorded by fMRI. However, these activity patterns can be manipulated by having people learn to self-regulate their own recorded neural activity. This technique, known as fMRI neurofeedback, faces challenges as many participants are unable to self-regulate. The causes of this non-responder effect are not well understood due to the cost and complexity of such investigation in the MRI scanner. Here, we investigated the temporal dynamics of the hemodynamic response measured by fMRI as a potential cause of the non-responder effect. Learning to self-regulate the hemodynamic response involves a difficult temporal credit-assignment problem because this signal is both delayed and blurred over time. Two factors critical to this problem are the prescribed self-regulation strategy (cognitive or automatic) and feedback timing (continuous or intermittent). Here, we sought to evaluate how these factors interact with the temporal dynamics of fMRI without using the MRI scanner. We first examined the role of cognitive strategies by having participants learn to regulate a simulated neurofeedback signal using a unidimensional strategy: pressing one of two buttons to rotate a visual grating that stimulates a model of visual cortex. Under these conditions, continuous feedback led to faster regulation compared to intermittent feedback. Yet, since many neurofeedback studies prescribe implicit self-regulation strategies, we created a computational model of automatic reward-based learning to examine whether this result held true for automatic processing. When feedback was delayed and blurred based on the hemodynamics of fMRI, this model learned more reliably from intermittent feedback compared to continuous feedback. These results suggest that different self-regulation mechanisms prefer different feedback timings, and that these factors can be effectively explored and optimized via simulation prior to deployment in the MRI scanner.

Closed-loop brain training: the science of neurofeedback.

Neurofeedback is a psychophysiological procedure in which online feedback of neural activation is provided to the participant for the purpose of self-regulation. Learning control over specific neural substrates has been shown to change specific behaviours. As a progenitor of brain-machine interfaces, neurofeedback has provided a novel way to investigate brain function and neuroplasticity. In this Review, we examine the mechanisms underlying neurofeedback, which have started to be uncovered. We also discuss how neurofeedback is being used in novel experimental and clinical paradigms from a multidisciplinary perspective, encompassing neuroscientific, neuroengineering and learning-science viewpoints.