Neurofeedback Training without Explicit Phantom Hand Movements and Hand-Like Visual Feedback to Modulate Pain: A Randomized Crossover Feasibility Trial.

Phantom limb pain is attributed to abnormal sensorimotor cortical representations, although the causal relationship between phantom limb pain and sensorimotor cortical representations suffers from the potentially confounding effects of phantom hand movements. We developed neurofeedback training to change sensorimotor cortical representations without explicit phantom hand movements or hand-like visual feedback. We tested the feasibility of neurofeedback training in fourteen patients with phantom limb pain. Neurofeedback training was performed in a single-blind, randomized, crossover trial using two decoders constructed using motor cortical currents measured during phantom hand movements; the motor cortical currents contralateral or ipsilateral to the phantom hand (contralateral and ipsilateral training) were estimated from magnetoencephalograms. Patients were instructed to control the size of a disk, which was proportional to the decoding results, but to not move their phantom hands or other body parts. The pain assessed by the visual analogue scale was significantly greater after contralateral training than after ipsilateral training. Classification accuracy of phantom hand movements significantly increased only after contralateral training. These results suggested that the proposed neurofeedback training changed phantom hand representation and modulated pain without explicit phantom hand movements or hand-like visual feedback, thus showing the relation between the phantom hand representations and pain. PERSPECTIVE: Our work demonstrates the feasibility of using neurofeedback training to change phantom hand representation and modulate pain perception without explicit phantom hand movements and hand-like visual feedback. The results enhance the mechanistic understanding of certain treatments, such as mirror therapy, that change the sensorimotor cortical representation.

Voluntary control of semantic neural representations by imagery with conflicting visual stimulation.

Neural representations of visual perception are affected by mental imagery and attention. Although attention is known to modulate neural representations, it is unknown how imagery changes neural representations when imagined and perceived images semantically conflict. We hypothesized that imagining an image would activate a neural representation during its perception even while watching a conflicting image. To test this hypothesis, we developed a closed-loop system to show images inferred from electrocorticograms using a visual semantic space. The successful control of the feedback images demonstrated that the semantic vector inferred from electrocorticograms became closer to the vector of the imagined category, even while watching images from different categories. Moreover, modulation of the inferred vectors by mental imagery depended asymmetrically on the perceived and imagined categories. Shared neural representation between mental imagery and perception was still activated by the imagery under semantically conflicting perceptions depending on the semantic category.

Deep image reconstruction from human brain activity.

The mental contents of perception and imagery are thought to be encoded in hierarchical representations in the brain, but previous attempts to visualize perceptual contents have failed to capitalize on multiple levels of the hierarchy, leaving it challenging to reconstruct internal imagery. Recent work showed that visual cortical activity measured by functional magnetic resonance imaging (fMRI) can be decoded (translated) into the hierarchical features of a pre-trained deep neural network (DNN) for the same input image, providing a way to make use of the information from hierarchical visual features. Here, we present a novel image reconstruction method, in which the pixel values of an image are optimized to make its DNN features similar to those decoded from human brain activity at multiple layers. We found that our method was able to reliably produce reconstructions that resembled the viewed natural images. A natural image prior introduced by a deep generator neural network effectively rendered semantically meaningful details to the reconstructions. Human judgment of the reconstructions supported the effectiveness of combining multiple DNN layers to enhance the visual quality of generated images. While our model was solely trained with natural images, it successfully generalized to artificial shapes, indicating that our model was not simply matching to exemplars. The same analysis applied to mental imagery demonstrated rudimentary reconstructions of the subjective content. Our results suggest that our method can effectively combine hierarchical neural representations to reconstruct perceptual and subjective images, providing a new window into the internal contents of the brain.

Generic decoding of seen and imagined objects using hierarchical visual features.

Object recognition is a key function in both human and machine vision. While brain decoding of seen and imagined objects has been achieved, the prediction is limited to training examples. We present a decoding approach for arbitrary objects using the machine vision principle that an object category is represented by a set of features rendered invariant through hierarchical processing. We show that visual features, including those derived from a deep convolutional neural network, can be predicted from fMRI patterns, and that greater accuracy is achieved for low-/high-level features with lower-/higher-level visual areas, respectively. Predicted features are used to identify seen/imagined object categories (extending beyond decoder training) from a set of computed features for numerous object images. Furthermore, decoding of imagined objects reveals progressive recruitment of higher-to-lower visual representations. Our results demonstrate a homology between human and machine vision and its utility for brain-based information retrieval.

Selective Suppression of Local Circuits during Movement Preparation in the Mouse Motor Cortex.

Prepared movements are more efficient than those that are not prepared for. Although changes in cortical activity have been observed prior to a forthcoming action, the circuits involved in motor preparation remain unclear. Here, we use in vivo two-photon calcium imaging to uncover changes in the motor cortex during variable waiting periods prior to a forepaw reaching task in mice. Consistent with previous reports, we observed a subset of neurons with increased activity during the waiting period; however, these neurons did not account for the degree of preparation as defined by reaction time (RT). Instead, the suppression of activity of distinct neurons in the same cortical area better accounts for RT. This suppression of neural activity resulted in a distinct and reproducible pattern when mice were well prepared. Thus, the selective suppression of network activity in the motor cortex may be a key feature of prepared movements.

Induced sensorimotor brain plasticity controls pain in phantom limb patients.

The cause of pain in a phantom limb after partial or complete deafferentation is an important problem. A popular but increasingly controversial theory is that it results from maladaptive reorganization of the sensorimotor cortex, suggesting that experimental induction of further reorganization should affect the pain, especially if it results in functional restoration. Here we use a brain-machine interface (BMI) based on real-time magnetoencephalography signals to reconstruct affected hand movements with a robotic hand. BMI training induces significant plasticity in the sensorimotor cortex, manifested as improved discriminability of movement information and enhanced prosthetic control. Contrary to our expectation that functional restoration would reduce pain, the BMI training with the phantom hand intensifies the pain. In contrast, BMI training designed to dissociate the prosthetic and phantom hands actually reduces pain. These results reveal a functional relevance between sensorimotor cortical plasticity and pain, and may provide a novel treatment with BMI neurofeedback.

[Decoding subjective mental states from FMRI activity patterns].

In recent years, functional magnetic resonance imaging (fMRI) decoding has emerged as a powerful tool to read out detailed stimulus features from multi-voxel brain activity patterns. Moreover, the method has been extended to perform a primitive form of 'mind-reading,' by applying a decoder "objectively" trained using stimulus features to more "subjective" conditions. In this paper, we first introduce basic procedures for fMRI decoding based on machine learning techniques. Second, we discuss the source of information used for decoding, in particular, the possibility of extracting information from subvoxel neural structures. We next introduce two experimental designs for decoding subjective mental states: the "objective-to-subjective design" and the "subjective-to-subjective design." Then, we illustrate recent studies on the decoding of a variety of mental states, such as, attention, awareness, decision making, memory, and mental imagery. Finally, we discuss the challenges and new directions of fMRI decoding.