Turning the operating room into a mixed-reality environment: a prospective clinical investigation for cerebral aneurysm clipping.

OBJECTIVE: The overall aim of this study was to demonstrate the potential benefit of a novel mixed-reality-head-mounted display (MR-HMD) on the spatial orientation of surgeons. METHODS: In a prospective clinical investigation, the authors applied for the first time a new multicamera navigation technology in an operating room setting that allowed them to directly compare MR-HMD navigation to standard monitor navigation. In the study, which included 14 patients with nonruptured middle cerebral artery aneurysms, the authors investigated how intuitively and effectively surgical instruments could be guided in 5 different visual navigation conditions. RESULTS: The authors demonstrate that multicamera tracking can be reliably integrated in a clinical setting (usability score 1.12 ± 0.31). Moreover, the technology captures large volumes of the operating room, allowing the team to track and integrate different devices and instruments, including MR-HMDs. Directly comparing mixed-reality navigation to standard monitor navigation revealed a significantly improved intuition in mixed reality, leading to navigation times that were twice as fast (2.1×, p ≤ 0.01). Despite the enhanced speed, the same targeting accuracy (approximately 2.5 mm, freehand tool use) in comparison to monitor navigation could be observed. Intraoperative planning strategies with mixed reality clearly outperformed classic preoperative planning: surgeons scored the mixed-reality plan as the best trajectory in 63% of the cases (chance level 33%). CONCLUSIONS: The incorporation of mixed reality in neurosurgical operations marks a significant advancement in the field. The use of mixed reality in brain surgery enhances the spatial awareness of surgeons, enabling more instinctive and precise surgical interventions. This technological integration promises to refine the execution of complex procedures without compromising accuracy.

Using camera-guided electrode microdrive navigation for precise 3D targeting of macaque brain sites.

Spatial accuracy in electrophysiological investigations is paramount, as precise localization and reliable access to specific brain regions help the advancement of our understanding of the brain's complex neural activity. Here, we introduce a novel, multi camera-based, frameless neuronavigation technique for precise, 3-dimensional electrode positioning in awake monkeys. The investigation of neural functions in awake primates often requires stable access to the brain with thin and delicate recording electrodes. This is usually realized by implanting a chronic recording chamber onto the skull of the animal that allows direct access to the dura. Most recording and positioning techniques utilize this implanted recording chamber as a holder of the microdrive or to hold a grid. This in turn reduces the degrees of freedom in positioning. To solve this problem, we require innovative, flexible, but precise tools for neuronal recordings. We instead mount the electrode microdrive above the animal on an arch, equipped with a series of translational and rotational micromanipulators, allowing movements in all axes. Here, the positioning is controlled by infrared cameras tracking the location of the microdrive and the monkey, allowing precise and flexible trajectories. To verify the accuracy of this technique, we created iron deposits in the tissue that could be detected by MRI. Our results demonstrate a remarkable precision with the confirmed physical location of these deposits averaging less than 0.5 mm from their planned position. Pilot electrophysiological recordings additionally demonstrate the accuracy and flexibility of this method. Our innovative approach could significantly enhance the accuracy and flexibility of neural recordings, potentially catalyzing further advancements in neuroscientific research.

Information-processing dynamics in neural networks of macaque cerebral cortex reflect cognitive state and behavior.

One of the essential functions of biological neural networks is the processing of information. This includes everything from processing sensory information to perceive the environment, up to processing motor information to interact with the environment. Due to methodological limitations, it has been historically unclear how information processing changes during different cognitive or behavioral states and to what extent information is processed within or between the network of neurons in different brain areas. In this study, we leverage recent advances in the calculation of information dynamics to explore neural-level processing within and between the frontoparietal areas AIP, F5, and M1 during a delayed grasping task performed by three macaque monkeys. While information processing was high within all areas during all cognitive and behavioral states of the task, interareal processing varied widely: During visuomotor transformation, AIP and F5 formed a reciprocally connected processing unit, while no processing was present between areas during the memory period. Movement execution was processed globally across all areas with predominance of processing in the feedback direction. Furthermore, the fine-scale network structure reconfigured at the neuron level in response to different grasping conditions, despite no differences in the overall amount of information present. These results suggest that areas dynamically form higher-order processing units according to the cognitive or behavioral demand and that the information-processing network is hierarchically organized at the neuron level, with the coarse network structure determining the behavioral state and finer changes reflecting different conditions.

A Turntable Setup for Testing Visual and Tactile Grasping Movements in Non-human Primates.

Grasping movements are some of the most common movements primates do every day. They are important for social interactions as well as picking up objects or food. Usually, these grasping movements are guided by vision but proprioceptive and haptic inputs contribute greatly. Since grasping behaviors are common and easy to motivate, they represent an ideal task for understanding the role of different brain areas during planning and execution of complex voluntary movements in primates. For experimental purposes, a stable and repeatable presentation of the same object as well as the variation of objects is important in order to understand the neural control of movement generation. This is even more the case when investigating the role of different senses for movement planning, where objects need to be presented in specific sensory modalities. We developed a turntable setup for non-human primates (macaque monkeys) to investigate visually and tactually guided grasping movements with an option to easily exchange objects. The setup consists of a turntable that can fit six different objects and can be exchanged easily during the experiment to increase the number of presented objects. The object turntable is connected to a stepper motor through a belt system to automate rotation and hence object presentation. By increasing the distance between the turntable and the stepper motor, metallic components of the stepper motor are kept at a distance to the actual recording setup, which allows using a magnetic-based data glove to track hand kinematics. During task execution, the animal sits in the dark and is instructed to grasp the object in front of it. Options to turn on a light above the object allow for visual presentation of the objects, while the object can also remain in the dark for exclusive tactile exploration. A red LED is projected onto the object by a one-way mirror that serves as a grasp cue instruction for the animal to start grasping the object. By comparing kinematic data from the magnetic-based data glove with simultaneously recorded neural signals, this setup enables the systematic investigation of neural population activity involved in the neural control of hand grasping movements.

A goal-driven modular neural network predicts parietofrontal neural dynamics during grasping.

One of the primary ways we interact with the world is using our hands. In macaques, the circuit spanning the anterior intraparietal area, the hand area of the ventral premotor cortex, and the primary motor cortex is necessary for transforming visual information into grasping movements. However, no comprehensive model exists that links all steps of processing from vision to action. We hypothesized that a recurrent neural network mimicking the modular structure of the anatomical circuit and trained to use visual features of objects to generate the required muscle dynamics used by primates to grasp objects would give insight into the computations of the grasping circuit. Internal activity of modular networks trained with these constraints strongly resembled neural activity recorded from the grasping circuit during grasping and paralleled the similarities between brain regions. Network activity during the different phases of the task could be explained by linear dynamics for maintaining a distributed movement plan across the network in the absence of visual stimulus and then generating the required muscle kinematics based on these initial conditions in a module-specific way. These modular models also outperformed alternative models at explaining neural data, despite the absence of neural data during training, suggesting that the inputs, outputs, and architectural constraints imposed were sufficient for recapitulating processing in the grasping circuit. Finally, targeted lesioning of modules produced deficits similar to those observed in lesion studies of the grasping circuit, providing a potential model for how brain regions may coordinate during the visually guided grasping of objects.

Resting state functional connectivity patterns associated with pharmacological treatment resistance in temporal lobe epilepsy.

There are no functional imaging based biomarkers for pharmacological treatment response in temporal lobe epilepsy (TLE). In this study, we investigated whether there is an association between resting state functional brain connectivity (RsFC) and seizure control in TLE. We screened a large database containing resting state functional magnetic resonance imaging (Rs-fMRI) data from 286 epilepsy patients. Patient medical records were screened for seizure characterization, EEG reports for lateralization and location of seizure foci to establish uniformity of seizure localization within patient groups. Rs-fMRI data from patients with well-controlled left TLE, patients with treatment-resistant left TLE, and healthy controls were analyzed. Healthy controls and cTLE showed similar functional connectivity patterns, whereas trTLE exhibited a significant bilateral decrease in thalamo-hippocampal functional connectivity. This work is the first to demonstrate differences in neural network connectivity between well-controlled and treatment-resistant TLE. These differences are spatially highly focused and suggest sites for the etiology and possibly treatment of TLE. Altered thalamo-hippocampal RsFC thus is a potential new biomarker for TLE treatment resistance.

Uniting functional network topology and oscillations in the fronto-parietal single unit network of behaving primates.

The functional communication of neurons in cortical networks underlies higher cognitive processes. Yet, little is known about the organization of the single neuron network or its relationship to the synchronization processes that are essential for its formation. Here, we show that the functional single neuron network of three fronto-parietal areas during active behavior of macaque monkeys is highly complex. The network was closely connected (small-world) and consisted of functional modules spanning these areas. Surprisingly, the importance of different neurons to the network was highly heterogeneous with a small number of neurons contributing strongly to the network function (hubs), which were in turn strongly inter-connected (rich-club). Examination of the network synchronization revealed that the identified rich-club consisted of neurons that were synchronized in the beta or low frequency range, whereas other neurons were mostly non-oscillatory synchronized. Therefore, oscillatory synchrony may be a central communication mechanism for highly organized functional spiking networks.

Object vision to hand action in macaque parietal, premotor, and motor cortices.

Grasping requires translating object geometries into appropriate hand shapes. How the brain computes these transformations is currently unclear. We investigated three key areas of the macaque cortical grasping circuit with microelectrode arrays and found cooperative but anatomically separated visual and motor processes. The parietal area AIP operated primarily in a visual mode. Its neuronal population revealed a specialization for shape processing, even for abstract geometries, and processed object features ultimately important for grasping. Premotor area F5 acted as a hub that shared the visual coding of AIP only temporarily and switched to highly dominant motor signals towards movement planning and execution. We visualize these non-discrete premotor signals that drive the primary motor cortex M1 to reflect the movement of the grasping hand. Our results reveal visual and motor features encoded in the grasping circuit and their communication to achieve transformation for grasping.

Representation of continuous hand and arm movements in macaque areas M1, F5, and AIP: a comparative decoding study.

OBJECTIVE: In the last decade, multiple brain areas have been investigated with respect to their decoding capability of continuous arm or hand movements. So far, these studies have mainly focused on motor or premotor areas like M1 and F5. However, there is accumulating evidence that anterior intraparietal area (AIP) in the parietal cortex also contains information about continuous movement. APPROACH: In this study, we decoded 27 degrees of freedom representing complete hand and arm kinematics during a delayed grasping task from simultaneously recorded activity in areas M1, F5, and AIP of two macaque monkeys (Macaca mulatta). MAIN RESULTS: We found that all three areas provided decoding performances that lay significantly above chance. In particular, M1 yielded highest decoding accuracy followed by F5 and AIP. Furthermore, we provide support for the notion that AIP does not only code categorical visual features of objects to be grasped, but also contains a substantial amount of temporal kinematic information. SIGNIFICANCE: This fact could be utilized in future developments of neural interfaces restoring hand and arm movements.

Decoding a wide range of hand configurations from macaque motor, premotor, and parietal cortices.

Despite recent advances in decoding cortical activity for motor control, the development of hand prosthetics remains a major challenge. To reduce the complexity of such applications, higher cortical areas that also represent motor plans rather than just the individual movements might be advantageous. We investigated the decoding of many grip types using spiking activity from the anterior intraparietal (AIP), ventral premotor (F5), and primary motor (M1) cortices. Two rhesus monkeys were trained to grasp 50 objects in a delayed task while hand kinematics and spiking activity from six implanted electrode arrays (total of 192 electrodes) were recorded. Offline, we determined 20 grip types from the kinematic data and decoded these hand configurations and the grasped objects with a simple Bayesian classifier. When decoding from AIP, F5, and M1 combined, the mean accuracy was 50% (using planning activity) and 62% (during motor execution) for predicting the 50 objects (chance level, 2%) and substantially larger when predicting the 20 grip types (planning, 74%; execution, 86%; chance level, 5%). When decoding from individual arrays, objects and grip types could be predicted well during movement planning from AIP (medial array) and F5 (lateral array), whereas M1 predictions were poor. In contrast, predictions during movement execution were best from M1, whereas F5 performed only slightly worse. These results demonstrate for the first time that a large number of grip types can be decoded from higher cortical areas during movement preparation and execution, which could be relevant for future neuroprosthetic devices that decode motor plans.

Real-time position reconstruction with hippocampal place cells.

Brain-computer interfaces (BCI) are using the electroencephalogram, the electrocorticogram and trains of action potentials as inputs to analyze brain activity for communication purposes and/or the control of external devices. Thus far it is not known whether a BCI system can be developed that utilizes the states of brain structures that are situated well below the cortical surface, such as the hippocampus. In order to address this question we used the activity of hippocampal place cells (PCs) to predict the position of an rodent in real-time. First, spike activity was recorded from the hippocampus during foraging and analyzed off-line to optimize the spike sorting and position reconstruction algorithm of rats. Then the spike activity was recorded and analyzed in real-time. The rat was running in a box of 80 cm × 80 cm and its locomotor movement was captured with a video tracking system. Data were acquired to calculate the rat's trajectories and to identify place fields. Then a Bayesian classifier was trained to predict the position of the rat given its neural activity. This information was used in subsequent trials to predict the rat's position in real-time. The real-time experiments were successfully performed and yielded an error between 12.2 and 17.4% using 5-6 neurons. It must be noted here that the encoding step was done with data recorded before the real-time experiment and comparable accuracies between off-line (mean error of 15.9% for three rats) and real-time experiments (mean error of 14.7%) were achieved. The experiment shows proof of principle that position reconstruction can be done in real-time, that PCs were stable and spike sorting was robust enough to generalize from the training run to the real-time reconstruction phase of the experiment. Real-time reconstruction may be used for a variety of purposes, including creating behavioral-neuronal feedback loops or for implementing neuroprosthetic control.