Distributional reinforcement learning in prefrontal cortex.

The prefrontal cortex is crucial for learning and decision-making. Classic reinforcement learning (RL) theories center on learning the expectation of potential rewarding outcomes and explain a wealth of neural data in the prefrontal cortex. Distributional RL, on the other hand, learns the full distribution of rewarding outcomes and better explains dopamine responses. In the present study, we show that distributional RL also better explains macaque anterior cingulate cortex neuronal responses, suggesting that it is a common mechanism for reward-guided learning.

Intraoperative Laser Speckle Contrast Imaging to Assess Vessel Flow in Neurosurgery: A Pilot Study.

BACKGROUND AND OBJECTIVES: Laser speckle contrast imaging (LSCI) has emerged as a promising tool for assessment of vessel flow during neurosurgery. We aimed to investigate the feasibility of visualizing vessel flow in the macrocirculation with a new fully microscope-integrated LSCI system and assess the validity and objectivity of findings compared with fluorescence angiography (FA). METHODS: This is a single-center prospective observational study enrolling adult patients requiring microsurgical treatment for brain vascular pathologies or brain tumors. Three independent raters, blinded toward findings of FA, reviewed regions of interest (ROIs) placed in exposed vessels and target structures. The primary end point was the validity of LSCI for assessment of vessel flow as measured by the agreement with FA. The secondary end point was objectivity, measured as the inter-rater agreement of LSCI findings. RESULTS: During 18 surgical procedures, 23 observations using FA and LSCI were captured simultaneously. Using LSCI, vessel flow was assessable in 62 (86.1%) and not assessable in 10 (13.9%) ROIs. The agreement between LSCI and FA was 86.1%, with an agreement coefficient of 0.85 (95% CI: 0.75-0.94). Disagreement between LSCI and FA was observed in the 10 ROIs that were not assessable. The agreement between ROIs that were assessable using LSCI and FA was 100%. The inter-rater agreement of LSCI findings was 87.9%, with an agreement coefficient of 0.86 (95% CI: 0.79-0.94). CONCLUSION: Fully microscope-integrated LSCI is feasible and has a high potential for clinical utility. Because of its characteristics, LSCI can be viewed as a full-field visual micro-Doppler that can be used as a complementary method to FA for assessing vessel flow during neurosurgery. Despite technical limitations related to the early development phase of the fully microscope-integrated system, we demonstrated reasonable validity and objectivity of findings compared with FA. Further research and refinement of the system may enhance its value in neurosurgical applications.

Generative replay underlies compositional inference in the hippocampal-prefrontal circuit.

Human reasoning depends on reusing pieces of information by putting them together in new ways. However, very little is known about how compositional computation is implemented in the brain. Here, we ask participants to solve a series of problems that each require constructing a whole from a set of elements. With fMRI, we find that representations of novel constructed objects in the frontal cortex and hippocampus are relational and compositional. With MEG, we find that replay assembles elements into compounds, with each replay sequence constituting a hypothesis about a possible configuration of elements. The content of sequences evolves as participants solve each puzzle, progressing from predictable to uncertain elements and gradually converging on the correct configuration. Together, these results suggest a computational bridge between apparently distinct functions of hippocampal-prefrontal circuitry and a role for generative replay in compositional inference and hypothesis testing.

A cognitive map for value-guided choice in ventromedial prefrontal cortex.

The prefrontal cortex is crucial for economic decision-making and representing the value of options. However, how such representations facilitate flexible decisions remains unknown. We reframe economic decision-making in prefrontal cortex in line with representations of structure within the medial temporal lobe because such cognitive map representations are known to facilitate flexible behaviour. Specifically, we framed choice between different options as a navigation process in value space. Here we show that choices in a 2D value space defined by reward magnitude and probability were represented with a grid-like code, analogous to that found in spatial navigation. The grid-like code was present in ventromedial prefrontal cortex (vmPFC) local field potential theta frequency and the result replicated in an independent dataset. Neurons in vmPFC similarly contained a grid-like code, in addition to encoding the linear value of the chosen option. Importantly, both signals were modulated by theta frequency - occurring at theta troughs but on separate theta cycles. Furthermore, we found sharp-wave ripples - a key neural signature of planning and flexible behaviour - in vmPFC, which were modulated by accuracy and reward. These results demonstrate that multiple cognitive map-like computations are deployed in vmPFC during economic decision-making, suggesting a new framework for the implementation of choice in prefrontal cortex.

Association of perioperative adverse events with subsequent therapy and overall survival in patients with WHO grade III and IV gliomas.

Background: Maximum safe resection followed by chemoradiotherapy as current standard of care for WHO grade III and IV gliomas can be influenced by the occurrence of perioperative adverse events (AE). The aim of this study was to determine the association of AE with the timing and choice of subsequent treatments as well as with overall survival (OS). Methods: Prospectively collected data of 283 adult patients undergoing surgery for WHO grade III and IV gliomas at the University Hospital Zurich between January 2013 and June 2017 were analyzed. We assessed basic patient characteristics, KPS, extent of resection, and WHO grade, and we classified AE as well as modality, timing of subsequent treatment (delay, interruption, or non-initiation), and OS. Results: In 117 patients (41%), an AE was documented between surgery and the 3-month follow-up. There was a significant association of AE with an increased time to initiation of subsequent therapy (p = 0.005) and a higher rate of interruption (p < 0.001) or non-initiation (p < 0.001). AE grades correlated with time to initiation of subsequent therapy (p = 0.038). AEs were associated with shorter OS in univariate analysis (p < 0.001). Conclusion: AEs are associated with delayed and/or altered subsequent therapy and can therefore limit OS. These data emphasize the importance of safety within the maximum-safe-resection concept.

Maximal surgical tumour load reduction in immune-checkpoint inhibitor naïve patients with melanoma brain metastases correlates with prolonged survival.

BACKGROUND: Recent therapeutic advances in metastatic melanoma have led to improved overall survival (OS) rates, with consequently an increased incidence of brain metastases (BM). The role of BM resection in the era of targeted and immunotherapy should be reassessed. In the current study we analysed the role of residual intracranial tumour load in a cohort of melanoma BM patients. METHODS: Retrospective single-centre analysis of a prospective registry of resected melanoma BM from 2013 to 2021. Correlations of residual tumour volume and outcome were determined with respect to patient, tumour and treatment regimens characteristics. RESULTS: 121 individual patients (66% male, mean age 59.9 years) were identified and included in the study. Pre- and postoperative systemic treatments included BRAF/MEK inhibitors, as well as combination or monotherapy of immune-checkpoint inhibitors (ICIs). Median OS of the entire cohort was 20 months. Cox proportional-hazard analysis revealed postoperative anti-CTLA4+anti-PD-1 therapy (HR 0.07, p = .01) and postoperative residual intracranial tumour burden (HR 1.4, p = .027) as significant predictors for OS. Further analysis revealed that ICI-naïve patients with residual tumour volume ≤3.5 cm3 and postoperative ICI showed significantly prolonged OS compared to patients with residual volume >3.5 cm3 (p < .0001). Subgroup analysis of ICI-naïve patients showed steroid intake postoperatively to be negatively associated with OS, however residual tumour volume ≤3.5 cm3 remained independently correlated with superior OS (HR 0.14, p < .001). CONCLUSION: Besides known predictive factors like postoperative ICI, a maximal intracranial tumour burden reduction seems to be beneficial, especially in ICI-naïve patients. This highlights the importance of local CNS control and the need to further investigating the role of initial surgical tumour load reduction in randomised clinical trials.

Anatomical phenotyping and staging of brain tumours.

Unlike other tumours, the anatomical extent of brain tumours is not objectified and quantified through staging. Staging systems are based on understanding the anatomical sequence of tumour progression and its relationship to histopathological dedifferentiation and survival. The aim of this study was to describe the spatiotemporal phenotype of the most frequent brain tumour entities, to assess the association of anatomical tumour features with survival probability and to develop a staging system for WHO grade 2 and 3 gliomas and glioblastoma. Anatomical phenotyping was performed on a consecutive cohort of 1000 patients with first diagnosis of a primary or secondary brain tumour. Tumour probability in different topographic, phylogenetic and ontogenetic parcellation units was assessed on preoperative MRI through normalization of the relative tumour prevalence to the relative volume of the respective structure. We analysed the spatiotemporal tumour dynamics by cross-referencing preoperative against preceding and subsequent MRIs of the respective patient. The association between anatomical phenotype and outcome defined prognostically critical anatomical tumour features at diagnosis. Based on a hypothesized sequence of anatomical tumour progression, we developed a three-level staging system for WHO grade 2 and 3 gliomas and glioblastoma. This staging system was validated internally in the original cohort and externally in an independent cohort of 300 consecutive patients. While primary CNS lymphoma showed highest probability along white matter tracts, metastases enriched along terminal arterial flow areas. Neuroepithelial tumours mapped along all sectors of the ventriculocortical axis, while adjacent units were spared, consistent with a transpallial behaviour within phylo-ontogenetic radial units. Their topographic pattern correlated with morphogenetic processes of convergence and divergence of radial units during phylo- and ontogenesis. While a ventriculofugal growth dominated in neuroepithelial tumours, a gradual deviation from this neuroepithelial spatiotemporal behaviour was found with progressive histopathological dedifferentiation. The proposed three-level staging system for WHO grade 2 and 3 gliomas and glioblastoma correlated with the degree of histological dedifferentiation and proved accurate in terms of survival upon both internal and external validation. In conclusion, this study identified specific spatiotemporal phenotypes in brain tumours through topographic probability and growth pattern assessment. The association of anatomical tumour features with survival defined critical steps in the anatomical sequence of neuroepithelial tumour progression, based on which a staging system for WHO grade 2 and 3 gliomas and glioblastoma was developed and validated.

Impact of additional resection on new ischemic lesions and their clinical relevance after intraoperative 3 Tesla MRI in neuro-oncological surgery.

Intraoperative MRI (ioMRI) has become a frequently used tool to improve maximum safe resection in brain tumor surgery. The usability of intraoperatively acquired diffusion-weighted imaging sequences to predict the extent and clinical relevance of new infarcts has not yet been studied. Furthermore, the question of whether more aggressive surgery after ioMRI leads to more or larger infarcts is of crucial interest for the surgeons' operative strategy. Retrospective single-center analysis of a prospective registry of procedures from 2013 to 2019 with ioMRI was used. Infarct volumes in ioMRI/poMRI, lesion localization, mRS, and NIHSS were analyzed for each case. A total of 177 individual operations (60% male, mean age 45.5 years old) met the inclusion criteria. In 61% of the procedures, additional resection was performed after ioMRI, which resulted in a significantly higher number of new ischemic lesions postoperatively (p < .001). The development of new or enlarged ischemic areas upon additional resection could also be shown volumetrically (mean volume in ioMRI 0.39 cm3 vs. poMRI 2.97 cm3; p < .001). Despite the surgically induced new infarcts, mRS and NIHSS did not worsen significantly in cases with additional resection. Additionally, new perilesional ischemia in eloquently located tumors was not associated with an impaired neurological outcome. Additional resection after ioMRI leads to new or enlarged ischemic areas. However, these new infarcts do not necessarily result in an impaired neurological outcome, even when in eloquent brain areas.

COVID-19 global pandemic planning: Dry heat incubation and ambient temperature fail to consistently inactivate SARS-CoV-2 on N95 respirators.

The ongoing pandemic of the novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has placed a substantial strain on the supply of personal protective equipment, particularly the availability of N95 respirators for frontline healthcare personnel. These shortages have led to the creation of protocols to disinfect and reuse potentially contaminated personal protective equipment. A simple and inexpensive decontamination procedure that does not rely on the use of consumable supplies is dry heat incubation. Although reprocessing with this method has been shown to maintain the integrity of N95 respirators after multiple decontamination procedures, information on the ability of dry heat incubation to inactivate SARS-CoV-2 is largely unreported. Here, we show that dry heat incubation does not consistently inactivate SARS-CoV-2-contaminated N95 respirators, and that variation in experimental conditions can dramatically affect viability of the virus. Furthermore, we show that SARS-CoV-2 can survive on N95 respirators that remain at room temperature for at least five days. Collectively, our findings demonstrate that dry heat incubation procedures and ambient temperature for five days are not viable methods for inactivating SARS-CoV-2 on N95 respirators for potential reuse. We recommend that decontamination procedures being considered for the reuse of N95 respirators be validated at each individual site and that validation of the process must be thoroughly conducted using a defined protocol.

Entorhinal and ventromedial prefrontal cortices abstract and generalize the structure of reinforcement learning problems.

Knowledge of the structure of a problem, such as relationships between stimuli, enables rapid learning and flexible inference. Humans and other animals can abstract this structural knowledge and generalize it to solve new problems. For example, in spatial reasoning, shortest-path inferences are immediate in new environments. Spatial structural transfer is mediated by cells in entorhinal and (in humans) medial prefrontal cortices, which maintain their co-activation structure across different environments and behavioral states. Here, using fMRI, we show that entorhinal and ventromedial prefrontal cortex (vmPFC) representations perform a much broader role in generalizing the structure of problems. We introduce a task-remapping paradigm, where subjects solve multiple reinforcement learning (RL) problems differing in structural or sensory properties. We show that, as with space, entorhinal representations are preserved across different RL problems only if task structure is preserved. In vmPFC and ventral striatum, representations of prediction error also depend on task structure.

Design and Construction of a Biosafety Level 3 Autopsy Laboratory.

CONTEXT.­: Autopsy pathologists, including medical examiners, provide valuable public health support for infectious disease deaths through surveillance for deaths of public health concern including emerging infections, identifying causative organisms for unexplained deaths, and providing insights into the pathology and pathogenesis of novel or unusual infections. However, autopsy poses biosafety risks to workers within and outside the laboratory. The highest rates of laboratory-acquired infections occur in autopsy workers. OBJECTIVE.­: To design and construct an appropriately biosafe autopsy laboratory. DESIGN.­: We conducted a biosafety risk assessment for autopsy workers using the process developed by the US Centers for Disease Control and Prevention and National Institutes of Health and applied these findings as the basis of laboratory design and construction. RESULTS.­: Autopsy workers are unpredictably exposed to a variety of infectious organisms, including hepatitis C virus, HIV, and Mycobacterium tuberculosis. Hazardous autopsy procedures include using and encountering sharp objects and the generation of aerosols from dissection, fluid aspiration, rinsing tissues, and dividing bone with an oscillating saw. CONCLUSIONS.­: Exposure to blood-borne and airborne pathogens from procedures that can cause cutaneous inoculation and inhalation of aerosols indicates that human autopsies should be performed at biosafety level 3. We designed a large, entirely biosafety level 3 medical examiner autopsy laboratory using design principles and characteristics that can be scaled to accommodate smaller academic or other hospital-based autopsy spaces. Containment was achieved through a concentric ring design, with access control at interface zones. As new autopsy laboratories are planned, we strongly recommend that they be designed to function uniformly at biosafety level 3.

The Tolman-Eichenbaum Machine: Unifying Space and Relational Memory through Generalization in the Hippocampal Formation.

The hippocampal-entorhinal system is important for spatial and relational memory tasks. We formally link these domains, provide a mechanistic understanding of the hippocampal role in generalization, and offer unifying principles underlying many entorhinal and hippocampal cell types. We propose medial entorhinal cells form a basis describing structural knowledge, and hippocampal cells link this basis with sensory representations. Adopting these principles, we introduce the Tolman-Eichenbaum machine (TEM). After learning, TEM entorhinal cells display diverse properties resembling apparently bespoke spatial responses, such as grid, band, border, and object-vector cells. TEM hippocampal cells include place and landmark cells that remap between environments. Crucially, TEM also aligns with empirically recorded representations in complex non-spatial tasks. TEM also generates predictions that hippocampal remapping is not random as previously believed; rather, structural knowledge is preserved across environments. We confirm this structural transfer over remapping in simultaneously recorded place and grid cells.

COVID-19 global pandemic planning: Decontamination and reuse processes for N95 respirators.

IMPACT STATEMENT: There is a critical shortage of personal protective equipment (PPE) around the globe. This article describes the safe collection, storage, and decontamination of N95 respirators using hydrogen peroxide vapor (HPV). This article is unique because it describes the HPV process in an operating room, and is therefore, a deployable method for many healthcare settings. Results presented here offer creative solutions to the current PPE shortage.

Reinforcement Learning: Full Glass or Empty - Depends Who You Ask.

An extension of the prediction error theory of dopamine, imported from artificial intelligence, represents the full distribution over future rewards rather than only the average and better explains dopamine responses.

Control of entropy in neural models of environmental state.

Humans and animals construct internal models of their environment in order to select appropriate courses of action. The representation of uncertainty about the current state of the environment is a key feature of these models that controls the rate of learning as well as directly affecting choice behaviour. To maintain flexibility, given that uncertainty naturally decreases over time, most theoretical inference models include a dedicated mechanism to drive up model uncertainty. Here we probe the long-standing hypothesis that noradrenaline is involved in determining the uncertainty, or entropy, and thus flexibility, of neural models. Pupil diameter, which indexes neuromodulatory state including noradrenaline release, predicted increases (but not decreases) in entropy in a neural state model encoded in human medial orbitofrontal cortex, as measured using multivariate functional MRI. Activity in anterior cingulate cortex predicted pupil diameter. These results provide evidence for top-down, neuromodulatory control of entropy in neural state models.

What Is a Cognitive Map? Organizing Knowledge for Flexible Behavior.

It is proposed that a cognitive map encoding the relationships between entities in the world supports flexible behavior, but the majority of the neural evidence for such a system comes from studies of spatial navigation. Recent work describing neuronal parallels between spatial and non-spatial behaviors has rekindled the notion of a systematic organization of knowledge across multiple domains. We review experimental evidence and theoretical frameworks that point to principles unifying these apparently disparate functions. These principles describe how to learn and use abstract, generalizable knowledge and suggest that map-like representations observed in a spatial context may be an instance of general coding mechanisms capable of organizing knowledge of all kinds. We highlight how artificial agents endowed with such principles exhibit flexible behavior and learn map-like representations observed in the brain. Finally, we speculate on how these principles may offer insight into the extreme generalizations, abstractions, and inferences that characterize human cognition.

Detection of Pathogen Exposure in African Buffalo Using Non-Specific Markers of Inflammation.

Detecting exposure to new or emerging pathogens is a critical challenge to protecting human, domestic animal, and wildlife health. Yet, current techniques to detect infections typically target known pathogens of humans or economically important animals. In the face of the current surge in infectious disease emergence, non-specific disease surveillance tools are urgently needed. Tracking common host immune responses indicative of recent infection may have potential as a non-specific diagnostic approach for disease surveillance. The challenge to immunologists is to identify the most promising markers, which ideally should be highly conserved across pathogens and host species, become upregulated rapidly and consistently in response to pathogen invasion, and remain elevated beyond clearance of infection. This study combined an infection experiment and a longitudinal observational study to evaluate the utility of non-specific markers of inflammation [NSMI; two acute phase proteins (haptoglobin and serum amyloid A), two pro-inflammatory cytokines (IFNγ and TNF-α)] as indicators of pathogen exposure in a wild mammalian species, African buffalo (Syncerus caffer). Specifically, in the experimental study, we asked (1) How quickly do buffalo mount NSMI responses upon challenge with an endemic pathogen, foot-and-mouth disease virus; (2) for how long do NSMI remain elevated after viral clearance and; (3) how pronounced is the difference between peak NSMI concentration and baseline NSMI concentration? In the longitudinal study, we asked (4) Are elevated NSMI associated with recent exposure to a suite of bacterial and viral respiratory pathogens in a wild population? Among the four NSMI that we tested, haptoglobin showed the strongest potential as a surveillance marker in African buffalo: concentrations quickly and consistently reached high levels in response to experimental infection, remaining elevated for almost a month. Moreover, elevated haptoglobin was indicative of recent exposure to two respiratory pathogens assessed in the longitudinal study. We hope this work motivates studies investigating suites of NSMI as indicators for pathogen exposure in a broader range of both pathogen and host species, potentially transforming how we track disease burden in natural populations.

Dual processing of visual rotation for bipedal stance control.

KEY POINTS: When standing, the gain of the body-movement response to a sinusoidally moving visual scene has been shown to get smaller with faster stimuli, possibly through changes in the apportioning of visual flow to self-motion or environment motion. We investigated whether visual-flow speed similarly influences the postural response to a discrete, unidirectional rotation of the visual scene in the frontal plane. Contrary to expectation, the evoked postural response consisted of two sequential components with opposite relationships to visual motion speed. With faster visual rotation the early component became smaller, not through a change in gain but by changes in its temporal structure, while the later component grew larger. We propose that the early component arises from the balance control system minimising apparent self-motion, while the later component stems from the postural system realigning the body with gravity. ABSTRACT: The source of visual motion is inherently ambiguous such that movement of objects in the environment can evoke self-motion illusions and postural adjustments. Theoretically, the brain can mitigate this problem by combining visual signals with other types of information. A Bayesian model that achieves this was previously proposed and predicts a decreasing gain of postural response with increasing visual motion speed. Here we test this prediction for discrete, unidirectional, full-field visual rotations in the frontal plane of standing subjects. The speed (0.75-48 deg s(-1) ) and direction of visual rotation was pseudo-randomly varied and mediolateral responses were measured from displacements of the trunk and horizontal ground reaction forces. The behaviour evoked by this visual rotation was more complex than has hitherto been reported, consisting broadly of two consecutive components with respective latencies of ∼190 ms and >0.7 s. Both components were sensitive to visual rotation speed, but with diametrically opposite relationships. Thus, the early component decreased with faster visual rotation, while the later component increased. Furthermore, the decrease in size of the early component was not achieved by a simple attenuation of gain, but by a change in its temporal structure. We conclude that the two components represent expressions of different motor functions, both pertinent to the control of bipedal stance. We propose that the early response stems from the balance control system attempting to minimise unintended body motion, while the later response arises from the postural control system attempting to align the body with gravity.

Reward activates stimulus-specific and task-dependent representations in visual association cortices.

Humans reliably learn which actions lead to rewards. One prominent question is how credit is assigned to environmental stimuli that are acted upon. Recent functional magnetic resonance imaging (fMRI) studies have provided evidence that representations of rewarded stimuli are activated upon reward delivery, providing possible eligibility traces for credit assignment. Our study sought evidence of postreward activation in sensory cortices satisfying two conditions of instrumental learning: postreward activity should reflect the stimulus category that preceded reward (stimulus specificity), and should occur only if the stimulus was acted on to obtain reward (task dependency). Our experiment implemented two tasks in the fMRI scanner. The first was a perceptual decision-making task on degraded face and house stimuli. Stimulus specificity was evident as rewards activated the sensory cortices associated with face versus house perception more strongly after face versus house decisions, respectively, particularly in the fusiform face area. Stimulus specificity was further evident in a psychophysiological interaction analysis wherein face-sensitive areas correlated with nucleus accumbens activity after face-decision rewards, whereas house-sensitive areas correlated with nucleus accumbens activity after house-decision rewards. The second task required participants to make an instructed response. The criterion of task dependency was fulfilled as rewards after face versus house responses activated the respective association cortices to a larger degree when faces and houses were relevant to the performed task. Our study is the first to show that postreward sensory cortex activity meets these two key criteria of credit assignment, and does so independently from bottom-up perceptual processing.