Emergent neural dynamics and geometry for generalization in a transitive inference task.

Relational cognition-the ability to infer relationships that generalize to novel combinations of objects-is fundamental to human and animal intelligence. Despite this importance, it remains unclear how relational cognition is implemented in the brain due in part to a lack of hypotheses and predictions at the levels of collective neural activity and behavior. Here we discovered, analyzed, and experimentally tested neural networks (NNs) that perform transitive inference (TI), a classic relational task (if A > B and B > C, then A > C). We found NNs that (i) generalized perfectly, despite lacking overt transitive structure prior to training, (ii) generalized when the task required working memory (WM), a capacity thought to be essential to inference in the brain, (iii) emergently expressed behaviors long observed in living subjects, in addition to a novel order-dependent behavior, and (iv) expressed different task solutions yielding alternative behavioral and neural predictions. Further, in a large-scale experiment, we found that human subjects performing WM-based TI showed behavior inconsistent with a class of NNs that characteristically expressed an intuitive task solution. These findings provide neural insights into a classical relational ability, with wider implications for how the brain realizes relational cognition.

Low-dimensional dynamics for working memory and time encoding.

Our decisions often depend on multiple sensory experiences separated by time delays. The brain can remember these experiences and, simultaneously, estimate the timing between events. To understand the mechanisms underlying working memory and time encoding, we analyze neural activity recorded during delays in four experiments on nonhuman primates. To disambiguate potential mechanisms, we propose two analyses, namely, decoding the passage of time from neural data and computing the cumulative dimensionality of the neural trajectory over time. Time can be decoded with high precision in tasks where timing information is relevant and with lower precision when irrelevant for performing the task. Neural trajectories are always observed to be low-dimensional. In addition, our results further constrain the mechanisms underlying time encoding as we find that the linear "ramping" component of each neuron's firing rate strongly contributes to the slow timescale variations that make decoding time possible. These constraints rule out working memory models that rely on constant, sustained activity and neural networks with high-dimensional trajectories, like reservoir networks. Instead, recurrent networks trained with backpropagation capture the time-encoding properties and the dimensionality observed in the data.

Visual perception as retrospective Bayesian decoding from high- to low-level features.

When a stimulus is presented, its encoding is known to progress from low- to high-level features. How these features are decoded to produce perception is less clear, and most models assume that decoding follows the same low- to high-level hierarchy of encoding. There are also theories arguing for global precedence, reversed hierarchy, or bidirectional processing, but they are descriptive without quantitative comparison with human perception. Moreover, observers often inspect different parts of a scene sequentially to form overall perception, suggesting that perceptual decoding requires working memory, yet few models consider how working-memory properties may affect decoding hierarchy. We probed decoding hierarchy by comparing absolute judgments of single orientations and relative/ordinal judgments between two sequentially presented orientations. We found that lower-level, absolute judgments failed to account for higher-level, relative/ordinal judgments. However, when ordinal judgment was used to retrospectively decode memory representations of absolute orientations, striking aspects of absolute judgments, including the correlation and forward/backward aftereffects between two reported orientations in a trial, were explained. We propose that the brain prioritizes decoding of higher-level features because they are more behaviorally relevant, and more invariant and categorical, and thus easier to specify and maintain in noisy working memory, and that more reliable higher-level decoding constrains less reliable lower-level decoding.

full-FORCE: A target-based method for training recurrent networks.

Trained recurrent networks are powerful tools for modeling dynamic neural computations. We present a target-based method for modifying the full connectivity matrix of a recurrent network to train it to perform tasks involving temporally complex input/output transformations. The method introduces a second network during training to provide suitable "target" dynamics useful for performing the task. Because it exploits the full recurrent connectivity, the method produces networks that perform tasks with fewer neurons and greater noise robustness than traditional least-squares (FORCE) approaches. In addition, we show how introducing additional input signals into the target-generating network, which act as task hints, greatly extends the range of tasks that can be learned and provides control over the complexity and nature of the dynamics of the trained, task-performing network.

Natural grouping of neural responses reveals spatially segregated clusters in prearcuate cortex.

A fundamental challenge in studying the frontal lobe is to parcellate this cortex into "natural" functional modules despite the absence of topographic maps, which are so helpful in primary sensory areas. Here we show that unsupervised clustering algorithms, applied to 96-channel array recordings from prearcuate gyrus, reveal spatially segregated subnetworks that remain stable across behavioral contexts. Looking for natural groupings of neurons based on response similarities, we discovered that the recorded area includes at least two spatially segregated subnetworks that differentially represent behavioral choice and reaction time. Importantly, these subnetworks are detectable during different behavioral states and, surprisingly, are defined better by "common noise" than task-evoked responses. Our parcellation process works well on "spontaneous" neural activity, and thus bears strong resemblance to the identification of "resting-state" networks in fMRI data sets. Our results demonstrate a powerful new tool for identifying cortical subnetworks by objective classification of simultaneously recorded electrophysiological activity.

Dynamics of neural population responses in prefrontal cortex indicate changes of mind on single trials.

Decision making is a complex process in which different sources of information are combined into a decision variable (DV) that guides action [1, 2]. Neurophysiological studies have typically sought insight into the dynamics of the decision-making process and its neural mechanisms through statistical analysis of large numbers of trials from sequentially recorded single neurons or small groups of neurons [3-6]. However, detecting and analyzing the DV on individual trials has been challenging [7]. Here we show that by recording simultaneously from hundreds of units in prearcuate gyrus of macaque monkeys performing a direction discrimination task, we can predict the monkey's choices with high accuracy and decode DV dynamically as the decision unfolds on individual trials. This advance enabled us to study changes of mind (CoMs) that occasionally happen before the final commitment to a decision [8-10]. On individual trials, the decoded DV varied significantly over time and occasionally changed its sign, identifying a potential CoM. Interrogating the system by random stopping of the decision-making process during the delay period after stimulus presentation confirmed the validity of identified CoMs. Importantly, the properties of the candidate CoMs also conformed to expectations based on prior theoretical and behavioral studies [8]: they were more likely to go from an incorrect to a correct choice, they were more likely for weak and intermediate stimuli than for strong stimuli, and they were more likely earlier in the trial. We suggest that simultaneous recording of large neural populations provides a good estimate of DV and explains idiosyncratic aspects of the decision-making process that were inaccessible before.