Multiple traces and altered signal-to-noise in systems consolidation: Evidence from the 7T fMRI Natural Scenes Dataset.

The brain mechanisms of memory consolidation remain elusive. Here, we examine blood-oxygen-level-dependent (BOLD) correlates of image recognition through the scope of multiple influential systems consolidation theories. We utilize the longitudinal Natural Scenes Dataset, a 7-Tesla functional magnetic resonance imaging human study in which ∼135,000 trials of image recognition were conducted over the span of a year among eight subjects. We find that early- and late-stage image recognition associates with both medial temporal lobe (MTL) and visual cortex when evaluating regional activations and a multivariate classifier. Supporting multiple-trace theory (MTT), parts of the MTL activation time course show remarkable fit to a 20-y-old MTT time-dynamical model predicting early trace intensity increases and slight subsequent interference (R2 > 0.90). These findings contrast a simplistic, yet common, view that memory traces are transferred from MTL to cortex. Next, we test the hypothesis that the MTL trace signature of memory consolidation should also reflect synaptic "desaturation," as evidenced by an increased signal-to-noise ratio. We find that the magnitude of relative BOLD enhancement among surviving memories is positively linked to the rate of removal (i.e., forgetting) of competing traces. Moreover, an image-feature and time interaction of MTL and visual cortex functional connectivity suggests that consolidation mechanisms improve the specificity of a distributed trace. These neurobiological effects do not replicate on a shorter timescale (within a session), implicating a prolonged, offline process. While recognition can potentially involve cognitive processes outside of memory retrieval (e.g., re-encoding), our work largely favors MTT and desaturation as perhaps complementary consolidative memory mechanisms.

NeuroGen: Activation optimized image synthesis for discovery neuroscience.

Functional MRI (fMRI) is a powerful technique that has allowed us to characterize visual cortex responses to stimuli, yet such experiments are by nature constructed based on a priori hypotheses, limited to the set of images presented to the individual while they are in the scanner, are subject to noise in the observed brain responses, and may vary widely across individuals. In this work, we propose a novel computational strategy, which we call NeuroGen, to overcome these limitations and develop a powerful tool for human vision neuroscience discovery. NeuroGen combines an fMRI-trained neural encoding model of human vision with a deep generative network to synthesize images predicted to achieve a target pattern of macro-scale brain activation. We demonstrate that the reduction of noise that the encoding model provides, coupled with the generative network's ability to produce images of high fidelity, results in a robust discovery architecture for visual neuroscience. By using only a small number of synthetic images created by NeuroGen, we demonstrate that we can detect and amplify differences in regional and individual human brain response patterns to visual stimuli. We then verify that these discoveries are reflected in the several thousand observed image responses measured with fMRI. We further demonstrate that NeuroGen can create synthetic images predicted to achieve regional response patterns not achievable by the best-matching natural images. The NeuroGen framework extends the utility of brain encoding models and opens up a new avenue for exploring, and possibly precisely controlling, the human visual system.

Spectral Encoding of Seen and Attended Object Categories in the Human Brain.

Local field potentials (LFPs) encode visual information via variations in power at many frequencies. These variations are complex and depend on stimulus and cognitive state in ways that have yet to be fully characterized. Specifically, the frequencies (or combinations of frequencies) that most robustly encode specific types of visual information are not fully known. To address this knowledge gap, we used intracranial EEG to record LFPs at 858 widely distributed recording sites as human subjects (six males, five females) indicated whether briefly presented natural scenes depicted one of three attended object categories. Principal component analysis applied to power spectra of the LFPs near stimulus onset revealed a broadband component (1-100 Hz) and two narrowband components (1-8 and 8-30 Hz, respectively) that encoded information about both seen and attended categories. Interestingly, we found that seen and attended categories were not encoded with the same fidelity by these distinct spectral components. Model-based tuning and decoding analyses revealed that power variations along the broadband component were most sharply tuned and offered more accurate decoding for seen than for attended categories. Power along the narrowband delta-theta (1-8 Hz) component robustly decoded information about both seen and attended categories, while the alpha-beta (8-30 Hz) component was specialized for attention. We conclude that, when viewing natural scenes, information about the seen category is encoded via broadband and sub-gamma (<30 Hz) power variations, while the attended category is most robustly encoded in the sub-gamma range. More generally, these results suggest that power variation along different spectral components can encode qualitatively different kinds of visual information.SIGNIFICANCE STATEMENT In this article, we characterize how changes in visual stimuli depicting specific objects (cars, faces, and buildings) and changes in attention to those objects affect the frequency content of local field potentials in the human brain. In contrast to many previous studies that have investigated encoding by variations in power at high (>30 Hz) frequencies, we find that the most important variation patterns are broadband (i.e., distributed across multiple frequencies) and narrowband, but in lower frequencies (<30 Hz). Interestingly, we find that seen and attended categories are not encoded with the same fidelity by these distinct spectral encoding patterns, suggesting that power at different frequencies can encode qualitatively different kinds of information.

Voxel-to-voxel predictive models reveal unexpected structure in unexplained variance.

Encoding models based on deep convolutional neural networks (DCNN) predict BOLD responses to natural scenes in the human visual system more accurately than many other currently available models. However, DCNN-based encoding models fail to predict a significant amount of variance in the activity of most voxels in all visual areas. This failure could reflect limitations in the data (e.g., a noise ceiling), or could reflect limitations of the DCNN as a model of computation in the brain. Understanding the source and structure of the unexplained variance could therefore provide helpful clues for improving models of brain computation. Here, we characterize the structure of the variance that DCNN-based encoding models cannot explain. Using a publicly available dataset of BOLD responses to natural scenes, we determined if the source of unexplained variance was shared across voxels, individual brains, retinotopic locations, and hierarchically distant visual brain areas. We answered these questions using voxel-to-voxel (vox2vox) models that predict activity in a target voxel given activity in a population of source voxels. We found that simple linear vox2vox models increased within-subject prediction accuracy over DCNN-based models for any pair of source/target visual areas, clearly demonstrating that the source of unexplained variance is widely shared within and across visual brain areas. However, vox2vox models were not more accurate than DCNN-based encoding models when source and target voxels came from different brains, demonstrating that the source of unexplained variance was not shared across brains. Importantly, control analyses demonstrated that the source of unexplained variance was not encoded in the mean activity of source voxels, or the activity of voxels in white matter. Interestingly, the weights of vox2vox models revealed preferential connection of target voxel activity to source voxels with adjacent receptive fields, even when source and target voxels were in different functional brain areas. Finally, we found that the prediction accuracy of the vox2vox models decayed with hierarchical distance between the source and target voxels but showed detailed patterns of dependence on hierarchical relationships that we did not observe in DCNNs. Given these results, we argue that the structured variance unexplained by DCNN-based encoding models is unlikely to be entirely caused by non-neural artifacts (e.g., spatially correlated measurement noise) or a failure of DCNNs to approximate the features encoded in brain activity; rather, our results point to a need for brain models that provide both mechanistic and computational explanations for structured ongoing activity in the brain. Keywords: fMRI, encoding models, deep neural networks, functional connectivity.

The feature-weighted receptive field: an interpretable encoding model for complex feature spaces.

We introduce the feature-weighted receptive field (fwRF), an encoding model designed to balance expressiveness, interpretability and scalability. The fwRF is organized around the notion of a feature map-a transformation of visual stimuli into visual features that preserves the topology of visual space (but not necessarily the native resolution of the stimulus). The key assumption of the fwRF model is that activity in each voxel encodes variation in a spatially localized region across multiple feature maps. This region is fixed for all feature maps; however, the contribution of each feature map to voxel activity is weighted. Thus, the model has two separable sets of parameters: "where" parameters that characterize the location and extent of pooling over visual features, and "what" parameters that characterize tuning to visual features. The "where" parameters are analogous to classical receptive fields, while "what" parameters are analogous to classical tuning functions. By treating these as separable parameters, the fwRF model complexity is independent of the resolution of the underlying feature maps. This makes it possible to estimate models with thousands of high-resolution feature maps from relatively small amounts of data. Once a fwRF model has been estimated from data, spatial pooling and feature tuning can be read-off directly with no (or very little) additional post-processing or in-silico experimentation. We describe an optimization algorithm for estimating fwRF models from data acquired during standard visual neuroimaging experiments. We then demonstrate the model's application to two distinct sets of features: Gabor wavelets and features supplied by a deep convolutional neural network. We show that when Gabor feature maps are used, the fwRF model recovers receptive fields and spatial frequency tuning functions consistent with known organizational principles of the visual cortex. We also show that a fwRF model can be used to regress entire deep convolutional networks against brain activity. The ability to use whole networks in a single encoding model yields state-of-the-art prediction accuracy. Our results suggest a wide variety of uses for the feature-weighted receptive field model, from retinotopic mapping with natural scenes, to regressing the activities of whole deep neural networks onto measured brain activity.

A voxel-wise encoding model for early visual areas decodes mental images of remembered scenes.

Recent multi-voxel pattern classification (MVPC) studies have shown that in early visual cortex patterns of brain activity generated during mental imagery are similar to patterns of activity generated during perception. This finding implies that low-level visual features (e.g., space, spatial frequency, and orientation) are encoded during mental imagery. However, the specific hypothesis that low-level visual features are encoded during mental imagery is difficult to directly test using MVPC. The difficulty is especially acute when considering the representation of complex, multi-object scenes that can evoke multiple sources of variation that are distinct from low-level visual features. Therefore, we used a voxel-wise modeling and decoding approach to directly test the hypothesis that low-level visual features are encoded in activity generated during mental imagery of complex scenes. Using fMRI measurements of cortical activity evoked by viewing photographs, we constructed voxel-wise encoding models of tuning to low-level visual features. We also measured activity as subjects imagined previously memorized works of art. We then used the encoding models to determine if putative low-level visual features encoded in this activity could pick out the imagined artwork from among thousands of other randomly selected images. We show that mental images can be accurately identified in this way; moreover, mental image identification accuracy depends upon the degree of tuning to low-level visual features in the voxels selected for decoding. These results directly confirm the hypothesis that low-level visual features are encoded during mental imagery of complex scenes. Our work also points to novel forms of brain-machine interaction: we provide a proof-of-concept demonstration of an internet image search guided by mental imagery.

Identifying natural images from human brain activity.

A challenging goal in neuroscience is to be able to read out, or decode, mental content from brain activity. Recent functional magnetic resonance imaging (fMRI) studies have decoded orientation, position and object category from activity in visual cortex. However, these studies typically used relatively simple stimuli (for example, gratings) or images drawn from fixed categories (for example, faces, houses), and decoding was based on previous measurements of brain activity evoked by those same stimuli or categories. To overcome these limitations, here we develop a decoding method based on quantitative receptive-field models that characterize the relationship between visual stimuli and fMRI activity in early visual areas. These models describe the tuning of individual voxels for space, orientation and spatial frequency, and are estimated directly from responses evoked by natural images. We show that these receptive-field models make it possible to identify, from a large set of completely novel natural images, which specific image was seen by an observer. Identification is not a mere consequence of the retinotopic organization of visual areas; simpler receptive-field models that describe only spatial tuning yield much poorer identification performance. Our results suggest that it may soon be possible to reconstruct a picture of a person's visual experience from measurements of brain activity alone.

Disentangling signal and noise in neural responses through generative modeling.

Measurements of neural responses to identically repeated experimental events often exhibit large amounts of variability. This noise is distinct from signal, operationally defined as the average expected response across repeated trials for each given event. Accurately distinguishing signal from noise is important, as each is a target that is worthy of study (many believe noise reflects important aspects of brain function) and it is important not to confuse one for the other. Here, we introduce a principled modeling approach in which response measurements are explicitly modeled as the sum of samples from multivariate signal and noise distributions. In our proposed method-termed Generative Modeling of Signal and Noise (GSN)-the signal distribution is estimated by subtracting the estimated noise distribution from the estimated data distribution. We validate GSN using ground-truth simulations and demonstrate the application of GSN to empirical fMRI data. In doing so, we illustrate a simple consequence of GSN: by disentangling signal and noise components in neural responses, GSN denoises principal components analysis and improves estimates of dimensionality. We end by discussing other situations that may benefit from GSN's characterization of signal and noise, such as estimation of noise ceilings for computational models of neural activity. A code toolbox for GSN is provided with both MATLAB and Python implementations.

How does the primate brain combine generative and discriminative computations in vision?

Vision is widely understood as an inference problem. However, two contrasting conceptions of the inference process have each been influential in research on biological vision as well as the engineering of machine vision. The first emphasizes bottom-up signal flow, describing vision as a largely feedforward, discriminative inference process that filters and transforms the visual information to remove irrelevant variation and represent behaviorally relevant information in a format suitable for downstream functions of cognition and behavioral control. In this conception, vision is driven by the sensory data, and perception is direct because the processing proceeds from the data to the latent variables of interest. The notion of "inference" in this conception is that of the engineering literature on neural networks, where feedforward convolutional neural networks processing images are said to perform inference. The alternative conception is that of vision as an inference process in Helmholtz's sense, where the sensory evidence is evaluated in the context of a generative model of the causal processes that give rise to it. In this conception, vision inverts a generative model through an interrogation of the sensory evidence in a process often thought to involve top-down predictions of sensory data to evaluate the likelihood of alternative hypotheses. The authors include scientists rooted in roughly equal numbers in each of the conceptions and motivated to overcome what might be a false dichotomy between them and engage the other perspective in the realm of theory and experiment. The primate brain employs an unknown algorithm that may combine the advantages of both conceptions. We explain and clarify the terminology, review the key empirical evidence, and propose an empirical research program that transcends the dichotomy and sets the stage for revealing the mysterious hybrid algorithm of primate vision.

Second Sight: Using brain-optimized encoding models to align image distributions with human brain activity.

Two recent developments have accelerated progress in image reconstruction from human brain activity: large datasets that offer samples of brain activity in response to many thousands of natural scenes, and the open-sourcing of powerful stochastic image-generators that accept both low- and high-level guidance. Most work in this space has focused on obtaining point estimates of the target image, with the ultimate goal of approximating literal pixel-wise reconstructions of target images from the brain activity patterns they evoke. This emphasis belies the fact that there is always a family of images that are equally compatible with any evoked brain activity pattern, and the fact that many image-generators are inherently stochastic and do not by themselves offer a method for selecting the single best reconstruction from among the samples they generate. We introduce a novel reconstruction procedure (Second Sight) that iteratively refines an image distribution to explicitly maximize the alignment between the predictions of a voxel-wise encoding model and the brain activity patterns evoked by any target image. We use an ensemble of brain-optimized deep neural networks trained on the Natural Scenes Dataset (NSD) as our encoding model, and a latent diffusion model as our image generator. At each iteration, we generate a small library of images and select those that best approximate the measured brain activity when passed through our encoding model. We extract semantic and structural guidance from the selected images, used for generating the next library. We show that this process converges on a distribution of high-quality reconstructions by refining both semantic content and low-level image details across iterations. Images sampled from these converged image distributions are competitive with state-of-the-art reconstruction algorithms. Interestingly, the time-to-convergence varies systematically across visual cortex, with earlier visual areas generally taking longer and converging on narrower image distributions, relative to higher-level brain areas. Second Sight thus offers a succinct and novel method for exploring the diversity of representations across visual brain areas.

Reconstructing seen images from human brain activity via guided stochastic search.

Visual reconstruction algorithms are an interpretive tool that map brain activity to pixels. Past reconstruction algorithms employed brute-force search through a massive library to select candidate images that, when passed through an encoding model, accurately predict brain activity. Here, we use conditional generative diffusion models to extend and improve this search-based strategy. We decode a semantic descriptor from human brain activity (7T fMRI) in voxels across most of visual cortex, then use a diffusion model to sample a small library of images conditioned on this descriptor. We pass each sample through an encoding model, select the images that best predict brain activity, and then use these images to seed another library. We show that this process converges on high-quality reconstructions by refining low-level image details while preserving semantic content across iterations. Interestingly, the time-to-convergence differs systematically across visual cortex, suggesting a succinct new way to measure the diversity of representations across visual brain areas.

Stacked regressions and structured variance partitioning for interpretable brain maps.

Relating brain activity associated with a complex stimulus to different properties of that stimulus is a powerful approach for constructing functional brain maps. However, when stimuli are naturalistic, their properties are often correlated (e.g., visual and semantic features of natural images, or different layers of a convolutional neural network that are used as features of images). Correlated properties can act as confounders for each other and complicate the interpretability of brain maps, and can impact the robustness of statistical estimators. Here, we present an approach for brain mapping based on two proposed methods: stacking different encoding models and structured variance partitioning. Our stacking algorithm combines encoding models that each use as input a feature space that describes a different stimulus attribute. The algorithm learns to predict the activity of a voxel as a linear combination of the outputs of different encoding models. We show that the resulting combined model can predict held-out brain activity better or at least as well as the individual encoding models. Further, the weights of the linear combination are readily interpretable; they show the importance of each feature space for predicting a voxel. We then build on our stacking models to introduce structured variance partitioning, a new type of variance partitioning that takes into account the known relationships between features. Our approach constrains the size of the hypothesis space and allows us to ask targeted questions about the similarity between feature spaces and brain regions even in the presence of correlations between the feature spaces. We validate our approach in simulation, showcase its brain mapping potential on fMRI data, and release a Python package. Our methods can be useful for researchers interested in aligning brain activity with different layers of a neural network, or with other types of correlated feature spaces.

Brain-optimized inference improves reconstructions of fMRI brain activity.

The release of large datasets and developments in AI have led to dramatic improvements in decoding methods that reconstruct seen images from human brain activity. We evaluate the prospect of further improving recent decoding methods by optimizing for consistency between reconstructions and brain activity during inference. We sample seed reconstructions from a base decoding method, then iteratively refine these reconstructions using a brain-optimized encoding model that maps images to brain activity. At each iteration, we sample a small library of images from an image distribution (a diffusion model) conditioned on a seed reconstruction from the previous iteration. We select those that best approximate the measured brain activity when passed through our encoding model, and use these images for structural guidance during the generation of the small library in the next iteration. We reduce the stochasticity of the image distribution at each iteration, and stop when a criterion on the "width" of the image distribution is met. We show that when this process is applied to recent decoding methods, it outperforms the base decoding method as measured by human raters, a variety of image feature metrics, and alignment to brain activity. These results demonstrate that reconstruction quality can be significantly improved by explicitly aligning decoding distributions to brain activity distributions, even when the seed reconstruction is output from a state-of-the-art decoding algorithm. Interestingly, the rate of refinement varies systematically across visual cortex, with earlier visual areas generally converging more slowly and preferring narrower image distributions, relative to higher-level brain areas. Brain-optimized inference thus offers a succinct and novel method for improving reconstructions and exploring the diversity of representations across visual brain areas.

Brain-optimized deep neural network models of human visual areas learn non-hierarchical representations.

Deep neural networks (DNNs) optimized for visual tasks learn representations that align layer depth with the hierarchy of visual areas in the primate brain. One interpretation of this finding is that hierarchical representations are necessary to accurately predict brain activity in the primate visual system. To test this interpretation, we optimized DNNs to directly predict brain activity measured with fMRI in human visual areas V1-V4. We trained a single-branch DNN to predict activity in all four visual areas jointly, and a multi-branch DNN to predict each visual area independently. Although it was possible for the multi-branch DNN to learn hierarchical representations, only the single-branch DNN did so. This result shows that hierarchical representations are not necessary to accurately predict human brain activity in V1-V4, and that DNNs that encode brain-like visual representations may differ widely in their architecture, ranging from strict serial hierarchies to multiple independent branches.

Re-expression of CA1 and entorhinal activity patterns preserves temporal context memory at long timescales.

Converging, cross-species evidence indicates that memory for time is supported by hippocampal area CA1 and entorhinal cortex. However, limited evidence characterizes how these regions preserve temporal memories over long timescales (e.g., months). At long timescales, memoranda may be encountered in multiple temporal contexts, potentially creating interference. Here, using 7T fMRI, we measured CA1 and entorhinal activity patterns as human participants viewed thousands of natural scene images distributed, and repeated, across many months. We show that memory for an image's original temporal context was predicted by the degree to which CA1/entorhinal activity patterns from the first encounter with an image were re-expressed during re-encounters occurring minutes to months later. Critically, temporal memory signals were dissociable from predictors of recognition confidence, which were carried by distinct medial temporal lobe expressions. These findings suggest that CA1 and entorhinal cortex preserve temporal memories across long timescales by coding for and reinstating temporal context information.

Color-biased regions in the ventral visual pathway are food selective.

Color-biased regions have been found between face- and place-selective areas in the ventral visual pathway. To investigate the function of the color-biased regions in a pathway responsible for object recognition, we analyzed the natural scenes dataset (NSD), a large 7T fMRI dataset from 8 participants who each viewed up to 30,000 trials of images of colored natural scenes over more than 30 scanning sessions. In a whole-brain analysis, we correlated the average color saturation of the images with voxel responses, revealing color-biased regions that diverge into two streams, beginning in V4 and extending medially and laterally relative to the fusiform face area in both hemispheres. We drew regions of interest (ROIs) for the two streams and found that the images for each ROI that evoked the largest responses had certain characteristics: they contained food, circular objects, warmer hues, and had higher color saturation. Further analyses showed that food images were the strongest predictor of activity in these regions, implying the existence of medial and lateral ventral food streams (VFSs). We found that color also contributed independently to voxel responses, suggesting that the medial and lateral VFSs use both color and form to represent food. Our findings illustrate how high-resolution datasets such as the NSD can be used to disentangle the multifaceted contributions of many visual features to the neural representations of natural scenes.

Modulation of Spectral Representation and Connectivity Patterns in Response to Visual Narrative in the Human Brain.

The regional brain networks and the underlying neurophysiological mechanisms subserving the cognition of visual narrative in humans have largely been studied with non-invasive brain recording. In this study, we specifically investigated how regional and cross-regional cortical activities support visual narrative interpretation using intracranial stereotactic electroencephalograms recordings from thirteen human subjects (6 females, and 7 males). Widely distributed recording sites across the brain were sampled while subjects were explicitly instructed to observe images from fables presented in "sequential" order, and a set of images drawn from multiple fables presented in "scrambled" order. Broadband activity mainly within the frontal and temporal lobes were found to encode if a presented image is part of a visual narrative (sequential) or random image set (scrambled). Moreover, the temporal lobe exhibits strong activation in response to visual narratives while the frontal lobe is more engaged when contextually novel stimuli are presented. We also investigated the dynamics of interregional interactions between visual narratives and contextually novel series of images. Interestingly, the interregional connectivity is also altered between sequential and scrambled sequences. Together, these results suggest that both changes in regional neuronal activity and cross-regional interactions subserve visual narrative and contextual novelty processing.

A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence.

Extensive sampling of neural activity during rich cognitive phenomena is critical for robust understanding of brain function. Here we present the Natural Scenes Dataset (NSD), in which high-resolution functional magnetic resonance imaging responses to tens of thousands of richly annotated natural scenes were measured while participants performed a continuous recognition task. To optimize data quality, we developed and applied novel estimation and denoising techniques. Simple visual inspections of the NSD data reveal clear representational transformations along the ventral visual pathway. Further exemplifying the inferential power of the dataset, we used NSD to build and train deep neural network models that predict brain activity more accurately than state-of-the-art models from computer vision. NSD also includes substantial resting-state and diffusion data, enabling network neuroscience perspectives to constrain and enhance models of perception and memory. Given its unprecedented scale, quality and breadth, NSD opens new avenues of inquiry in cognitive neuroscience and artificial intelligence.

Encoding and decoding in fMRI.

Over the past decade fMRI researchers have developed increasingly sensitive techniques for analyzing the information represented in BOLD activity. The most popular of these techniques is linear classification, a simple technique for decoding information about experimental stimuli or tasks from patterns of activity across an array of voxels. A more recent development is the voxel-based encoding model, which describes the information about the stimulus or task that is represented in the activity of single voxels. Encoding and decoding are complementary operations: encoding uses stimuli to predict activity while decoding uses activity to predict information about the stimuli. However, in practice these two operations are often confused, and their respective strengths and weaknesses have not been made clear. Here we use the concept of a linearizing feature space to clarify the relationship between encoding and decoding. We show that encoding and decoding operations can both be used to investigate some of the most common questions about how information is represented in the brain. However, focusing on encoding models offers two important advantages over decoding. First, an encoding model can in principle provide a complete functional description of a region of interest, while a decoding model can provide only a partial description. Second, while it is straightforward to derive an optimal decoding model from an encoding model it is much more difficult to derive an encoding model from a decoding model. We propose a systematic modeling approach that begins by estimating an encoding model for every voxel in a scan and ends by using the estimated encoding models to perform decoding.

Mapping of the preferred direction in the motor cortex.

Directional tuning is a basic functional property of cell activity in the motor cortex. Previous work has indicated that cells with similar preferred directions are organized in columns perpendicular to the cortical surface. Here we show that these columns are organized in an orderly fashion in the tangential dimension on the cortical surface. Based on a large number of microelectrode penetrations and systematic exploration of the proximal arm area of the motor cortex while monkeys made free reaching 3D movements, it was estimated that (i) directional minicolumns are approximately 30 mum in width, (ii) minicolumns with similar preferred directions tend to occur in doublets or triplets, and (iii) such minicolumns tend to repeat every approximately 240 mum (estimated width of a column), with intermediate preferred directions represented in a gradient. These findings provide evidence for an orderly mapping of the preferred direction in the motor cortex.