The Dorsal Visual Pathway Represents Object-Centered Spatial Relations for Object Recognition.

Although there is mounting evidence that input from the dorsal visual pathway is crucial for object processes in the ventral pathway, the specific functional contributions of dorsal cortex to these processes remain poorly understood. Here, we hypothesized that dorsal cortex computes the spatial relations among an object's parts, a process crucial for forming global shape percepts, and transmits this information to the ventral pathway to support object categorization. Using fMRI with human participants (females and males), we discovered regions in the intraparietal sulcus (IPS) that were selectively involved in computing object-centered part relations. These regions exhibited task-dependent functional and effective connectivity with ventral cortex, and were distinct from other dorsal regions, such as those representing allocentric relations, 3D shape, and tools. In a subsequent experiment, we found that the multivariate response of posterior (p)IPS, defined on the basis of part-relations, could be used to decode object category at levels comparable to ventral object regions. Moreover, mediation and multivariate effective connectivity analyses further suggested that IPS may account for representations of part relations in the ventral pathway. Together, our results highlight specific contributions of the dorsal visual pathway to object recognition. We suggest that dorsal cortex is a crucial source of input to the ventral pathway and may support the ability to categorize objects on the basis of global shape.SIGNIFICANCE STATEMENT Humans categorize novel objects rapidly and effortlessly. Such categorization is achieved by representing an object's global shape structure, that is, the relations among object parts. Yet, despite their importance, it is unclear how part relations are represented neurally. Here, we hypothesized that object-centered part relations may be computed by the dorsal visual pathway, which is typically implicated in visuospatial processing. Using fMRI, we identified regions selective for the part relations in dorsal cortex. We found that these regions can support object categorization, and even mediate representations of part relations in the ventral pathway, the region typically thought to support object categorization. Together, these findings shed light on the broader network of brain regions that support object categorization.

Skeletal representations of shape in human vision: Evidence for a pruned medial axis model.

A representation of shape that is low dimensional and stable across minor disruptions is critical for object recognition. Computer vision research suggests that such a representation can be supported by the medial axis-a computational model for extracting a shape's internal skeleton. However, few studies have shown evidence of medial axis processing in humans, and even fewer have examined how the medial axis is extracted in the presence of disruptive contours. Here, we tested whether human skeletal representations of shape reflect the medial axis transform (MAT), a computation sensitive to all available contours, or a pruned medial axis, which ignores contours that may be considered "noise." Across three experiments, participants (N = 2062) were shown complete, perturbed, or illusory two-dimensional shapes on a tablet computer and were asked to tap the shapes anywhere once. When directly compared with another viable model of shape perception (based on principal axes), participants' collective responses were better fit by the medial axis, and a direct test of boundary avoidance suggested that this result was not likely because of a task-specific cognitive strategy (Experiment 1). Moreover, participants' responses reflected a pruned computation in shapes with small or large internal or external perturbations (Experiment 2) and under conditions of illusory contours (Experiment 3). These findings extend previous work by suggesting that humans extract a relatively stable medial axis of shapes. A relatively stable skeletal representation, reflected by a pruned model, may be well equipped to support real-world shape perception and object recognition.

Right idea, wrong magnitude system.

Leibovich et al. claim that number representations are non-existent early in life and that the associations between number and continuous magnitudes reside in stimulus confounds. We challenge both claims - positing, instead, that number is represented independently of continuous magnitudes already in infancy, but is nonetheless more deeply connected to other magnitudes through adulthood than acknowledged by the "sense of magnitude" theory.

Object geometry serves humans' intuitive physics of stability.

How do humans judge physical stability? A prevalent account emphasizes the mental simulation of physical events implemented by an intuitive physics engine in the mind. Here we test the extent to which the perceptual features of object geometry are sufficient for supporting judgments of falling direction. In all experiments, adults and children judged the falling direction of a tilted object and, across experiments, objects differed in the geometric features (i.e., geometric centroid, object height, base size and/or aspect ratio) relevant to the judgment. Participants' performance was compared to computational models trained on geometric features, as well as a deep convolutional neural network (ResNet-50), none of which incorporated mental simulation. Adult and child participants' performance was well fit by models of object geometry, particularly the geometric centroid. ResNet-50 also provided a good account of human performance. Altogether, our findings suggest that object geometry may be sufficient for judging the falling direction of tilted objects, independent of mental simulation.

Visual looming is a primitive for human emotion.

Looming objects afford threat of collision across the animal kingdom. Defensive responses to looming and neural computations for looming detection are strikingly conserved across species. In mammals, information about rapidly approaching threats is conveyed from the retina to the midbrain superior colliculus, where variables that indicate the position and velocity of approach are computed to enable defensive behavior. Although neuroscientific theories posit that midbrain representations contribute to emotion through connectivity with distributed brain systems, it remains unknown whether a computational system for looming detection can predict both defensive behavior and phenomenal experience in humans. Here, we show that a shallow convolutional neural network based on the Drosophila visual system predicts defensive blinking to looming objects in infants and superior colliculus responses to optical expansion in adults. Further, the responses of the convolutional network to a broad array of naturalistic video clips predict self-reported emotion largely on the basis of subjective arousal. Our findings illustrate how motor and experiential components of human emotion relate to species-general systems for survival in unpredictable environments.

Visual looming is a primitive for human emotion.

The neural computations for looming detection are strikingly similar across species. In mammals, information about approaching threats is conveyed from the retina to the midbrain superior colliculus, where approach variables are computed to enable defensive behavior. Although neuroscientific theories posit that midbrain representations contribute to emotion through connectivity with distributed brain systems, it remains unknown whether a computational system for looming detection can predict both defensive behavior and phenomenal experience in humans. Here, we show that a shallow convolutional neural network based on the Drosophila visual system predicts defensive blinking to looming objects in infants and superior colliculus responses to optical expansion in adults. Further, the neural network's responses to naturalistic video clips predict self-reported emotion largely by way of subjective arousal. These findings illustrate how a simple neural network architecture optimized for a species-general task relevant for survival explains motor and experiential components of human emotion.

Temporal asymmetries and interactions between dorsal and ventral visual pathways during object recognition.

Despite their anatomical and functional distinctions, there is growing evidence that the dorsal and ventral visual pathways interact to support object recognition. However, the exact nature of these interactions remains poorly understood. Is the presence of identity-relevant object information in the dorsal pathway simply a byproduct of ventral input? Or, might the dorsal pathway be a source of input to the ventral pathway for object recognition? In the current study, we used high-density EEG-a technique with high temporal precision and spatial resolution sufficient to distinguish parietal and temporal lobes-to characterise the dynamics of dorsal and ventral pathways during object viewing. Using multivariate analyses, we found that category decoding in the dorsal pathway preceded that in the ventral pathway. Importantly, the dorsal pathway predicted the multivariate responses of the ventral pathway in a time-dependent manner, rather than the other way around. Together, these findings suggest that the dorsal pathway is a critical source of input to the ventral pathway for object recognition.

Differential functional reorganization of ventral and dorsal visual pathways following childhood hemispherectomy.

Hemispherectomy is a surgical procedure in which an entire hemisphere of a patient's brain is resected or functionally disconnected to manage seizures in individuals with drug-resistant epilepsy. Despite the extensive loss of both ventral and dorsal visual pathways in one hemisphere, pediatric patients who have undergone hemispherectomy show a remarkably high degree of perceptual function across many domains. In the current study, we sought to understand the extent to which functions of the ventral and dorsal visual pathways reorganize to the contralateral hemisphere following childhood hemispherectomy. To this end, we collected fMRI data from an equal number of left and right hemispherectomy patients who completed tasks that typically elicit lateralized responses from the ventral or the dorsal pathway, namely, word (left ventral), face (right ventral), tool (left dorsal), and global form (right dorsal) perception. Overall, there was greater evidence of functional reorganization in the ventral pathway than in the dorsal pathway. Importantly, because ventral and dorsal reorganization was tested within the very same patients, these results cannot be explained by idiosyncratic factors such as disease etiology, age at the time of surgery, or age at testing. These findings suggest that because the dorsal pathway may mature earlier, it may have a shorter developmental window of plasticity than the ventral pathway and, hence, be less malleable after perturbation.

Differential functional reorganization of ventral and dorsal visual pathways following childhood hemispherectomy.

Hemispherectomy is a surgical procedure in which an entire hemisphere of a patient's brain is resected or functionally disconnected to manage seizures in individuals with drug-resistant epilepsy. Despite the extensive loss of input from both ventral and dorsal visual pathways of one hemisphere, pediatric patients who have undergone hemispherectomy show a remarkably high degree of perceptual function across many domains. In the current study, we sought to understand the extent to which functions of the ventral and dorsal visual pathways reorganize to the contralateral hemisphere following childhood hemispherectomy. To this end, we collected fMRI data from an equal number of left and right hemispherectomy patients who completed tasks that typically elicit lateralized responses from the ventral or the dorsal pathway, namely, word (left ventral), face (right ventral), tool (left dorsal), and global form (right dorsal) perception. Overall, there was greater evidence of functional reorganization in the ventral pathway than in the dorsal pathway. Importantly, because ventral and dorsal reorganization was tested in the very same patients, these results cannot be explained by idiosyncratic factors such as disease etiology, age at the time of surgery, or age at testing. These findings suggest that because the dorsal pathway may mature earlier, it may have a shorter developmental window of plasticity than the ventral pathway and, hence, be less malleable.

Perception of an object's global shape is best described by a model of skeletal structure in human infants.

Categorization of everyday objects requires that humans form representations of shape that are tolerant to variations among exemplars. Yet, how such invariant shape representations develop remains poorly understood. By comparing human infants (6-12 months; N=82) to computational models of vision using comparable procedures, we shed light on the origins and mechanisms underlying object perception. Following habituation to a never-before-seen object, infants classified other novel objects across variations in their component parts. Comparisons to several computational models of vision, including models of high-level and low-level vision, revealed that infants' performance was best described by a model of shape based on the skeletal structure. Interestingly, infants outperformed a range of artificial neural network models, selected for their massive object experience and biological plausibility, under the same conditions. Altogether, these findings suggest that robust representations of shape can be formed with little language or object experience by relying on the perceptually invariant skeletal structure.

Does the brain's ventral visual pathway compute object shape?

A rich behavioral literature has shown that human object recognition is supported by a representation of shape that is tolerant to variations in an object's appearance. Such 'global' shape representations are achieved by describing objects via the spatial arrangement of their local features, or structure, rather than by the appearance of the features themselves. However, accumulating evidence suggests that the ventral visual pathway - the primary substrate underlying object recognition - may not represent global shape. Instead, ventral representations may be better described as a basis set of local image features. We suggest that this evidence forces a reevaluation of the role of the ventral pathway in object perception and posits a broader network for shape perception that encompasses contributions from the dorsal pathway.

Skeletal representations of shape in the human visual cortex.

Shape perception is crucial for object recognition. However, it remains unknown exactly how shape information is represented and used by the visual system. Here, we tested the hypothesis that the visual system represents object shape via a skeletal structure. Using functional magnetic resonance imaging (fMRI) and representational similarity analysis (RSA), we found that a model of skeletal similarity explained significant unique variance in the response profiles of V3 and LO. Moreover, the skeletal model remained predictive in these regions even when controlling for other models of visual similarity that approximate low-to high-level visual features (i.e., Gabor-jet, GIST, HMAX, and AlexNet), and across different surface forms, a manipulation that altered object contours while preserving the underlying skeleton. Together, these findings shed light on shape processing in human vision, as well as the computational properties of V3 and LO. We discuss how these regions may support two putative roles of shape skeletons: namely, perceptual organization and object recognition.

Skeletal descriptions of shape provide unique perceptual information for object recognition.

With seemingly little effort, humans can both identify an object across large changes in orientation and extend category membership to novel exemplars. Although researchers argue that object shape is crucial in these cases, there are open questions as to how shape is represented for object recognition. Here we tested whether the human visual system incorporates a three-dimensional skeletal descriptor of shape to determine an object's identity. Skeletal models not only provide a compact description of an object's global shape structure, but also provide a quantitative metric by which to compare the visual similarity between shapes. Our results showed that a model of skeletal similarity explained the greatest amount of variance in participants' object dissimilarity judgments when compared with other computational models of visual similarity (Experiment 1). Moreover, parametric changes to an object's skeleton led to proportional changes in perceived similarity, even when controlling for another model of structure (Experiment 2). Importantly, participants preferentially categorized objects by their skeletons across changes to local shape contours and non-accidental properties (Experiment 3). Our findings highlight the importance of skeletal structure in vision, not only as a shape descriptor, but also as a diagnostic cue of object identity.

Pupillometry reveals the physiological underpinnings of the aversion to holes.

An unusual, but common, aversion to images with clusters of holes is known as trypophobia. Recent research suggests that trypophobic reactions are caused by visual spectral properties also present in aversive images of evolutionary threatening animals (e.g., snakes and spiders). However, despite similar spectral properties, it remains unknown whether there is a shared emotional response to holes and threatening animals. Whereas snakes and spiders are known to elicit a fear reaction, associated with the sympathetic nervous system, anecdotal reports from self-described trypophobes suggest reactions more consistent with disgust, which is associated with activation of the parasympathetic nervous system. Here we used pupillometry in a novel attempt to uncover the distinct emotional response associated with a trypophobic response to holes. Across two experiments, images of holes elicited greater constriction compared to images of threatening animals and neutral images. Moreover, this effect held when controlling for level of arousal and accounting for the pupil grating response. This pattern of pupillary response is consistent with involvement of the parasympathetic nervous system and suggests a disgust, not a fear, response to images of holes. Although general aversion may be rooted in shared visual-spectral properties, we propose that the specific emotion is determined by cognitive appraisal of the distinct image content.

A general magnitude system in human adults: Evidence from a subliminal priming paradigm.

Despite general agreement that number and other magnitudes share analog format, there is disagreement about the extent to which representations of numerical and non-numerical magnitude recruit common cognitive and neural resources. Cross-dimensional interactions between number and other magnitudes on Stroop-like tasks have been taken as evidence for integration across magnitudes, but such effects are subject to alternative interpretations that allow for differentiated representations. Here we use a subliminal priming paradigm to test for interactions between different magnitudes (number and area) when one magnitude is not consciously detectable. Across two experiments, we first provide evidence for the feasibility of this paradigm by demonstrating that transfer occurs within the dimension of number; that is, symbolic numerals (Arabic digits) that were subliminally primed affected judgments of non-symbolic numerosities in target displays. Crucially, we also found transfer across magnitudes-from subliminally primed numerals to target displays of cumulative surface area whether participants made an ordinal judgment (i.e., "which array is larger in area?") or judged whether two arrays were the same or different in area. These findings suggest that representations of number and area are not fully differentiated. Moreover, they provide unique support for a general magnitude system that includes direct connections, or overlap, between the neural codes for numerical and non-numerical magnitudes.

Gamble on gaze: Eye movements reflect the numerical value of blackjack hands.

We used a novel task-a blackjack game that naturally involves mental summation of numerical values-to investigate the role of attention in the mental number line (MNL) and to provide insight into the ecological validity of this representational format. By analyzing the spatial position of participants' spontaneous, task-irrelevant eye movements, we avoided some of the limitations of previous research on the MNL, in which the findings could be attributed to task-specific factors such as the use of overt spatial cues. In two experiments, we found that eye movements along the horizontal axis reflected the overall numerical value of participants' hands, with smaller-value hands eliciting fixations toward the left of the screen and larger-value hands eliciting fixations toward the right. This pattern held even when controlling for the number of cards in the hand and the value of the card most recently dealt-suggesting that the effects were driven by mental summation of values, not merely by the processing of serial order or individual numbers. Vertical eye movements, in contrast, reflected hand value less reliably. In showing that spontaneous eye movements along the horizontal axis track the magnitude of internally computed sums in an ecologically relevant task, our findings provide evidence for a dynamic MNL that supports magnitude-driven shifts of attention and that may be recruited during everyday forms of numerical reasoning.