Brain Representation of Animal and Non-Animal Images in Patients with Mild Cognitive Impairment and Alzheimer's Disease.

Background: In early Alzheimer's disease (AD), high-level visual functions and processing speed are impacted. Few functional magnetic resonance imaging (fMRI) studies have investigated high-level visual deficits in AD, yet none have explored brain activity patterns during rapid animal/non-animal categorization tasks. Objective: To address this, we utilized the previously known Integrated Cognitive Assessment (ICA) to collect fMRI data and compare healthy controls (HC) to individuals with mild cognitive impairment (MCI) and mild AD. Methods: The ICA encompasses a rapid visual categorization task that involves distinguishing between animals and non-animals within natural scenes. To comprehensively explore variations in brain activity levels and patterns, we conducted both univariate and multivariate analyses of fMRI data. Results: The ICA task elicited activation across a range of brain regions, encompassing the temporal, parietal, occipital, and frontal lobes. Univariate analysis, which compared responses to animal versus non-animal stimuli, showed no significant differences in the regions of interest (ROIs) across all groups, with the exception of the left anterior supramarginal gyrus in the HC group. In contrast, multivariate analysis revealed that in both HC and MCI groups, several regions could differentiate between animals and non-animals based on distinct patterns of activity. Notably, such differentiation was absent within the mild AD group. Conclusions: Our study highlights the ICA task's potential as a valuable cognitive assessment tool designed for MCI and AD. Additionally, our use of fMRI pattern analysis provides valuable insights into the complex changes in brain function associated with AD. This approach holds promise for enhancing our understanding of the disease's progression.

The use of the integrated cognitive assessment to improve the efficiency of primary care referrals to memory services in the accelerating dementia pathway technologies study.

Background: Current primary care cognitive assessment tools are either crude or time-consuming instruments that can only detect cognitive impairment when it is well established. This leads to unnecessary or late referrals to memory services, by which time the disease may have already progressed into more severe stages. Due to the COVID-19 pandemic, some memory services have adapted to the new environment by shifting to remote assessments of patients to meet service user demand. However, the use of remote cognitive assessments has been inconsistent, and there has been little evaluation of the outcome of such a change in clinical practice. Emerging research has highlighted computerized cognitive tests, such as the Integrated Cognitive Assessment (ICA), as the leading candidates for adoption in clinical practice. This is true both during the pandemic and in the post-COVID-19 era as part of healthcare innovation. Objectives: The Accelerating Dementias Pathways Technologies (ADePT) Study was initiated in order to address this challenge and develop a real-world evidence basis to support the adoption of ICA as an inexpensive screening tool for the detection of cognitive impairment and improving the efficiency of the dementia care pathway. Methods: Ninety-nine patients aged 55-90 who have been referred to a memory clinic by a general practitioner (GP) were recruited. Participants completed the ICA either at home or in the clinic along with medical history and usability questionnaires. The GP referral and ICA outcome were compared with the specialist diagnosis obtained at the memory clinic.Participants were given the option to carry out a retest visit where they were again given the chance to take the ICA test either remotely or face-to-face. Results: The primary outcome of the study compared GP referral with specialist diagnosis of mild cognitive impairment (MCI) and dementia. Of those the GP referred to memory clinics, 78% were necessary referrals, with ~22% unnecessary referrals, or patients who should have been referred to other services as they had disorders other than MCI/dementia. In the same population the ICA was able to correctly identify cognitive impairment in ~90% of patients, with approximately 9% of patients being false negatives. From the subset of unnecessary GP referrals, the ICA classified ~72% of those as not having cognitive impairment, suggesting that these unnecessary referrals may not have been made if the ICA was in use. ICA demonstrated a sensitivity of 93% for dementia and 83% for MCI, with a specificity of 80% for both conditions in detecting cognitive impairment. Additionally, the test-retest prediction agreement for the ICA was 87.5%. Conclusion: The results from this study demonstrate the potential of the ICA as a screening tool, which can be used to support accurate referrals from primary care settings, along with the work conducted in memory clinics and in secondary care. The ICA's sensitivity and specificity in detecting cognitive impairment in MCI surpassed the overall standard of care reported in existing literature.

Health economic analysis of the integrated cognitive assessment tool to aid dementia diagnosis in the United Kingdom.

Objectives: The aim of this study was to develop a comprehensive economic evaluation of the integrated cognitive assessment (ICA) tool compared with standard cognitive tests when used for dementia screening in primary care and for initial patient triage in memory clinics. Methods: ICA was compared with standard of care comprising a mixture of cognitive assessment tools over a lifetime horizon and employing the UK health and social care perspective. The model combined a decision tree to capture the initial outcomes of the cognitive testing with a Markov structure that estimated long-term outcomes of people with dementia. Quality of life outcomes were quantified using quality-adjusted life years (QALYs), and the economic benefits were assessed using net monetary benefit (NMB). Both costs and QALYs were discounted at 3.5% per annum and cost-effectiveness was assessed using a threshold of £20,000 per QALY gained. Results: ICA dominated standard cognitive assessment tools in both the primary care and memory clinic settings. Introduction of the ICA tool was estimated to result in a lifetime cost saving of approximately £123 and £226 per person in primary care and memory clinics, respectively. QALY gains associated with early diagnosis were modest (0.0016 in primary care and 0.0027 in memory clinic). The net monetary benefit (NMB) of ICA introduction was estimated at £154 in primary care and £281 in the memory clinic settings. Conclusion: Introduction of ICA as a tool to screen primary care patients for dementia and perform initial triage in memory clinics could be cost saving to the UK public health and social care payer.

Effect of deep gray matter atrophy on information processing speed in early relapsing-remitting multiple sclerosis.

BACKGROUND: Cognitive dysfunction, including reduced Information processing speed (IPS), is relatively common in multiple sclerosis(MS). IPS deficits have profound effects on several aspects of patients' life. Previous studies showed that deep gray matter atrophy is highly correlated with overall cognitive impairment in MS. However, the effect of deep gray matter atrophy on IPS deficits is not well understood. In this study, we evaluated the effects of deep gray matter volume changes on IPS in people with early relapse-remitting MS (RRMS) compared to healthy control. METHODS: In this case-control study, we enrolled 63 case with RRMS and 36 healthy controls. All patients were diagnosed within 6 years. IPS was evaluated using the Integrated Cognitive Assessment (ICA) test. We also performed a 1.5T MRI to evaluate deep gray matter structures. RESULTS: People with RRMS had lower accuracy in the ICA test (p = .01). However, the reaction time did not significantly differ between RRMS and control groups (p = .6). Thalamus volume was significantly lower in the RRMS group with impaired IPS compared to the RRMS with normal IPS and control groups (p < 10-4). Other deep gray matter structures were not significantly different between the RRMS with impaired IPS group and the RRMS with normal IPS group. CONCLUSION: Some people with MS are impaired in IPS even in the early stages of the disease. Thalamic atrophy affected IPS in these patients, however atrophy in other deep gray matter structures, including caudate, putamen, globus pallidus, hippocampus, amygdala, accumbens, and cerebellum, were not significantly correlated with IPS impairment in early RRMS.

A multivariate comparison of electroencephalogram and functional magnetic resonance imaging to electrocorticogram using visual object representations in humans.

Today, most neurocognitive studies in humans employ the non-invasive neuroimaging techniques functional magnetic resonance imaging (fMRI) and electroencephalogram (EEG). However, how the data provided by fMRI and EEG relate exactly to the underlying neural activity remains incompletely understood. Here, we aimed to understand the relation between EEG and fMRI data at the level of neural population codes using multivariate pattern analysis. In particular, we assessed whether this relation is affected when we change stimuli or introduce identity-preserving variations to them. For this, we recorded EEG and fMRI data separately from 21 healthy participants while participants viewed everyday objects in different viewing conditions, and then related the data to electrocorticogram (ECoG) data recorded for the same stimulus set from epileptic patients. The comparison of EEG and ECoG data showed that object category signals emerge swiftly in the visual system and can be detected by both EEG and ECoG at similar temporal delays after stimulus onset. The correlation between EEG and ECoG was reduced when object representations tolerant to changes in scale and orientation were considered. The comparison of fMRI and ECoG overall revealed a tighter relationship in occipital than in temporal regions, related to differences in fMRI signal-to-noise ratio. Together, our results reveal a complex relationship between fMRI, EEG, and ECoG signals at the level of population codes that critically depends on the time point after stimulus onset, the region investigated, and the visual contents used.

Temporal dynamics of animacy categorization in the brain of patients with mild cognitive impairment.

Electroencephalography (EEG) has been commonly used to measure brain alterations in Alzheimer's Disease (AD). However, reported changes are limited to those obtained from using univariate measures, including activation level and frequency bands. To look beyond the activation level, we used multivariate pattern analysis (MVPA) to extract patterns of information from EEG responses to images in an animacy categorization task. Comparing healthy controls (HC) with patients with mild cognitive impairment (MCI), we found that the neural speed of animacy information processing is decreased in MCI patients. Moreover, we found critical time-points during which the representational pattern of animacy for MCI patients was significantly discriminable from that of HC, while the activation level remained unchanged. Together, these results suggest that the speed and pattern of animacy information processing provide clinically useful information as a potential biomarker for detecting early changes in MCI and AD patients.

The Use of a Computerized Cognitive Assessment to Improve the Efficiency of Primary Care Referrals to Memory Services: Protocol for the Accelerating Dementia Pathway Technologies (ADePT) Study.

BACKGROUND: Existing primary care cognitive assessment tools are crude or time-consuming screening instruments which can only detect cognitive impairment when it is well established. Due to the COVID-19 pandemic, memory services have adapted to the new environment by moving to remote patient assessments to continue meeting service user demand. However, the remote use of cognitive assessments has been variable while there has been scant evaluation of the outcome of such a change in clinical practice. Emerging research in remote memory clinics has highlighted computerized cognitive tests, such as the Integrated Cognitive Assessment (ICA), as prominent candidates for adoption in clinical practice both during the pandemic and for post-COVID-19 implementation as part of health care innovation. OBJECTIVE: The aim of the Accelerating Dementia Pathway Technologies (ADePT) study is to develop a real-world evidence basis to support the adoption of ICA as an inexpensive screening tool for the detection of cognitive impairment to improve the efficiency of the dementia care pathway. METHODS: Patients who have been referred to a memory clinic by a general practitioner (GP) are recruited. Participants complete the ICA either at home or in the clinic along with medical history and usability questionnaires. The GP referral and ICA outcome are compared with the specialist diagnosis obtained at the memory clinic. The clinical outcomes as well as National Health Service reference costing data will be used to assess the potential health and economic benefits of the use of the ICA in the dementia diagnosis pathway. RESULTS: The ADePT study was funded in January 2020 by Innovate UK (Project Number 105837). As of September 2021, 86 participants have been recruited in the study, with 23 participants also completing a retest visit. Initially, the study was designed for in-person visits at the memory clinic; however, in light of the COVID-19 pandemic, the study was amended to allow remote as well as face-to-face visits. The study was also expanded from a single site to 4 sites in the United Kingdom. We expect results to be published by the second quarter of 2022. CONCLUSIONS: The ADePT study aims to improve the efficiency of the dementia care pathway at its very beginning and supports systems integration at the intersection between primary and secondary care. The introduction of a standardized, self-administered, digital assessment tool for the timely detection of neurodegeneration as part of a decision support system that can signpost accordingly can reduce unnecessary referrals, service backlog, and assessment variability. TRIAL REGISTRATION: ISRCTN 16596456; https://www.isrctn.com/ISRCTN16596456. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/34475.

Validity and Cultural Generalisability of a 5-Minute AI-Based, Computerised Cognitive Assessment in Mild Cognitive Impairment and Alzheimer's Dementia.

Introduction: Early detection and monitoring of mild cognitive impairment (MCI) and Alzheimer's Disease (AD) patients are key to tackling dementia and providing benefits to patients, caregivers, healthcare providers and society. We developed the Integrated Cognitive Assessment (ICA); a 5-min, language independent computerised cognitive test that employs an Artificial Intelligence (AI) model to improve its accuracy in detecting cognitive impairment. In this study, we aimed to evaluate the generalisability of the ICA in detecting cognitive impairment in MCI and mild AD patients. Methods: We studied the ICA in 230 participants. 95 healthy volunteers, 80 MCI, and 55 mild AD participants completed the ICA, Montreal Cognitive Assessment (MoCA) and Addenbrooke's Cognitive Examination (ACE) cognitive tests. Results: The ICA demonstrated convergent validity with MoCA (Pearson r=0.58, p<0.0001) and ACE (r=0.62, p<0.0001). The ICA AI model was able to detect cognitive impairment with an AUC of 81% for MCI patients, and 88% for mild AD patients. The AI model demonstrated improved performance with increased training data and showed generalisability in performance from one population to another. The ICA correlation of 0.17 (p = 0.01) with education years is considerably smaller than that of MoCA (r = 0.34, p < 0.0001) and ACE (r = 0.41, p < 0.0001) which displayed significant correlations. In a separate study the ICA demonstrated no significant practise effect over the duration of the study. Discussion: The ICA can support clinicians by aiding accurate diagnosis of MCI and AD and is appropriate for large-scale screening of cognitive impairment. The ICA is unbiased by differences in language, culture, and education.

A self-administered, artificial intelligence (AI) platform for cognitive assessment in multiple sclerosis (MS).

BACKGROUND: Cognitive impairment is common in patients with multiple sclerosis (MS). Accurate and repeatable measures of cognition have the potential to be used as markers of disease activity. METHODS: We developed a 5-min computerized test to measure cognitive dysfunction in patients with MS. The proposed test - named the Integrated Cognitive Assessment (ICA) - is self-administered and language-independent. Ninety-one MS patients and 83 healthy controls (HC) took part in Substudy 1, in which each participant took the ICA test and the Brief International Cognitive Assessment for MS (BICAMS). We assessed ICA's test-retest reliability, its correlation with BICAMS, its sensitivity to discriminate patients with MS from the HC group, and its accuracy in detecting cognitive dysfunction. In Substudy 2, we recruited 48 MS patients, 38 of which had received an 8-week physical and cognitive rehabilitation programme and 10 MS patients who did not. We examined the association between the level of serum neurofilament light (NfL) in these patients and their ICA scores and Symbol Digit Modalities Test (SDMT) scores pre- and post-rehabilitation. RESULTS: The ICA demonstrated excellent test-retest reliability (r = 0.94), with no learning bias, and showed a high level of convergent validity with BICAMS. The ICA was sensitive in discriminating the MS patients from the HC group, and demonstrated high accuracy (AUC = 95%) in discriminating cognitively normal from cognitively impaired participants. Additionally, we found a strong association (r = - 0.79) between ICA score and the level of NfL in MS patients before and after rehabilitation. CONCLUSIONS: The ICA has the potential to be used as a digital marker of cognitive impairment and to monitor response to therapeutic interventions. In comparison to standard cognitive tools for MS, the ICA is shorter in duration, does not show a learning bias, and is independent of language.

Beyond core object recognition: Recurrent processes account for object recognition under occlusion.

Core object recognition, the ability to rapidly recognize objects despite variations in their appearance, is largely solved through the feedforward processing of visual information. Deep neural networks are shown to achieve human-level performance in these tasks, and explain the primate brain representation. On the other hand, object recognition under more challenging conditions (i.e. beyond the core recognition problem) is less characterized. One such example is object recognition under occlusion. It is unclear to what extent feedforward and recurrent processes contribute in object recognition under occlusion. Furthermore, we do not know whether the conventional deep neural networks, such as AlexNet, which were shown to be successful in solving core object recognition, can perform similarly well in problems that go beyond the core recognition. Here, we characterize neural dynamics of object recognition under occlusion, using magnetoencephalography (MEG), while participants were presented with images of objects with various levels of occlusion. We provide evidence from multivariate analysis of MEG data, behavioral data, and computational modelling, demonstrating an essential role for recurrent processes in object recognition under occlusion. Furthermore, the computational model with local recurrent connections, used here, suggests a mechanistic explanation of how the human brain might be solving this problem.

Integrated Cognitive Assessment: Speed and Accuracy of Visual Processing as a Reliable Proxy to Cognitive Performance.

Various mental disorders are accompanied by some degree of cognitive impairment. Particularly in neurodegenerative disorders, cognitive impairment is the phenotypical hallmark of the disease. Effective, accurate and timely cognitive assessment is key to early diagnosis of this family of mental disorders. Current standard-of-care techniques for cognitive assessment are primarily paper-based, and need to be administered by a healthcare professional; they are additionally language and education-dependent and typically suffer from a learning bias. These tests are thus not ideal for large-scale pro-active cognitive screening and disease progression monitoring. We developed the Integrated Cognitive Assessment (referred to as CGN_ICA), a 5-minute computerized cognitive assessment tool based on a rapid visual categorization task, in which a series of carefully selected natural images of varied difficulty are presented to participants. Overall 448 participants, across a wide age-range with different levels of education took the CGN_ICA test. We compared participants' CGN_ICA test results with a variety of standard pen-and-paper tests, such as Symbol Digit Modalities Test (SDMT) and Montreal Cognitive Assessment (MoCA), that are routinely used to assess cognitive performance. CGN_ICA had excellent test-retest reliability, showed convergent validity with the standard-of-care cognitive tests used here, and demonstrated to be suitable for micro-monitoring of cognitive performance.

Tracking the Spatiotemporal Neural Dynamics of Real-world Object Size and Animacy in the Human Brain.

Animacy and real-world size are properties that describe any object and thus bring basic order into our perception of the visual world. Here, we investigated how the human brain processes real-world size and animacy. For this, we applied representational similarity to fMRI and MEG data to yield a view of brain activity with high spatial and temporal resolutions, respectively. Analysis of fMRI data revealed that a distributed and partly overlapping set of cortical regions extending from occipital to ventral and medial temporal cortex represented animacy and real-world size. Within this set, parahippocampal cortex stood out as the region representing animacy and size stronger than most other regions. Further analysis of the detailed representational format revealed differences among regions involved in processing animacy. Analysis of MEG data revealed overlapping temporal dynamics of animacy and real-world size processing starting at around 150 msec and provided the first neuromagnetic signature of real-world object size processing. Finally, to investigate the neural dynamics of size and animacy processing simultaneously in space and time, we combined MEG and fMRI with a novel extension of MEG-fMRI fusion by representational similarity. This analysis revealed partly overlapping and distributed spatiotemporal dynamics, with parahippocampal cortex singled out as a region that represented size and animacy persistently when other regions did not. Furthermore, the analysis highlighted the role of early visual cortex in representing real-world size. A control analysis revealed that the neural dynamics of processing animacy and size were distinct from the neural dynamics of processing low-level visual features. Together, our results provide a detailed spatiotemporal view of animacy and size processing in the human brain.

Towards building a more complex view of the lateral geniculate nucleus: Recent advances in understanding its role.

The lateral geniculate nucleus (LGN) has often been treated in the past as a linear filter that adds little to retinal processing of visual inputs. Here we review anatomical, neurophysiological, brain imaging, and modeling studies that have in recent years built up a much more complex view of LGN. These include effects related to nonlinear dendritic processing, cortical feedback, synchrony and oscillations across LGN populations, as well as involvement of LGN in higher level cognitive processing. Although recent studies have provided valuable insights into early visual processing including the role of LGN, a unified model of LGN responses to real-world objects has not yet been developed. In the light of recent data, we suggest that the role of LGN deserves more careful consideration in developing models of high-level visual processing.

Fixed versus mixed RSA: Explaining visual representations by fixed and mixed feature sets from shallow and deep computational models.

Studies of the primate visual system have begun to test a wide range of complex computational object-vision models. Realistic models have many parameters, which in practice cannot be fitted using the limited amounts of brain-activity data typically available. Task performance optimization (e.g. using backpropagation to train neural networks) provides major constraints for fitting parameters and discovering nonlinear representational features appropriate for the task (e.g. object classification). Model representations can be compared to brain representations in terms of the representational dissimilarities they predict for an image set. This method, called representational similarity analysis (RSA), enables us to test the representational feature space as is (fixed RSA) or to fit a linear transformation that mixes the nonlinear model features so as to best explain a cortical area's representational space (mixed RSA). Like voxel/population-receptive-field modelling, mixed RSA uses a training set (different stimuli) to fit one weight per model feature and response channel (voxels here), so as to best predict the response profile across images for each response channel. We analysed response patterns elicited by natural images, which were measured with functional magnetic resonance imaging (fMRI). We found that early visual areas were best accounted for by shallow models, such as a Gabor wavelet pyramid (GWP). The GWP model performed similarly with and without mixing, suggesting that the original features already approximated the representational space, obviating the need for mixing. However, a higher ventral-stream visual representation (lateral occipital region) was best explained by the higher layers of a deep convolutional network and mixing of its feature set was essential for this model to explain the representation. We suspect that mixing was essential because the convolutional network had been trained to discriminate a set of 1000 categories, whose frequencies in the training set did not match their frequencies in natural experience or their behavioural importance. The latter factors might determine the representational prominence of semantic dimensions in higher-level ventral-stream areas. Our results demonstrate the benefits of testing both the specific representational hypothesis expressed by a model's original feature space and the hypothesis space generated by linear transformations of that feature space.

A specialized face-processing model inspired by the organization of monkey face patches explains several face-specific phenomena observed in humans.

Converging reports indicate that face images are processed through specialized neural networks in the brain -i.e. face patches in monkeys and the fusiform face area (FFA) in humans. These studies were designed to find out how faces are processed in visual system compared to other objects. Yet, the underlying mechanism of face processing is not completely revealed. Here, we show that a hierarchical computational model, inspired by electrophysiological evidence on face processing in primates, is able to generate representational properties similar to those observed in monkey face patches (posterior, middle and anterior patches). Since the most important goal of sensory neuroscience is linking the neural responses with behavioral outputs, we test whether the proposed model, which is designed to account for neural responses in monkey face patches, is also able to predict well-documented behavioral face phenomena observed in humans. We show that the proposed model satisfies several cognitive face effects such as: composite face effect and the idea of canonical face views. Our model provides insights about the underlying computations that transfer visual information from posterior to anterior face patches.

Perceptual similarity of visual patterns predicts dynamic neural activation patterns measured with MEG.

Perceptual similarity is a cognitive judgment that represents the end-stage of a complex cascade of hierarchical processing throughout visual cortex. Previous studies have shown a correspondence between the similarity of coarse-scale fMRI activation patterns and the perceived similarity of visual stimuli, suggesting that visual objects that appear similar also share similar underlying patterns of neural activation. Here we explore the temporal relationship between the human brain's time-varying representation of visual patterns and behavioral judgments of perceptual similarity. The visual stimuli were abstract patterns constructed from identical perceptual units (oriented Gabor patches) so that each pattern had a unique global form or perceptual 'Gestalt'. The visual stimuli were decodable from evoked neural activation patterns measured with magnetoencephalography (MEG), however, stimuli differed in the similarity of their neural representation as estimated by differences in decodability. Early after stimulus onset (from 50ms), a model based on retinotopic organization predicted the representational similarity of the visual stimuli. Following the peak correlation between the retinotopic model and neural data at 80ms, the neural representations quickly evolved so that retinotopy no longer provided a sufficient account of the brain's time-varying representation of the stimuli. Overall the strongest predictor of the brain's representation was a model based on human judgments of perceptual similarity, which reached the limits of the maximum correlation with the neural data defined by the 'noise ceiling'. Our results show that large-scale brain activation patterns contain a neural signature for the perceptual Gestalt of composite visual features, and demonstrate a strong correspondence between perception and complex patterns of brain activity.

Visual representations are dominated by intrinsic fluctuations correlated between areas.

Intrinsic cortical dynamics are thought to underlie trial-to-trial variability of visually evoked responses in animal models. Understanding their function in the context of sensory processing and representation is a major current challenge. Here we report that intrinsic cortical dynamics strongly affect the representational geometry of a brain region, as reflected in response-pattern dissimilarities, and exaggerate the similarity of representations between brain regions. We characterized the representations in several human visual areas by representational dissimilarity matrices (RDMs) constructed from fMRI response-patterns for natural image stimuli. The RDMs of different visual areas were highly similar when the response-patterns were estimated on the basis of the same trials (sharing intrinsic cortical dynamics), and quite distinct when patterns were estimated on the basis of separate trials (sharing only the stimulus-driven component). We show that the greater similarity of the representational geometries can be explained by coherent fluctuations of regional-mean activation within visual cortex, reflecting intrinsic dynamics. Using separate trials to study stimulus-driven representations revealed clearer distinctions between the representational geometries: a Gabor wavelet pyramid model explained representational geometry in visual areas V1-3 and a categorical animate-inanimate model in the object-responsive lateral occipital cortex.

Deep supervised, but not unsupervised, models may explain IT cortical representation.

Inferior temporal (IT) cortex in human and nonhuman primates serves visual object recognition. Computational object-vision models, although continually improving, do not yet reach human performance. It is unclear to what extent the internal representations of computational models can explain the IT representation. Here we investigate a wide range of computational model representations (37 in total), testing their categorization performance and their ability to account for the IT representational geometry. The models include well-known neuroscientific object-recognition models (e.g. HMAX, VisNet) along with several models from computer vision (e.g. SIFT, GIST, self-similarity features, and a deep convolutional neural network). We compared the representational dissimilarity matrices (RDMs) of the model representations with the RDMs obtained from human IT (measured with fMRI) and monkey IT (measured with cell recording) for the same set of stimuli (not used in training the models). Better performing models were more similar to IT in that they showed greater clustering of representational patterns by category. In addition, better performing models also more strongly resembled IT in terms of their within-category representational dissimilarities. Representational geometries were significantly correlated between IT and many of the models. However, the categorical clustering observed in IT was largely unexplained by the unsupervised models. The deep convolutional network, which was trained by supervision with over a million category-labeled images, reached the highest categorization performance and also best explained IT, although it did not fully explain the IT data. Combining the features of this model with appropriate weights and adding linear combinations that maximize the margin between animate and inanimate objects and between faces and other objects yielded a representation that fully explained our IT data. Overall, our results suggest that explaining IT requires computational features trained through supervised learning to emphasize the behaviorally important categorical divisions prominently reflected in IT.

Feedforward object-vision models only tolerate small image variations compared to human.

Invariant object recognition is a remarkable ability of primates' visual system that its underlying mechanism has constantly been under intense investigations. Computational modeling is a valuable tool toward understanding the processes involved in invariant object recognition. Although recent computational models have shown outstanding performances on challenging image databases, they fail to perform well in image categorization under more complex image variations. Studies have shown that making sparse representation of objects by extracting more informative visual features through a feedforward sweep can lead to higher recognition performances. Here, however, we show that when the complexity of image variations is high, even this approach results in poor performance compared to humans. To assess the performance of models and humans in invariant object recognition tasks, we built a parametrically controlled image database consisting of several object categories varied in different dimensions and levels, rendered from 3D planes. Comparing the performance of several object recognition models with human observers shows that only in low-level image variations the models perform similar to humans in categorization tasks. Furthermore, the results of our behavioral experiments demonstrate that, even under difficult experimental conditions (i.e., briefly presented masked stimuli with complex image variations), human observers performed outstandingly well, suggesting that the models are still far from resembling humans in invariant object recognition. Taken together, we suggest that learning sparse informative visual features, although desirable, is not a complete solution for future progresses in object-vision modeling. We show that this approach is not of significant help in solving the computational crux of object recognition (i.e., invariant object recognition) when the identity-preserving image variations become more complex.

The impact of the lateral geniculate nucleus and corticogeniculate interactions on efficient coding and higher-order visual object processing.

Principles of efficient coding suggest that the peripheral units of any sensory processing system are designed for efficient coding. The function of the lateral geniculate nucleus (LGN) as an early stage in the visual system is not well understood. Some findings indicate that similar to the retina that decorrelates input signals spatially, the LGN tends to perform a temporal decorrelation. There is evidence suggesting that corticogeniculate connections may account for this decorrelation in the LGN. In this study, we propose a computational model based on biological evidence reported by Wang et al. (2006), who demonstrated that the influence pattern of V1 feedback is phase-reversed. The output of our model shows how corticogeniculate connections decorrelate LGN responses and make an efficient representation. We evaluated our model using criteria that have previously been tested on LGN neurons through cell recording experiments, including sparseness, entropy, power spectra, and information transfer. We also considered the role of the LGN in higher-order visual object processing, comparing the categorization performance of human subjects with a cortical object recognition model in the presence and absence of our LGN input-stage model. Our results show that the new model that considers the role of the LGN, more closely follows the categorization performance of human subjects.