Signatures of cross-modal alignment in children's early concepts.

Whether supervised or unsupervised, human and machine learning is usually characterized as event-based. However, learning may also proceed by systems alignment in which mappings are inferred between entire systems, such as visual and linguistic systems. Systems alignment is possible because items that share similar visual contexts, such as a car and a truck, will also tend to share similar linguistic contexts. Because of the mirrored similarity relationships across systems, the visual and linguistic systems can be aligned at some later time absent either input. In a series of simulation studies, we considered whether children's early concepts support systems alignment. We found that children's early concepts are close to optimal for inferring novel concepts through systems alignment, enabling agents to correctly infer more than 85% of visual-word mappings absent supervision. One possible explanation for why children's early concepts support systems alignment is that they are distinguished structurally by their dense semantic neighborhoods. Artificial agents using these structural features to select concepts proved highly effective, both in environments mirroring children's conceptual world and those that exclude the concepts that children commonly acquire. For children, systems alignment and event-based learning likely complement one another. Likewise, artificial systems can benefit from incorporating these developmental principles.

Modeling Similarity and Psychological Space.

Similarity and categorization are fundamental processes in human cognition that help complex organisms make sense of the cacophony of information in their environment. These processes are critical for tasks such as recognizing objects, making decisions, and forming memories. In this review, we provide an overview of the current state of knowledge on similarity and psychological spaces, discussing the theories, methods, and empirical findings that have been generated over the years. Although the concept of similarity has important limitations, it plays a key role in cognitive modeling. The review surfaces three key themes. First, similarity and mental representations are merely two sides of the same coin, existing as a similarity-representation duality that defines a psychological space. Second, both the brain's mental representations and the study of mental representations are made possible by exploiting second-order isomorphism. Third, similarity analysis has near-universal applicability across all levels of cognition, providing a common research language.

A too-good-to-be-true prior to reduce shortcut reliance.

Despite their impressive performance in object recognition and other tasks under standard testing conditions, deep networks often fail to generalize to out-of-distribution (o.o.d.) samples. One cause for this shortcoming is that modern architectures tend to rely on ǣshortcutsǥ superficial features that correlate with categories without capturing deeper invariants that hold across contexts. Real-world concepts often possess a complex structure that can vary superficially across contexts, which can make the most intuitive and promising solutions in one context not generalize to others. One potential way to improve o.o.d. generalization is to assume simple solutions are unlikely to be valid across contexts and avoid them, which we refer to as the too-good-to-be-true prior. A low-capacity network (LCN) with a shallow architecture should only be able to learn surface relationships, including shortcuts. We find that LCNs can serve as shortcut detectors. Furthermore, an LCN's predictions can be used in a two-stage approach to encourage a high-capacity network (HCN) to rely on deeper invariant features that should generalize broadly. In particular, items that the LCN can master are downweighted when training the HCN. Using a modified version of the CIFAR-10 dataset in which we introduced shortcuts, we found that the two-stage LCN-HCN approach reduced reliance on shortcuts and facilitated o.o.d. generalization.

Predicting the Ease of Human Category Learning Using Radial Basis Function Networks.

Our goal is to understand and optimize human concept learning by predicting the ease of learning of a particular exemplar or category. We propose a method for estimating ease values, quantitative measures of ease of learning, as an alternative to conducting costly empirical training studies. Our method combines a psychological embedding of domain exemplars with a pragmatic categorization model. The two components are integrated using a radial basis function network (RBFN) that predicts ease values. The free parameters of the RBFN are fit using human similarity judgments, circumventing the need to collect human training data to fit more complex models of human categorization. We conduct two category-training experiments to validate predictions of the RBFN. We demonstrate that an instance-based RBFN outperforms both a prototype-based RBFN and an empirical approach using the raw data. Although the human data were collected across diverse experimental conditions, the predicted ease values strongly correlate with human learning performance. Training can be sequenced by (predicted) ease, achieving what is known as fading in the psychology literature and curriculum learning in the machine-learning literature, both of which have been shown to facilitate learning.

Obtaining psychological embeddings through joint kernel and metric learning.

Psychological embeddings provide a powerful formalism for characterizing human-perceived similarity among members of a stimulus set. Obtaining high-quality embeddings can be costly due to algorithm design, software deployment, and participant compensation. This work aims to advance state-of-the-art embedding techniques and provide a comprehensive software package that makes obtaining high-quality psychological embeddings both easy and relatively efficient. Contributions are made on four fronts. First, the embedding procedure allows multiple trial configurations (e.g., triplets) to be used for collecting similarity judgments from participants. For example, trials can be configured to collect triplet comparisons or to sort items into groups. Second, a likelihood model is provided for three classes of similarity kernels allowing users to easily infer the parameters of their preferred model using gradient descent. Third, an active selection algorithm is provided that makes data collection more efficient by proposing comparisons that provide the strongest constraints on the embedding. Fourth, the likelihood model allows the specification of group-specific attention weight parameters. A series of experiments are included to highlight each of these contributions and their impact on converging to a high-quality embedding. Collectively, these incremental improvements provide a powerful and complete set of tools for inferring psychological embeddings. The relevant tools are available as the Python package PsiZ, which can be cloned from GitHub ( https://github.com/roads/psiz ).

The Dimensions of dimensionality.

Cognitive scientists often infer multidimensional representations from data. Whether the data involve text, neuroimaging, neural networks, or human judgments, researchers frequently infer and analyze latent representational spaces (i.e., embeddings). However, the properties of a latent representation (e.g., prediction performance, interpretability, compactness) depend on the inference procedure, which can vary widely across endeavors. For example, dimensions are not always globally interpretable and the dimensionality of different embeddings may not be readily comparable. Moreover, the dichotomy between multidimensional spaces and purportedly richer representational formats, such as graph representations, is misleading. We review what the different notions of dimension in cognitive science imply for how these latent representations should be used and interpreted.

System alignment supports cross-domain learning and zero-shot generalisation.

Recent findings suggest conceptual relationships hold across modalities. For instance, if two concepts occur in similar linguistic contexts, they also likely occur in similar visual contexts. These similarity structures may provide a valuable signal for alignment when learning to map between domains, such as when learning the names of objects. To assess this possibility, we conducted a paired-associate learning experiment in which participants mapped objects that varied on two visual features to locations that varied along two spatial dimensions. We manipulated whether the featural and spatial systems were aligned or misaligned. Although system alignment was not required to complete this supervised learning task, we found that participants learned more efficiently when systems aligned and that aligned systems facilitated zero-shot generalisation. We fit a variety of models to individuals' responses and found that models which included an offline unsupervised alignment mechanism best accounted for human performance. Our results provide empirical evidence that people align entire representation systems to accelerate learning, even when learning seemingly arbitrary associations between two domains.

The Costs and Benefits of Goal-Directed Attention in Deep Convolutional Neural Networks.

People deploy top-down, goal-directed attention to accomplish tasks, such as finding lost keys. By tuning the visual system to relevant information sources, object recognition can become more efficient (a benefit) and more biased toward the target (a potential cost). Motivated by selective attention in categorisation models, we developed a goal-directed attention mechanism that can process naturalistic (photographic) stimuli. Our attention mechanism can be incorporated into any existing deep convolutional neural networks (DCNNs). The processing stages in DCNNs have been related to ventral visual stream. In that light, our attentional mechanism incorporates top-down influences from prefrontal cortex (PFC) to support goal-directed behaviour. Akin to how attention weights in categorisation models warp representational spaces, we introduce a layer of attention weights to the mid-level of a DCNN that amplify or attenuate activity to further a goal. We evaluated the attentional mechanism using photographic stimuli, varying the attentional target. We found that increasing goal-directed attention has benefits (increasing hit rates) and costs (increasing false alarm rates). At a moderate level, attention improves sensitivity (i.e. increases d ' ) at only a moderate increase in bias for tasks involving standard images, blended images and natural adversarial images chosen to fool DCNNs. These results suggest that goal-directed attention can reconfigure general-purpose DCNNs to better suit the current task goal, much like PFC modulates activity along the ventral stream. In addition to being more parsimonious and brain consistent, the mid-level attention approach performed better than a standard machine learning approach for transfer learning, namely retraining the final network layer to accommodate the new task.

Similarity as a Window on the Dimensions of Object Representation.

Hebart et al. recently analysed 1.5 million human similarity judgments and found that natural objects are described by a small set of interpretable dimensions. Such large-scale analyses offer new opportunities to characterise how people represent their knowledge, but also challenges, including scaling to even larger data sets and integrating accounts of semantic representation.

The easy-to-hard training advantage with real-world medical images.

Many medical professions require practitioners to perform visual categorizations in domains such as radiology, dermatology, and neurology. However, acquiring visual expertise is tedious and time-consuming and the perceptual strategies mediating visual categorization skills are poorly understood. In this paper, the Ease algorithm was developed to predict an item's categorization difficulty (Ease value) based on the item's perceptual similarity to all within-category items versus between-category items in the dataset. In this study, Ease values were used to construct an easy-to-hard and hard-to-easy training schedule for teaching melanoma diagnosis. Whereas previous visual training studies suggest that an easy-to-hard schedule benefits learning outcomes, no studies to date have demonstrated the easy-to-hard advantage with complex, real-world images. In our study, 237 melanoma and benign images were collected for training and testing purposes. The diagnostic accuracy of images was verified by an expert dermatologist. Based on their Ease values, the items were grouped into easy, medium, and hard categories, each containing an equal number of melanoma and benign lesions. During training, participants categorized images of skin lesions as either benign or melanoma and were given corrective feedback after each trial. In the easy-to-hard training condition, participants learned to categorize all the easy items first, followed by the medium items, and finally the hard items. Participants in the hard-to-easy training condition learned items in the reverse order. Post-training results showed that training in both conditions transferred to the classification of new melanoma and benign images. Participants in the easy-to-hard condition showed modest advantages both in the acquisition and retention of the melanoma diagnosis skills, but neither scheduling condition exhibited a gross advantage. The Ease values of the items predicted categorization accuracy after, but not before training, suggesting that the Ease algorithm is a promising tool for optimizing medical training in visual categorization.

Improving Human-Machine Cooperative Classification Via Cognitive Theories of Similarity.

Acquiring perceptual expertise is slow and effortful. However, untrained novices can accurately make difficult classification decisions (e.g., skin-lesion diagnosis) by reformulating the task as similarity judgment. Given a query image and a set of reference images, individuals are asked to select the best matching reference. When references are suitably chosen, the procedure yields an implicit classification of the query image. To optimize reference selection, we develop and evaluate a predictive model of similarity-based choice. The model builds on existing psychological literature and accommodates stochastic, dynamic shifts of attention among visual feature dimensions. We perform a series of human experiments with two stimulus types (rectangles, faces) and nine classification tasks to validate the model and to demonstrate the model's potential to boost performance. Our system achieves high accuracy for participants who are naive as to the classification task, even when the classification task switches from trial to trial.