Clarifying status of DNNs as models of human vision.

On several key issues we agree with the commentators. Perhaps most importantly, everyone seems to agree that psychology has an important role to play in building better models of human vision, and (most) everyone agrees (including us) that deep neural networks (DNNs) will play an important role in modelling human vision going forward. But there are also disagreements about what models are for, how DNN-human correspondences should be evaluated, the value of alternative modelling approaches, and impact of marketing hype in the literature. In our view, these latter issues are contributing to many unjustified claims regarding DNN-human correspondences in vision and other domains of cognition. We explore all these issues in this response.

Does that sound right? A novel method of evaluating models of reading aloud : Rating nonword pronunciations.

Nonword pronunciation is a critical challenge for models of reading aloud but little attention has been given to identifying the best method for assessing model predictions. The most typical approach involves comparing the model's pronunciations of nonwords to pronunciations of the same nonwords by human participants and deeming the model's output correct if it matches with any transcription of the human pronunciations. The present paper introduces a new ratings-based method, in which participants are shown printed nonwords and asked to rate the plausibility of the provided pronunciations, generated here by a speech synthesiser. We demonstrate this method with reference to a previously published database of 915 disyllabic nonwords (Mousikou et al., 2017). We evaluated two well-known psychological models, RC00 and CDP++, as well as an additional grapheme-to-phoneme algorithm known as Sequitur, and compared our model assessment with the corpus-based method adopted by Mousikou et al. We find that the ratings method: a) is much easier to implement than a corpus-based method, b) has a high hit rate and low false-alarm rate in assessing nonword reading accuracy, and c) provided a similar outcome as the corpus-based method in its assessment of RC00 and CDP++. However, the two methods differed in their evaluation of Sequitur, which performed much better under the ratings method. Indeed, our evaluation of Sequitur revealed that the corpus-based method introduced a number of false positives and more often, false negatives. Implications of these findings are discussed.

Deep problems with neural network models of human vision.

Deep neural networks (DNNs) have had extraordinary successes in classifying photographic images of objects and are often described as the best models of biological vision. This conclusion is largely based on three sets of findings: (1) DNNs are more accurate than any other model in classifying images taken from various datasets, (2) DNNs do the best job in predicting the pattern of human errors in classifying objects taken from various behavioral datasets, and (3) DNNs do the best job in predicting brain signals in response to images taken from various brain datasets (e.g., single cell responses or fMRI data). However, these behavioral and brain datasets do not test hypotheses regarding what features are contributing to good predictions and we show that the predictions may be mediated by DNNs that share little overlap with biological vision. More problematically, we show that DNNs account for almost no results from psychological research. This contradicts the common claim that DNNs are good, let alone the best, models of human object recognition. We argue that theorists interested in developing biologically plausible models of human vision need to direct their attention to explaining psychological findings. More generally, theorists need to build models that explain the results of experiments that manipulate independent variables designed to test hypotheses rather than compete on making the best predictions. We conclude by briefly summarizing various promising modeling approaches that focus on psychological data.

The human visual system and CNNs can both support robust online translation tolerance following extreme displacements.

Visual translation tolerance refers to our capacity to recognize objects over a wide range of different retinal locations. Although translation is perhaps the simplest spatial transform that the visual system needs to cope with, the extent to which the human visual system can identify objects at previously unseen locations is unclear, with some studies reporting near complete invariance over 10 degrees and other reporting zero invariance at 4 degrees of visual angle. Similarly, there is confusion regarding the extent of translation tolerance in computational models of vision, as well as the degree of match between human and model performance. Here, we report a series of eye-tracking studies (total N = 70) demonstrating that novel objects trained at one retinal location can be recognized at high accuracy rates following translations up to 18 degrees. We also show that standard deep convolutional neural networks (DCNNs) support our findings when pretrained to classify another set of stimuli across a range of locations, or when a global average pooling (GAP) layer is added to produce larger receptive fields. Our findings provide a strong constraint for theories of human vision and help explain inconsistent findings previously reported with convolutional neural networks (CNNs).

Are there any 'object detectors' in the hidden layers of CNNs trained to identify objects or scenes?

Various methods of measuring unit selectivity have been developed with the aim of better understanding how neural networks work. But the different measures provide divergent estimates of selectivity, and this has led to different conclusions regarding the conditions in which selective object representations are learned and the functional relevance of these representations. In an attempt to better characterize object selectivity, we undertake a comparison of various selectivity measures on a large set of units in AlexNet, including localist selectivity, precision, class-conditional mean activity selectivity (CCMAS), the human interpretation of activation maximization (AM) images, and standard signal-detection measures. We find that the different measures provide different estimates of object selectivity, with precision and CCMAS measures providing misleadingly high estimates. Indeed, the most selective units had a poor hit-rate or a high false-alarm rate (or both) in object classification, making them poor object detectors. We fail to find any units that are even remotely as selective as the 'grandmother cell' units reported in recurrent neural networks. In order to generalize these results, we compared selectivity measures on units in VGG-16 and GoogLeNet trained on the ImageNet or Places-365 datasets that have been described as 'object detectors'. Again, we find poor hit-rates and high false-alarm rates for object classification. We conclude that signal-detection measures provide a better assessment of single-unit selectivity compared to common alternative approaches, and that deep convolutional networks of image classification do not learn object detectors in their hidden layers.

Children's Acquisition of the English Past-Tense: Evidence for a Single-Route Account From Novel Verb Production Data.

This study adjudicates between two opposing accounts of morphological productivity, using English past-tense as its test case. The single-route model (e.g., Bybee & Moder, ) posits that both regular and irregular past-tense forms are generated by analogy across stored exemplars in associative memory. In contrast, the dual-route model (e.g., Prasada & Pinker, ) posits that regular inflection requires use of a formal "add -ed" rule that does not require analogy across regular past-tense forms. Children (aged 3-4; 5-6; 6-7; 9-10) saw animations of an animal performing a novel action described with a novel verb (e.g., gezz; chake). Past-tense forms of novel verbs were elicited by prompting the child to describe what the animal "did yesterday." Collapsing across age group (since no interaction was observed), the likelihood of a verb being produced in regular past-tense form (e.g., gezzed; chaked) was positively associated with the verb's similarity to existing regular verbs, consistent with the single-route model only. Results indicate that children's acquisition of the English past-tense is best explained by a single-route analogical mechanism that does not incorporate a role for formal rules.

A connectionist model of the retreat from verb argument structure overgeneralization.

A central question in language acquisition is how children build linguistic representations that allow them to generalize verbs from one construction to another (e.g., The boy gave a present to the girl → The boy gave the girl a present), whilst appropriately constraining those generalizations to avoid non-adultlike errors (e.g., I said no to her → *I said her no). Although a consensus is emerging that learners solve this problem using both statistical and semantics-based learning procedures (e.g., entrenchment, pre-emption, and semantic verb class formation), there currently exist few - if any - proposals for a learning model that combines these mechanisms. The present study used a connectionist model to test an account that argues for competition between constructions based on (a) verb-in construction frequency, (b) relevance of constructions for the speaker's intended message, and (c) fit between the fine-grained semantic properties of individual verbs and individual constructions. The model was able not only (a) to simulate the overall pattern of overgeneralization-then-retreat, but also (b) to use the semantics of novel verbs to predict their argument structure privileges (just as real learners do), and

Children use statistics and semantics in the retreat from overgeneralization.

How do children learn to restrict their productivity and avoid ungrammatical utterances? The present study addresses this question by examining why some verbs are used with un- prefixation (e.g., unwrap) and others are not (e.g., *unsqueeze). Experiment 1 used a priming methodology to examine children's (3-4; 5-6) grammatical restrictions on verbal un- prefixation. To elicit production of un-prefixed verbs, test trials were preceded by a prime sentence, which described reversal actions with grammatical un- prefixed verbs (e.g., Marge folded her arms and then she unfolded them). Children then completed target sentences by describing cartoon reversal actions corresponding to (potentially) un- prefixed verbs. The younger age-group's production probability of verbs in un- form was negatively related to the frequency of the target verb in bare form (e.g., squeez/e/ed/es/ing), while the production probability of verbs in un- form for both age groups was negatively predicted by the frequency of synonyms to a verb's un- form (e.g., release/*unsqueeze). In Experiment 2, the same children rated the grammaticality of all verbs in un- form. The older age-group's grammaticality judgments were (a) positively predicted by the extent to which each verb was semantically consistent with a semantic "cryptotype" of meanings - where "cryptotype" refers to a covert category of overlapping, probabilistic meanings that are difficult to access - hypothesised to be shared by verbs which take un-, and (b) negatively predicted by the frequency of synonyms to a verb's un- form. Taken together, these experiments demonstrate that children as young as 4;0 employ pre-emption and entrenchment to restrict generalizations, and that use of a semantic cryptotype to guide judgments of overgeneralizations is also evident by age 6;0. Thus, even early developmental accounts of children's restriction of productivity must encompass a mechanism in which a verb's semantic and statistical properties interact.