Investigating variability in morphological processing with Bayesian distributional models.

We investigated the processing of morphologically complex words adopting an approach that goes beyond estimating average effects and allows testing predictions about variability in performance. We tested masked morphological priming effects with English derived ('printer') and inflected ('printed') forms priming their stems ('print') in non-native speakers, a population that is characterized by large variability. We modeled reaction times with a shifted-lognormal distribution using Bayesian distributional models, which allow assessing effects of experimental manipulations on both the mean of the response distribution ('mu') and its standard deviation ('sigma'). Our results show similar effects on mean response times for inflected and derived primes, but a difference between the two on the sigma of the distribution, with inflectional priming increasing response time variability to a significantly larger extent than derivational priming. This is in line with previous research on non-native processing, which shows more variable results across studies for the processing of inflected forms than for derived forms. More generally, our study shows that treating variability in performance as a direct object of investigation can crucially inform models of language processing, by disentangling effects which would otherwise be indistinguishable. We therefore emphasize the importance of looking beyond average performance and testing predictions on other parameters of the distribution rather than just its central tendency.

Constructing Semantic Models From Words, Images, and Emojis.

A number of recent models of semantics combine linguistic information, derived from text corpora, and visual information, derived from image collections, demonstrating that the resulting multimodal models are better than either of their unimodal counterparts, in accounting for behavioral data. Empirical work on semantic processing has shown that emotion also plays an important role especially in abstract concepts; however, models integrating emotion along with linguistic and visual information are lacking. Here, we first improve on visual and affective representations, derived from state-of-the-art existing models, by choosing models that best fit available human semantic data and extending the number of concepts they cover. Crucially then, we assess whether adding affective representations (obtained from a neural network model designed to predict emojis from co-occurring text) improves the model's ability to fit semantic similarity/relatedness judgments from a purely linguistic and linguistic-visual model. We find that, given specific weights assigned to the models, adding both visual and affective representations improves performance, with visual representations providing an improvement especially for more concrete words, and affective representations improving especially the fit for more abstract words.

Bridging the theoretical gap between semantic representation models without the pressure of a ranking: some lessons learnt from LSA.

In recent years, latent semantic analysis (LSA) has reached a level of maturity at which its presence is ubiquitous in technology as well as in simulation of cognitive processes. In spite of this, in recent years there has been a trend of subjecting LSA to some criticisms, usually because it is compared to other models in very specific tasks and conditions and sometimes without having good knowledge of what the semantic representation of LSA means, and without exploiting all the possibilities of which LSA is capable other than the cosine. This paper provides a critical review to clarify some of the misunderstandings regarding LSA and other space models. The historical stability of the predecessors of LSA, the representational structure of word meaning and the multiple topologies that could arise from a semantic space, the computation of similarity, the myth that LSA dimensions have no meaning, the computational and algorithm plausibility to account for meaning acquisition in LSA (in contrast to others models based on online mechanisms), the possibilities of spatial models to substantiate recent proposals, and, in general, the characteristics of classic vector models and their ease and flexibility to simulate some cognitive phenomena will be reviewed. The review highlights the similarity between LSA and other techniques and proposes using long LSA experiences in other models, especially in predicting models such as word2vec. In sum, it emphasizes the lessons that can be learned from comparing LSA-based models to other models, rather than making statements about "the best."

Spicy Adjectives and Nominal Donkeys: Capturing Semantic Deviance Using Compositionality in Distributional Spaces.

Sophisticated senator and legislative onion. Whether or not you have ever heard of these things, we all have some intuition that one of them makes much less sense than the other. In this paper, we introduce a large dataset of human judgments about novel adjective-noun phrases. We use these data to test an approach to semantic deviance based on phrase representations derived with compositional distributional semantic methods, that is, methods that derive word meanings from contextual information, and approximate phrase meanings by combining word meanings. We present several simple measures extracted from distributional representations of words and phrases, and we show that they have a significant impact on predicting the acceptability of novel adjective-noun phrases even when a number of alternative measures classically employed in studies of compound processing and bigram plausibility are taken into account. Our results show that the extent to which an attributive adjective alters the distributional representation of the noun is the most significant factor in modeling the distinction between acceptable and deviant phrases. Our study extends current applications of compositional distributional semantic methods to linguistically and cognitively interesting problems, and it offers a new, quantitatively precise approach to the challenge of predicting when humans will find novel linguistic expressions acceptable and when they will not.