A conceptual framework for the study of demonstrative reference.

Language allows us to efficiently communicate about the things in the world around us. Seemingly simple words like this and that are a cornerstone of our capability to refer, as they contribute to guiding the attention of our addressee to the specific entity we are talking about. Such demonstratives are acquired early in life, ubiquitous in everyday talk, often closely tied to our gestural communicative abilities, and present in all spoken languages of the world. Based on a review of recent experimental work, here we introduce a new conceptual framework of demonstrative reference. In the context of this framework, we argue that several physical, psychological, and referent-intrinsic factors dynamically interact to influence whether a speaker will use one demonstrative form (e.g., this) or another (e.g., that) in a given setting. However, the relative influence of these factors themselves is argued to be a function of the cultural language setting at hand, the theory-of-mind capacities of the speaker, and the affordances of the specific context in which the speech event takes place. It is demonstrated that the framework has the potential to reconcile findings in the literature that previously seemed irreconcilable. We show that the framework may to a large extent generalize to instances of endophoric reference (e.g., anaphora) and speculate that it may also describe the specific form and kinematics a speaker's pointing gesture takes. Testable predictions and novel research questions derived from the framework are presented and discussed.

Talking about Relations: Factors Influencing the Production of Relational Descriptions.

In a production experiment (Experiment 1) and an acceptability rating one (Experiment 2), we assessed two factors, spatial position and salience, which may influence the production of relational descriptions (such as "the ball between the man and the drawer"). In Experiment 1, speakers were asked to refer unambiguously to a target object (a ball). In Experiment 1a, we addressed the role of spatial position, more specifically if speakers mention the entity positioned leftmost in the scene as (first) relatum. The results showed a small preference to start with the left entity, which leaves room for other factors that could influence spatial reference. Thus, in the following studies, we varied salience systematically, by making one of the relatum candidates animate (Experiment 1b), and by adding attention capture cues, first subliminally by priming one relatum candidate with a flash (Experiment 1c), then explicitly by using salient colors for objects (Experiment 1d). Results indicate that spatial position played a dominant role. Entities on the left were mentioned more often as (first) relatum than those on the right (Experiments 1a-d). Animacy affected reference production in one out of three studies (in Experiment 1d). When salience was manipulated by priming visual attention or by using salient colors, there were no significant effects (Experiments 1c, d). In the acceptability rating study (Experiment 2), participants expressed their preference for specific relata, by ranking descriptions on the basis of how good they thought the descriptions fitted the scene. Results show that participants preferred most the description that had an animate entity as the first mentioned relatum. The relevance of these results for models of reference production is discussed.

How Cognitive Load Influences Speakers' Choice of Referring Expressions.

We report on two experiments investigating the effect of an increased cognitive load for speakers on the choice of referring expressions. Speakers produced story continuations to addressees, in which they referred to characters that were either salient or non-salient in the discourse. In Experiment 1, referents that were salient for the speaker were non-salient for the addressee, and vice versa. In Experiment 2, all discourse information was shared between speaker and addressee. Cognitive load was manipulated by the presence or absence of a secondary task for the speaker. The results show that speakers under load are more likely to produce pronouns, at least when referring to less salient referents. We take this finding as evidence that speakers under load have more difficulties taking discourse salience into account, resulting in the use of expressions that are more economical for themselves.

Stored object knowledge and the production of referring expressions: the case of color typicality.

When speakers describe objects with atypical properties, do they include these properties in their referring expressions, even when that is not strictly required for unique referent identification? Based on previous work, we predict that speakers mention the color of a target object more often when the object is atypically colored, compared to when it is typical. Taking literature from object recognition and visual attention into account, we further hypothesize that this behavior is proportional to the degree to which a color is atypical, and whether color is a highly diagnostic feature in the referred-to object's identity. We investigate these expectations in two language production experiments, in which participants referred to target objects in visual contexts. In Experiment 1, we find a strong effect of color typicality: less typical colors for target objects predict higher proportions of referring expressions that include color. In Experiment 2 we manipulated objects with more complex shapes, for which color is less diagnostic, and we find that the color typicality effect is moderated by color diagnosticity: it is strongest for high-color-diagnostic objects (i.e., objects with a simple shape). These results suggest that the production of atypical color attributes results from a contrast with stored knowledge, an effect which is stronger when color is more central to object identification. Our findings offer evidence for models of reference production that incorporate general object knowledge, in order to be able to capture these effects of typicality on determining the content of referring expressions.

Evidence for the role of shape in mental representations of similes.

People mentally represent the shapes of objects. For instance, the mental representation of an eagle is different when one thinks about a flying or resting eagle. This study examined the role of shape in mental representations of similes (i.e., metaphoric comparisons). We tested the prediction that when people process a simile they will mentally represent the entities of the comparison as having a similar shape. We conducted two experiments in which participants read sentences that either did (experimental sentences) or did not (control sentences) invite comparing two entities. For the experimental sentences, the ground of the comparison was explicit in Experiment 1 ("X has the ability to Z, just like Y") and implicit in Experiment 2 ("X is like Y"). After having read the sentence, participants were presented with line drawings of the two objects, which were either similarly or dissimilarly shaped. They judged whether both objects were mentioned in the preceding sentence. For the experimental sentences, recognition latencies were shorter for similarly shaped objects than for dissimilarly shaped objects. For the control sentences, we did not find such an effect of similarity in shape. These findings suggest that a perceptual symbol of shape is activated when processing similes.

When a Stone Tries to Climb up a Slope: The Interplay between Lexical and Perceptual Animacy in Referential Choices.

Several studies suggest that referential choices are influenced by animacy. On the one hand, animate referents are more likely to be mentioned as subjects than inanimate referents. On the other hand, animate referents are more frequently pronominalized than inanimate referents. These effects have been analyzed as effects of conceptual accessibility. In this paper, we raise the question whether these effects are driven only by lexical concepts, such that referents described by animate lexical items (e.g., "toddler") are more accessible than referents described by inanimate lexical items (e.g., "shoe"), or can also be influenced by context-derived conceptualizations, such that referents that are perceived as animate in a particular context are more accessible than referents that are not. In two animation-retelling experiments, conducted in Dutch, we investigated the influence of lexical and perceptual animacy on the choice of referent and the choice of referring expression. If the effects of animacy are context-dependent, entities that are perceived as animate should yield more subject references and more pronouns than entities that are perceived as inanimate, irrespective of their lexical animacy. If the effects are tied to lexical concepts, entities described with animate lexical items should be mentioned as the subject and pronominalized more frequently than entities described with inanimate lexical items, irrespective of their perceptual animacy. The results show that while only lexical animacy appears to affect the choice of subject referent, perceptual animacy may overrule lexical animacy in the choice of referring expression. These findings suggest that referential choices can be influenced by conceptualizations based on the perceptual context.

How object shape affects visual metaphor processing.

In order to interpret novel metaphoric relations, we have to construct ad hoc categories under which the metaphorically related concepts can be subsumed. Shape is considered to be one of the primary vehicles of object categorization. Accordingly, shape might play a prominent role in interpreting visual metaphors (i.e., two metaphorically related objects depicted in one visual array). This study explores the role of object shape in visual metaphor interpretation of 10- to 12-year-olds. The experiment shows that participants can produce more correspondences between similarly shaped objects as compared to dissimilarly shaped objects and that they need less thinking time to do so. These findings suggest that similarity in shape facilitates the process of interpreting visual metaphors.

HIV/AIDS messages as a spur for conversation among young South Africans?

HIV/AIDS messages are often deliberately puzzling so as to increase the chance for them to be used as food for conversation. The South African health organisation 'loveLife,' for instance, uses messages that include complicated rhetorical expressions in their media campaigns, reasoning that those who find the messages puzzling and wonder about their meaning will be inclined to discuss the messages with their peers. In order to test the assumption that puzzlement about health messages is related to keenness to talk about these messages, structured interviews were held with 30 black learners, ages 15 to 19, from Limpopo Province, South Africa, about the messages of six HIV/ AIDS posters and six HIV/AIDS radio advertisements from 'loveLife' or another South African health organisation. No support was found for the assumption that presenting a puzzling health message will contribute to engaging the recipients in discussion. The participants indicated that they were willing to discuss the themes addressed in either a poster or radio advertisement because they appreciated the message and felt that its content was relevant to them, rather than because the message was puzzling or difficult to understand. The participants' overall actual comprehension of the messages, however, proved to be strikingly low.

Understanding visuals in HIV/AIDS education in South Africa: differences between literate and low-literate audiences.

This article reports on a research project aimed at determining the scope and nature of differences in picture comprehension between literate and low-literate audiences in the context of HIV and AIDS. Structured interviews were held with 30 low-literate and 24 literate adult speakers of African languages. The responses were coded and analysed both qualitatively and quantitatively. Consistent with previous research, we found that purely analogical visuals pose relatively few interpretation problems across the literacy spectrum. Literate and low-literate respondents recognised human beings and familiar analogous objects equally successfully. The interpretation of abstract items was problematic for respondents at both literacy levels, but relatively more so for low-literate respondents. Purely symbolic or conventional abstract elements, such as speech and thought balloons, and purely mathematical symbols are difficult for low-literate individuals since they do not have any analogical residue that will trigger relevant meaning aspects of the visual. Metaphors are difficult when they require culture-specific knowledge. The results strongly suggest that designers should exploit the expressive power of the human body in constructing (abstract) meaning. All humans have comparable experiences with associated basic actions and bodily expressions. Therefore, facial expressions and body postures and positions are powerful in transferring complex messages. We advise that pictorial metaphors, art styles that distort objects, complex pictures with partially symbolic content, as well as abstract symbols borrowed from written language should be omitted where possible.