Evidence for an event-integration window: A cognitive temporal window supports flexible integration of multimodal events.

Just as the perception of simple events such as clapping hands requires a linkage of sound with movements that produce the sound, the integration of more complex events such as describing how to give an injection requires a linkage between the instructor's utterances and their actions. However, the mechanism for integrating these complex multimodal events is unclear. For example, it is possible that predictive temporal relationships are important for multimodal event understanding, but it is also possible that this form of understanding arises more from meaningful causal between-event links that are temporally unspecified. This latter approach might be supported by a cognitive temporal window within which multimodal event information integrates flexibly with few default commitments about specific temporal relationships. To test this hypothesis, we assessed the consequences of disrupting temporal relationships between instructors' actions and their speech in both narrated screen-capture instructional videos (Experiment 1) and live-action instructional videos (Experiment 2) by displacing the audio channel forward or backward relative to the video by 0, 1, 3, or 7 s. We assessed learning, event segmentation, disruption awareness, segmentation uncertainty, and perceived workload. Across two experiments, 7-s temporal disruptions consistently increased uncertainty and workload and decreased learning in Experiment 2. None of these effects appeared for 3-s disruptions, which were barely detectable. One-second disruptions produced no effects and were undetectable, even though much intraevent information falls within this range. Our results suggest the presence of an event-integration window that supports the integration of events independent of constraining temporal relationships between subevents. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Video speeding can be efficient and speeding-induced preference cost can be lessened by selective speeding.

Over the past decade, screen-captured instructional videos have become popular tools for learning. Viewers wanting to learn efficiently can play these videos at faster-than-normal speeds, a feature offered by hosting services such as YouTube. Although previous research suggests that moderate speeding may not lessen learning, little research has tested this form of media for speeding-induced learning impairments. Further, even if learning is not impaired by speeding, the degree to which users find speed increases taxing and/or unpleasant is unknown. We therefore created a set of screen-captured instructional videos and tested whether speeding them by up to 250% affected learning, perceived workload, and preferences. Speed increases of up to 200% minimally affected learning, but even modest 150% speed increases substantially increased perceived workload and reduced viewer preferences. However, we were able to create videos that were more selectively speeded by concentrating speeding on pauses and relatively unimportant and slow speech. These videos were just as time efficient as the 150% speeded videos, but viewers preferred them. Our findings demonstrate that speeded instructional videos have the potential to facilitate efficient learning, and they suggest techniques such as selective speeding that may be used to support efficiency while lessening viewer preference costs. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Efficient calculations of NSS-based gaze similarity for time-dependent stimuli.

The degree of spatial similarity between the gaze of participants viewing dynamic stimuli such as videos has been previously measured using metrics which are based on the NSS (Normalized Scanpath Saliency). Methods currently used to calculate this metric rely upon a numerical grid, which can be computationally prohibitive for a variety of otherwise useful applications such as Monte Carlo analyses. In the present work we derive a new analytical calculation method for the same metric that yields equal or more accurate results, but with speeds than can be orders of magnitude faster (depending on parameters). Our analytical method scales well with dimensionality, and could also be of use for other applications. The drawback is that it can become very slow if the number of participants in the study is very large or if the gaze sampling rate is high. We provide performance benchmarks for a Fortran implementation of our method, and make available the source code developed.

The Incomplete Tyranny of Dynamic Stimuli: Gaze Similarity Predicts Response Similarity in Screen-Captured Instructional Videos.

Although eye tracking has been used extensively to assess cognitions for static stimuli, recent research suggests that the link between gaze and cognition may be more tenuous for dynamic stimuli such as videos. Part of the difficulty in convincingly linking gaze with cognition is that in dynamic stimuli, gaze position is strongly influenced by exogenous cues such as object motion. However, tests of the gaze-cognition link in dynamic stimuli have been done on only a limited range of stimuli often characterized by highly organized motion. Also, analyses of cognitive contrasts between participants have been mostly been limited to categorical contrasts among small numbers of participants that may have limited the power to observe more subtle influences. We, therefore, tested for cognitive influences on gaze for screen-captured instructional videos, the contents of which participants were tested on. Between-participant scanpath similarity predicted between-participant similarity in responses on test questions, but with imperfect consistency across videos. We also observed that basic gaze parameters and measures of attention to centers of interest only inconsistently predicted learning, and that correlations between gaze and centers of interest defined by other-participant gaze and cursor movement did not predict learning. It, therefore, appears that the search for eye movement indices of cognition during dynamic naturalistic stimuli may be fruitful, but we also agree that the tyranny of dynamic stimuli is real, and that links between eye movements and cognition are highly dependent on task and stimulus properties.

Optimistic metacognitive judgments predict poor performance in relatively complex visual tasks.

In some cases, people overestimate how much they can see. This can produce a metacognitive blind spot that may lead participants to devote fewer cognitive resources than a visual task demands. However, little research has tested whether individuals who are particularly optimistic about their visual capabilities are susceptible to poor visual performance. We tested whether optimistic metacognitive judgments would predict poor performance in a visual task, especially when it placed a large attentional load on the participant, and when it required balancing between multiple sources of information. We tested participants in a simplified battle command simulation in which they were asked to detect visual changes. Participants who predicted spatially expansive visual attention performance performed more poorly in the change detection task when the task required tracking larger numbers of aircraft, and when it included a secondary change-list display.

The interrelationship between concepts about agency and students' use of teachable-agent learning technology.

To successfully interact with software agents, people must call upon basic concepts about goals and intentionality and strategically deploy these concepts in a range of circumstances where specific entailments may or may not apply. We hypothesize that people who can effectively deploy agency concepts in new situations will be more effective in interactions with software agents. Further, we posit that interacting with a software agent can itself refine a person's deployment of agency concepts. We investigated this reciprocal relationship in one particularly important context: the classroom. In three experiments we examined connections between middle school students' concepts about agency and their success learning from a teachable-agent-based computer system called "Betty's Brain". We found that the students who made more intentional behavioral predictions about humans learned more effectively from the system. We also found that students who used the Betty's Brain system distinguished human behavior from machine behavior more strongly than students who did not. We conclude that the ability to effectively deploy agency concepts both supports, and is refined by, interactions with software agents.

Are failures to look, to represent, or to learn associated with change blindness during screen-capture video learning?

Although phenomena such as change blindness and inattentional blindness are robust, it is not entirely clear how these failures of visual awareness are related to failures to attend to visual information, to represent it, and to ultimately learn in visual environments. On some views, failures of visual awareness such as change blindness underestimate the true extent of otherwise rich visual representations. This might occur if people did represent the changing features but failed to compare them across views. In contrast, other approaches emphasize visual representations that are created only when they are functional. On this view, change blindness may be associated with poor representations of the changing properties. It is possible to compromise and propose that representational richness varies across contexts, but then it becomes important to detail relationships among attention, awareness, and learning in specific, but applicable, settings. We therefore assessed these relationships in an important visual setting: screen-captured instructional videos. In two experiments, we tested the degree to which attention (as measured by gaze) predicts change detection, and whether change detection is associated with visual representations and content learning. We observed that attention sometimes predicted change detection, and that change detection was associated with representations of attended objects. However, there was no relationship between change detection and learning.

Audio Facilitates the Perception of Cinematic Continuity by First-Time Viewers.

Previous studies concluded that first-time film viewers often had difficulty integrating shots into a coherent representation of the depicted events in the absence of a familiar action through the film cuts or a salient eye-gazing of a character in the film. In this study, we investigated whether diegetic sound (i.e., sound that seems to originate from the depicted cinematic space) could effectively bridge shots for first-time viewers. Across a range of films, both dialog, and salient environmental sound (e.g., barking dogs) helped first-time viewers connect shots. However, sound was not always successful in supporting first-time viewers' interpretations. While experienced viewers were able to understand less-familiar linking sounds and environments, first-time viewers found this difficult. Overall, a range of diegetic sounds helped first-time viewers understand spatiotemporal relations between shots, but these viewers still had difficulty integrating views of unfamiliar environments.

Bridging views in cinema: a review of the art and science of view integration.

Recently, there has been a surge of interest in the relationship between film and cognitive science. This is reflected in a new science of cinema that can help us both to understand this art form, and to produce new insights about cognition and perception. In this review, we begin by describing how the initial development of cinema involved close observation of audience response. This allowed filmmakers to develop an informal theory of visual cognition that helped them to isolate and creatively recombine fundamental elements of visual experience. We review research exploring naturalistic forms of visual perception and cognition that have opened the door to a productive convergence between the dynamic visual art of cinema and science of visual cognition that can enrich both. In particular, we discuss how parallel understandings of view integration in cinema and in cognitive science have been converging to support a new understanding of meaningful visual experience. WIREs Cogn Sci 2017, 8:e1436. doi: 10.1002/wcs.1436 For further resources related to this article, please visit the WIREs website.

Hearing in color: How expectations distort perception of skin tone.

Previous research has found that the perceived brightness of a face can be distorted by the social category of race. Thus, Levin and Banaji (2006) found, in a U.S. sample, that faces of identical brightness were perceived to be lighter if they had stereotypical White American features than if they had Black American features. Here, we present 2 experiments conducted in Natal, Brazil, that extend this line of research. Experiment 1 tested if the brightness distortion effect would generalize to a Brazilian population. Experiment 2 tested if speech accent would have a similar effect on brightness perception. In Experiment 1, we found that the brightness distortion effect clearly replicated in the Brazilian sample: Faces with Black racial features were perceived to be darker than faces with White racial features, even though their objective brightness was identical. In Experiment 2, we found that speech accent influenced brightness perception in a similar manner: Faces were perceived to be darker when paired with an accent associated with low socioeconomic status than when they were paired with an accent associated with high socioeconomic status. Whereas racial concepts in Brazil are often claimed to be much more fluid compared with the United States, our findings suggest that the populations are quite similar with respect to associations between facial features and skin tone. Our findings also demonstrate speech accent as an additional source of category information that perceptual cognition can take into account when modeling the world. (PsycINFO Database Record

The Face-Race Lightness Illusion Is Not Driven by Low-level Stimulus Properties: An Empirical Reply to Firestone and Scholl (2014).

Levin and Banaji (Journal of Experimental Psychology: General, 135, 501-512, 2006) reported a lightness illusion in which participants appeared to perceive Black faces to be darker than White faces, even though the faces were matched for overall brightness and contrast. Recently, this finding was challenged by Firestone and Scholl (Psychonomic Bulletin and Review, 2014), who argued that the nominal illusion remained even when the faces were blurred so as to make their race undetectable, and concluded that uncontrolled perceptual differences between the stimulus faces drove at least some observations of the original distortion effect. In this paper we report that measures of race perception used by Firestone and Scholl were insufficiently sensitive. We demonstrate that a forced choice race-identification task not only reveals that participants could detect the race of the blurred faces but also that participants' lightness judgments often aligned with their assignment of race.

Cognition can affect perception: Restating the evidence of a top-down effect.

We argue that Firestone & Scholl (F&S) provide worthwhile recommendations but that their critique of research by Levin and Banaji (2006) is unfounded. In addition, we argue that F&S apply unjustified level of skepticism about top-down effects relative to other broad hypotheses about the sources of perceptual intelligence.

The extent of default visual perspective taking in complex layouts.

Emerging research suggests that visual perspective taking might be based in part on a default, early developing cognitive process. This hypothesis receives support from experiments demonstrating that adults experience interference from task-irrelevant perspectives of depicted agents even when participants are making judgments about their own perspective. However, a number of recent articles conclude that this self-judgment interference effect may be because of simple directional cues alone, and might, therefore, not reflect processes specific to visual perspective taking. In 3 studies, we demonstrate that self-judgment interference is constrained by agents' apparent line-of-sight access to subspaces in realistic rendered scenes. Participants displayed processing costs when their perspective conflicted with that of an avatar, who faced in the direction of all possible targets but could not see some of the targets because of occlusion. This interference effect occurred using 2 different configurations of occluders, and disappeared when windows were added to the occluders, allowing avatars line of sight access to all of the targets visible to the participant. These results demonstrate that default perspective taking is not attributable to directional cues alone but instead reflects a relatively sophisticated calculation of an agent's line of sight. (PsycINFO Database Record

Default processing of event sequences.

In a wide range of circumstances, it is important to perceive and represent the sequence of events. For example, sequence perception is necessary to learn statistical contingencies between events, and to generate predictions about events when segmenting actions. However, viewer's awareness of event sequence is rarely tested, and at least some means of encoding event sequence are likely to be resource-intensive. Therefore, previous research may have overestimated the degree to which viewers are aware of specific event sequences. In the experiments reported here, we tested viewers' ability to detect anomalies during visual event sequences. Participants viewed videos containing events that either did or did not contain an out-of-order action. Participants were unable to consistently detect the misordered events, and performance on the task decreased significantly to very low levels when performing a secondary task. In addition, participants almost never detected misorderings in an incidental version of the task, and performance increased when videos ended immediately after the misordering, We argue that these results demonstrate that viewers can effectively perceive the elements of events, but do not consistently test their expectations about the specific sequence of natural events unless bidden to do so by task-specific demands. (PsycINFO Database Record

The role of relational triggers in event perception.

Research exploring visual attention has demonstrated that people are aware of only a small proportion of visual properties, and that people only track these properties over a subset of moments in time. This makes it critical to understand how our perceptual system leverages its limited capacity, such that properties are tracked across views only when they can support an understanding of meaningful events. In this paper, we propose that relational triggers induce between-view property comparisons when spatial relationships between objects appear inconsistent across views-moments that are particularly likely to mark the beginning of meaningful events. In these experiments, we activate relational triggers by violating heuristics that filmmakers use to create visuospatial continuity across views. We find that these violations increase change detection when they coincide with visual property changes, demonstrating that relational triggers induce a comparison of properties held in working memory. We also demonstrate that relational triggers increase the likelihood of event segmentation, and that change detection increases both in response to triggers and natural event boundaries. We propose that relational triggers are an effective heuristic cue that facilitates the comparison of properties when they are likely to be useful during event perception.

Concepts about agency constrain beliefs about visual experience.

Recent research exploring phenomena such as change blindness, inattentional blindness, attentional blink and repetition blindness has revealed a number of counterintuitive ways in which apparently salient visual stimuli often go unnoticed. In fact, large majorities of subjects sometimes predict that they would detect visual changes that actually are rarely noticed, suggesting that people have strong beliefs about visual experience that are demonstrably incorrect. However, for other kinds of visual metacognition, such as picture memory, people underpredict performance. This paper describes two experiments demonstrating that both these overpredictions of change detection, and underpredictions of visual memory can be linked with intuitions about the visual experience of different kinds of agents. Subjects predicted more visual change detection and poorer visual memory for mechanical representational systems (e.g. computer programs) when these were anthropomorphized using intentional terminology.

Visual search for rare targets: distracter tuning as a mechanism for learning from repeated target-absent searches.

In the typical visual search experiment, participants search for targets that are present on half of the trials and absent on the other half. However, many real-world tasks involve targets that are present only occasionally. Given this, it is important to know how people deal with the problem of finding targets they have little experience with. One possibility is that they develop an awareness of the degree to which they have effectively completed a search through complex target-absent scenes. To test this hypothesis, we had participants complete two relatively long search tasks in which only a minority of trials included targets. Stimuli were cluttered real-world scenes, and targets were defined by category. We examined participants' ability to terminate search on the target-absent scenes based on an accurate assessment of scene difficulty. Scene difficulty was estimated by computing the mean correct-trial response time (RT) for each of the target-absent scenes across all participants. These group RTs were then correlated with each participants' individual correct-trial RTs for the same stimuli to assess the degree to which a given participant's search-termination times were correlated with those of the group. These correlations successfully predicted participants' target-detection success in both experiments. These experiments suggest that an integral part of visual search is the need to calibrate search behaviour to scenes of varying levels of complexity even when no targets are present.

Remote control and children's understanding of robots.

Children use goal-directed motion to classify agents as living things from early in infancy. In the current study, we asked whether preschoolers are flexible in their application of this criterion by introducing them to robots that engaged in goal-directed motion. In one case the robot appeared to move fully autonomously, and in the other case it was controlled by a remote. We found that 4- and 5-year-olds attributed fewer living thing properties to the robot after seeing it controlled by a remote, suggesting that they are flexible in their application of the goal-directed motion criterion in the face of conflicting evidence of living thing status. Children can flexibly incorporate internal causes for an agent's behavior to enrich their understanding of novel agents.

Evidence for hypodescent and racial hierarchy in the categorization and perception of biracial individuals.

Individuals who qualify equally for membership in two racial groups provide a rare window into social categorization and perception. In 5 experiments, we tested the extent to which a rule of hypodescent, whereby biracial individuals are assigned the status of their socially subordinate parent group, would govern perceptions of Asian-White and Black-White targets. In Experiment 1, in spite of posing explicit questions concerning Asian-White and Black-White targets, hypodescent was observed in both cases and more strongly in Black-White social categorization. Experiments 2A and 2B used a speeded response task and again revealed evidence of hypodescent in both cases, as well as a stronger effect in the Black-White target condition. In Experiments 3A and 3B, social perception was studied with a face-morphing task. Participants required a face to be lower in proportion minority to be perceived as minority than in proportion White to be perceived as White. Again, the threshold for being perceived as White was higher for Black-White than for Asian-White targets. An independent categorization task in Experiment 3B further confirmed the rule of hypodescent and variation in it that reflected the current racial hierarchy in the United States. These results documenting biases in the social categorization and perception of biracials have implications for resistance to change in the American racial hierarchy.

How do young children deal with hybrids of living and non-living things: the case of humanoid robots.

In this experiment, we tested children's intuitions about entities that bridge the contrast between living and non-living things. Three- and four-year-olds were asked to attribute a range of properties associated with living things and machines to novel category-defying complex artifacts (humanoid robots), a familiar living thing (a girl), and a familiar complex artifact (a camera). Results demonstrated that 4-year-olds tended to treat the category-defying entities like members of the inanimate group, while 3-year-olds showed more variability in their responding. This finding suggests that preschoolers' ability to classify complex artifacts that cross the living-non-living divide becomes more stable between the ages of 3 and 4 and that children at both ages draw on a range of properties when classifying such entities.