Temporal characteristics of facial ensemble in individuals with autism spectrum disorder: examination from arousal and attentional allocation.

Introduction: Individuals with Autism Spectrum Disorder (ASD) show atypical recognition of facial emotions, which has been suggested to stem from arousal and attention allocation. Recent studies have focused on the ability to perceive an average expression from multiple spatially different expressions. This study investigated the effect of autistic traits on temporal ensemble, that is, the perception of the average expression from multiple changing expressions. Methods: We conducted a simplified temporal-ensemble task and analyzed behavioral responses, pupil size, and viewing times for eyes of a face. Participants with and without diagnosis of ASD viewed serial presentations of facial expressions that randomly switched between emotional and neutral. The temporal ratio of the emotional expressions was manipulated. The participants estimated the intensity of the facial emotions for the overall presentation. Results: We obtained three major results: (a) many participants with ASD were less susceptible to the ratio of anger expression for temporal ensembles, (b) they produced significantly greater pupil size for angry expressions (within-participants comparison) and smaller pupil size for sad expressions (between-groups comparison), and (c) pupil size and viewing time to eyes were not correlated with the temporal ensemble. Discussion: These results suggest atypical temporal integration of anger expression and arousal characteristics in individuals with ASD; however, the atypical integration is not fully explained by arousal or attentional allocation.

Effects of temporal properties of facial expressions on the perceived intensity of emotion.

A series of multiple facial expressions can be temporally perceived as an averaged facial expression in a process known as ensemble perception. This study examined the effect of temporal parameters on the perceived intensity of facial expression in each emotion, and how the effect varies with autistic traits in typically developing people. In the experiment, we presented facial expressions that switched from emotional to neutral expressions, and vice versa, for 3 s. Participants rated the overall perceived intensity of the facial emotions as a whole rather than rating individual items within the set. For the two tasks, a ratio of duration of emotional faces to duration of neutral faces (emotional ratio) and the timing for transitions were manipulated individually. The results showed that the intensity of facial emotion was perceived more strongly when the presentation ratio increased and when the emotional expression was presented last. The effects were different among the emotions (e.g. relatively weak in the anger expression). Moreover, the perceived intensity of angry expressions decreased with autistic traits. These results suggest that the properties and individual differences in the facial ensemble of each emotion affect emotional perceptions.

The effect of task-irrelevant spatial contexts on 360-degree attention.

The effect of spatial contexts on attention is important for evaluating the risk of human errors and the accessibility of information in different situations. In traditional studies, this effect has been investigated using display-based and non-laboratory procedures. However, these two procedures are inadequate for measuring attention directed toward 360-degree environments and controlling exogeneous stimuli. In order to resolve these limitations, we used a virtual-reality-based procedure and investigated how spatial contexts of 360-degree environments influence attention. In the experiment, 20 students were asked to search for and report a target that was presented at any location in 360-degree virtual spaces as accurately and quickly as possible. Spatial contexts comprised a basic context (a grey and objectless space) and three specific contexts (a square grid floor, a cubic room, and an infinite floor). We found that response times for the task and eye movements were influenced by the spatial context of 360-degree surrounding spaces. In particular, although total viewing times for the contexts did not match the saliency maps, the differences in total viewing times between the basic and specific contexts did resemble the maps. These results suggest that attention comprises basic and context-dependent characteristics, and the latter are influenced by the saliency of 360-degree contexts even when the contexts are irrelevant to a task.

Spatiotemporal Characteristics of 360-Degree Basic Attention.

The spatiotemporal characteristics of basic attention are important for understanding attending behaviours in real-life situations, and they are useful for evaluating the accessibility of visual information. However, although people are encircled by their 360-degree surroundings in real life, no study has addressed the general characteristics of attention to 360-degree surroundings. Here, we conducted an experiment using virtual reality technology to examine the spatiotemporal characteristics of attention in a highly controlled basic visual context consisting of a 360-degree surrounding. We measured response times and gaze patterns during the 360-degree search task and examined the spatial distribution of attention and its temporal variations in a 360-degree environment based on the participants' physical position. Data were collected from both younger adults and older adults to consider age-related differences. The results showed the fundamental spatiotemporal characteristics of 360-degree attention, which can be used as basic criteria to analyse the structure of exogenous effects on attention in complex 360-degree surroundings in real-life situations. For practical purposes, we created spherical criteria maps of 360-degree attention, which are useful for estimating attending behaviours to 360-degree environmental information or for evaluating visual information design in living environments, workspaces, or other real-life contexts.

Temporal and Spatial Predictability of an Irrelevant Event Differently Affect Detection and Memory of Items in a Visual Sequence.

We examined how the temporal and spatial predictability of a task-irrelevant visual event affects the detection and memory of a visual item embedded in a continuously changing sequence. Participants observed 11 sequentially presented letters, during which a task-irrelevant visual event was either present or absent. Predictabilities of spatial location and temporal position of the event were controlled in 2 × 2 conditions. In the spatially predictable conditions, the event occurred at the same location within the stimulus sequence or at another location, while, in the spatially unpredictable conditions, it occurred at random locations. In the temporally predictable conditions, the event timing was fixed relative to the order of the letters, while in the temporally unpredictable condition; it could not be predicted from the letter order. Participants performed a working memory task and a target detection reaction time (RT) task. Memory accuracy was higher for a letter simultaneously presented at the same location as the event in the temporally unpredictable conditions, irrespective of the spatial predictability of the event. On the other hand, the detection RTs were only faster for a letter simultaneously presented at the same location as the event when the event was both temporally and spatially predictable. Thus, to facilitate ongoing detection processes, an event must be predictable both in space and time, while memory processes are enhanced by temporally unpredictable (i.e., surprising) events. Evidently, temporal predictability has differential effects on detection and memory of a visual item embedded in a sequence of images.

The Effects of Letter Design Features and Aging on Legibility.

We examined how letter legibility is affected by letter design features (size, contrast, font-type, and complexity), and how this changes with age (by comparing younger [ N = 54] and older participants [ N = 54]). Types of letters in the study were Japanese characters, Latin alphabets, and digits. Participants verbally reported the letter presented on a Cathode Ray Tube (CRT) display in natural viewing conditions. The legible-contrast threshold was calculated as the contrast boundary of 50% correct answer. We found that letter complexity (i.e., number of strokes) did not affect legible contrast threshold, except for small (<0.35°) letters. Moreover, letter size even affected the threshold with a function similar but not identical to contrast sensitivity function. This supports the previous hypothesis proposed by Majaj et al. that optimal spatial frequency band exists for letter identification and shifts with its size. Another novel finding was the effect of aging on the legible-contrast threshold and how it related to other letter features. Constant aging effect on legible-contrast was found across letter sizes. Among older adults, the legible-contrast thresholds were 1.2 to 2 times those of younger adults. Moreover, the aging effects were greater for positive-contrast than for negative.

Exogenous temporal cues enhance recognition memory in an object-based manner.

Exogenous attention enhances the perception of attended items in both a space-based and an object-based manner. Exogenous attention also improves recognition memory for attended items in the space-based mode. However, it has not been examined whether object-based exogenous attention enhances recognition memory. To address this issue, we examined whether a sudden visual change in a task-irrelevant stimulus (an exogenous cue) would affect participants' recognition memory for items that were serially presented around a cued time. The results showed that recognition accuracy for an item was strongly enhanced when the visual cue occurred at the same location and time as the item (Experiments 1 and 2). The memory enhancement effect occurred when the exogenous visual cue and an item belonged to the same object (Experiments 3 and 4) and even when the cue was counterpredictive of the timing of an item to be asked about (Experiment 5). The present study suggests that an exogenous temporal cue automatically enhances the recognition accuracy for an item that is presented at close temporal proximity to the cue and that recognition memory enhancement occurs in an object-based manner.

Unpredictable visual changes cause temporal memory averaging.

Various factors influence the perceived timing of visual events. Yet, little is known about the ways in which transient visual stimuli affect the estimation of the timing of other visual events. In the present study, we examined how a sudden color change of an object would influence the remembered timing of another transient event. In each trial, subjects saw a green or red disk travel in circular motion. A visual flash (white frame) occurred at random times during the motion sequence. The color of the disk changed either at random times (unpredictable condition), at a fixed time relative to the motion sequence (predictable condition), or it did not change (no-change condition). The subjects' temporal memory of the visual flash in the predictable condition was as veridical as that in the no-change condition. In the unpredictable condition, however, the flash was reported to occur closer to the timing of the color change than actual timing. Thus, an unpredictable visual change distorts the temporal memory of another visual event such that the remembered moment of the event is closer to the timing of the unpredictable visual change.