Test-retest reliability for common tasks in vision science.

Research in perception and attention has typically sought to evaluate cognitive mechanisms according to the average response to a manipulation. Recently, there has been a shift toward appreciating the value of individual differences and the insight gained by exploring the impacts of between-participant variation on human cognition. However, a recent study suggests that many robust, well-established cognitive control tasks suffer from surprisingly low levels of test-retest reliability (Hedge, Powell, & Sumner, 2018b). We tested a large sample of undergraduate students (n = 160) in two sessions (separated by 1-3 weeks) on four commonly used tasks in vision science. We implemented measures that spanned a range of perceptual and attentional processes, including motion coherence (MoCo), useful field of view (UFOV), multiple-object tracking (MOT), and visual working memory (VWM). Intraclass correlations ranged from good to poor, suggesting that some task measures are more suitable for assessing individual differences than others. VWM capacity (intraclass correlation coefficient [ICC] = 0.77), MoCo threshold (ICC = 0.60), UFOV middle accuracy (ICC = 0.60), and UFOV outer accuracy (ICC = 0.74) showed good-to-excellent reliability. Other measures, namely the maximum number of items tracked in MOT (ICC = 0.41) and UFOV number accuracy (ICC = 0.48), showed moderate reliability; the MOT threshold (ICC = 0.36) and UFOV inner accuracy (ICC = 0.30) showed poor reliability. In this paper, we present these results alongside a summary of reliabilities estimated previously for other vision science tasks. We then offer useful recommendations for evaluating test-retest reliability when considering a task for use in evaluating individual differences.

The predictability of a target's motion influences gaze, head, and hand movements when trying to intercept it.

Does the predictability of a target's movement and of the interception location influence how the target is intercepted? In a first experiment, we manipulated the predictability of the interception location. A target moved along a haphazardly curved path, and subjects attempted to tap on it when it entered a hitting zone. The hitting zone was either a large ring surrounding the target's starting position (ring condition) or a small disk that became visible before the target appeared (disk condition). The interception location gradually became apparent in the ring condition, whereas it was immediately apparent in the disk condition. In the ring condition, subjects pursued the target with their gaze. Their heads and hands gradually moved in the direction of the future tap position. In the disk condition, subjects immediately directed their gaze toward the hitting zone by moving both their eyes and heads. They also moved their hands to the future tap position sooner than in the ring condition. In a second and third experiment, we made the target's movement more predictable. Although this made the targets easier to pursue, subjects now shifted their gaze to the hitting zone soon after the target appeared in the ring condition. In the disk condition, they still usually shifted their gaze to the hitting zone at the beginning of the trial. Together, the experiments show that predictability of the interception location is more important than predictability of target movement in determining how we move to intercept targets. NEW & NOTEWORTHY We show that if people are required to intercept a target at a known location, they direct their gaze to the interception point as soon as they can rather than pursuing the target with their eyes for as long as possible. The predictability of the interception location rather than the predictability of the path to that location largely determines how the eyes, head, and hand move.

Improvement in visual search with practice: mapping learning-related changes in neurocognitive stages of processing.

Practice can improve performance on visual search tasks; the neural mechanisms underlying such improvements, however, are not clear. Response time typically shortens with practice, but which components of the stimulus-response processing chain facilitate this behavioral change? Improved search performance could result from enhancements in various cognitive processing stages, including (1) sensory processing, (2) attentional allocation, (3) target discrimination, (4) motor-response preparation, and/or (5) response execution. We measured event-related potentials (ERPs) as human participants completed a five-day visual-search protocol in which they reported the orientation of a color popout target within an array of ellipses. We assessed changes in behavioral performance and in ERP components associated with various stages of processing. After practice, response time decreased in all participants (while accuracy remained consistent), and electrophysiological measures revealed modulation of several ERP components. First, amplitudes of the early sensory-evoked N1 component at 150 ms increased bilaterally, indicating enhanced visual sensory processing of the array. Second, the negative-polarity posterior-contralateral component (N2pc, 170-250 ms) was earlier and larger, demonstrating enhanced attentional orienting. Third, the amplitude of the sustained posterior contralateral negativity component (SPCN, 300-400 ms) decreased, indicating facilitated target discrimination. Finally, faster motor-response preparation and execution were observed after practice, as indicated by latency changes in both the stimulus-locked and response-locked lateralized readiness potentials (LRPs). These electrophysiological results delineate the functional plasticity in key mechanisms underlying visual search with high temporal resolution and illustrate how practice influences various cognitive and neural processing stages leading to enhanced behavioral performance.

Face symmetry assessment abilities: Clinical implications for diagnosing asymmetry.

INTRODUCTION: An accurate assessment of face symmetry is necessary for the development of a dentofacial diagnosis in orthodontics, and an understanding of individual differences in perception of face symmetry between patients and providers is needed to facilitate successful treatment. METHODS: Orthodontists, general dentists, and control participants completed a series of tasks to assess symmetry. Judgments were made on pairs of upright faces (similar to the longitudinal assessment of photographic patient records), inverted faces, and dot patterns. Participants completed questionnaires regarding clinical practice, education level, and self-confidence ratings for symmetry assessment abilities. RESULTS: Orthodontists showed expertise compared with controls (P <0.001), whereas dentists showed no advantage over controls. Orthodontists performed better than dentists, however, in only the most difficult face symmetry judgments (P = 0.006). For both orthodontists and dentists, accuracy increased significantly when assessing symmetry in upright vs inverted faces (t = 3.7, P = 0.001; t = 2.7, P = 0.02, respectively). CONCLUSIONS: Orthodontists showed expertise in assessing face symmetry compared with both laypersons and general dentists, and they were more accurate when judging upright than inverted faces. When using accurate longitudinal photographic records to assess changing face symmetry, orthodontists are likely to be incorrect in less than 15% of cases, suggesting that assistance from some additional technology is infrequently needed for diagnosis.

A bayesian optimal foraging model of human visual search.

Real-world visual searches often contain a variable and unknown number of targets. Such searches present difficult metacognitive challenges, as searchers must decide when to stop looking for additional targets, which results in high miss rates in multiple-target searches. In the study reported here, we quantified human strategies in multiple-target search via an ecological optimal foraging model and investigated whether searchers adapt their strategies to complex target-distribution statistics. Separate groups of individuals searched displays with the number of targets per trial sampled from different geometric distributions but with the same overall target prevalence. As predicted by optimal foraging theory, results showed that individuals searched longer when they expected more targets to be present and adjusted their expectations on-line during each search by taking into account the higher-order, across-trial target distributions. However, compared with modeled ideal observers, participants systematically responded as if the target distribution were more uniform than it was, which suggests that training could improve multiple-target search performance.

Visual search performance is predicted by both prestimulus and poststimulus electrical brain activity.

An individual's performance on cognitive and perceptual tasks varies considerably across time and circumstances. We investigated neural mechanisms underlying such performance variability using regression-based analyses to examine trial-by-trial relationships between response times (RTs) and different facets of electrical brain activity. Thirteen participants trained five days on a color-popout visual-search task, with EEG recorded on days one and five. The task was to find a color-popout target ellipse in a briefly presented array of ellipses and discriminate its orientation. Later within a session, better preparatory attention (reflected by less prestimulus Alpha-band oscillatory activity) and better poststimulus early visual responses (reflected by larger sensory N1 waves) correlated with faster RTs. However, N1 amplitudes decreased by half throughout each session, suggesting adoption of a more efficient search strategy within a session. Additionally, fast RTs were preceded by earlier and larger lateralized N2pc waves, reflecting faster and stronger attentional orienting to the targets. Finally, SPCN waves associated with target-orientation discrimination were smaller for fast RTs in the first but not the fifth session, suggesting optimization with practice. Collectively, these results delineate variations in visual search processes that change over an experimental session, while also pointing to cortical mechanisms underlying performance in visual search.

What can 1 billion trials tell us about visual search?

Mobile technology (e.g., smartphones and tablets) has provided psychologists with a wonderful opportunity: through careful design and implementation, mobile applications can be used to crowd source data collection. By garnering massive amounts of data from a wide variety of individuals, it is possible to explore psychological questions that have, to date, been out of reach. Here we discuss 2 examples of how data from the mobile game Airport Scanner (Kedlin Co., http://www.airportscannergame.com) can be used to address questions about the nature of visual search that pose intractable problems for laboratory-based research. Airport Scanner is a successful mobile game with millions of unique users and billions of individual trials, which allows for examining nuanced visual search questions. The goals of the current Observation Report were to highlight the growing opportunity that mobile technology affords psychological research and to provide an example roadmap of how to successfully collect usable data.

Context matters: the structure of task goals affects accuracy in multiple-target visual search.

Career visual searchers such as radiologists and airport security screeners strive to conduct accurate visual searches, but despite extensive training, errors still occur. A key difference between searches in radiology and airport security is the structure of the search task: Radiologists typically scan a certain number of medical images (fixed objective), and airport security screeners typically search X-rays for a specified time period (fixed duration). Might these structural differences affect accuracy? We compared performance on a search task administered either under constraints that approximated radiology or airport security. Some displays contained more than one target because the presence of multiple targets is an established source of errors for career searchers, and accuracy for additional targets tends to be especially sensitive to contextual conditions. Results indicate that participants searching within the fixed objective framework produced more multiple-target search errors; thus, adopting a fixed duration framework could improve accuracy for career searchers.

Enhanced Facial Symmetry Assessment in Orthodontists.

Assessing facial symmetry is an evolutionarily important process, which suggests that individual differences in this ability should exist. As existing data are inconclusive, the current study explored whether a group trained in facial symmetry assessment, orthodontists, possessed enhanced abilities. Symmetry assessment was measured using face and non-face stimuli among orthodontic residents and two control groups: university participants with no symmetry training and airport security luggage screeners, a group previously shown to possess expert visual search skills unrelated to facial symmetry. Orthodontic residents were more accurate at assessing symmetry in both upright and inverted faces compared to both control groups, but not for non-face stimuli. These differences are not likely due to motivational biases or a speed-accuracy tradeoff-orthodontic residents were slower than the university participants but not the security screeners. Understanding such individual differences in facial symmetry assessment may inform the perception of facial attractiveness.

Overcoming hurdles in translating visual search research between the lab and the field.

Research in visual search can be vital to improving performance in careers such as radiology and airport security screening. In these applied, or "field," searches, accuracy is critical, and misses are potentially fatal; however, despite the importance of performing optimally, radiological and airport security searches are nevertheless flawed. Extensive basic research in visual search has revealed cognitive mechanisms responsible for successful visual search as well as a variety of factors that tend to inhibit or improve performance. Ideally, the knowledge gained from such laboratory-based research could be directly applied to field searches, but several obstacles stand in the way of straightforward translation; the tightly controlled visual searches performed in the lab can be drastically different from field searches. For example, they can differ in terms of the nature of the stimuli, the environment in which the search is taking place, and the experience and characteristics of the searchers themselves. The goal of this chapter is to discuss these differences and how they can present hurdles to translating lab-based research to field-based searches. Specifically, most search tasks in the lab entail searching for only one target per trial, and the targets occur relatively frequently, but field searches may contain an unknown and unlimited number of targets, and the occurrence of targets can be rare. Additionally, participants in lab-based search experiments often perform under neutral conditions and have no formal training or experience in search tasks; conversely, career searchers may be influenced by the motivation to perform well or anxiety about missing a target, and they have undergone formal training and accumulated significant experience searching. This chapter discusses recent work that has investigated the impacts of these differences to determine how each factor can influence search performance. Knowledge gained from the scientific exploration of search can be applied to field searches but only when considering and controlling for the differences between lab and field.

Enhanced change detection performance reveals improved strategy use in avid action video game players.

Recent research has shown that avid action video game players (VGPs) outperform non-video game players (NVGPs) on a variety of attentional and perceptual tasks. However, it remains unknown exactly why and how such differences arise; while some prior research has demonstrated that VGPs' improvements stem from enhanced basic perceptual processes, other work indicates that they can stem from enhanced attentional control. The current experiment used a change-detection task to explore whether top-down strategies can contribute to VGPs' improved abilities. Participants viewed alternating presentations of an image and a modified version of the image and were tasked with detecting and localizing the changed element. Consistent with prior claims of enhanced perceptual abilities, VGPs were able to detect the changes while requiring less exposure to the change than NVGPs. Further analyses revealed this improved change detection performance may result from altered strategy use; VGPs employed broader search patterns when scanning scenes for potential changes. These results complement prior demonstrations of VGPs' enhanced bottom-up perceptual benefits by providing new evidence of VGPs' potentially enhanced top-down strategic benefits.