Inter-individual variability (but intra-individual stability) of overt versus covert rehearsal strategies in a digital Corsi task.

The Corsi (block-tapping) paradigm is a classic and well-established visuospatial working memory task in humans involving internal computations (memorizing of item sequences, organizing and updating the memorandum, and recall processes), as well as both overt and covert shifts of attention to facilitate rehearsal, serving to maintain the Corsi sequences during the retention phase. Here, we introduce a novel digital version of a Corsi task in which i) the difficulty of the memorandum (using sequence lengths ranging from 3 to 8) was controlled, ii) the execution of overt and/or covert attention as well as the visuospatial working memory load during the retention phase was manipulated, and iii) shifts of attention were quantified in all experimental phases. With this, we present behavioral data that demonstrate, characterize, and classify the individual effects of overt and covert strategies used as a means of encoding and rehearsal. In a full within-subject design, we tested 28 participants who had to solve three different Corsi conditions. While in condition A neither of the two strategies were restricted, in condition B the overt and in condition C the overt as well as the covert strategies were suppressed. Analyzing Corsi span, (eye) exploration index, and pupil size (change), data clearly show a continuum between overt and covert strategies over all participants (indicating inter-individual variability). Further, all participants showed stable strategy choice (indicating intra-individual stability), meaning that the preferred strategy was maintained in all three conditions, phases, and sequence lengths of the experiment.

Modality dependence and intermodal transfer in the Corsi Spatial Sequence Task: Screen vs. Floor.

Four versions of the Corsi Spatial Sequence Task (CSST) were tested in a complete within-subject design, investigating whether participants' performance depends on the modality of task presentation and reproduction that put different demands on spatial processing. Presentation of the sequence (encoding phase) and the reproduction (recall phase) were each carried out either on a computer screen or on the floor of a room, involving actual walking in the recall phase. Combinations of the two different encoding and recall procedures result in the modality conditions Screen-Screen, Screen-Floor, Floor-Screen, and Floor-Floor. Results show the expected decrease in performance with increasing sequence length, which is likely due to processing limitations of working memory. We also found differences in performance between the modality conditions indicating different involvements of spatial working memory processes. Participants performed best in the Screen-Screen modality condition. Floor-Screen and Floor-Floor modality conditions require additional working memory resources for reference frame transformation and spatial updating, respectively; the resulting impairment of the performance was about the same in these two conditions. Finally, the Screen-Floor modality condition requires both types of additional spatial demands and led to the poorest performance. Therefore, we suggest that besides the well-known spatial requirements of CSST, additional working memory resources are demanded in walking CSST supporting processes such as spatial updating, mental rotation, reference frame transformation, and the control of walking itself.

The perception of ego-motion change in environments with varying depth: Interaction of stereo and optic flow.

When estimating ego-motion in environments (e.g., tunnels, streets) with varying depth, human subjects confuse ego-acceleration with environment narrowing and ego-deceleration with environment widening. Festl, Recktenwald, Yuan, and Mallot (2012) demonstrated that in nonstereoscopic viewing conditions, this happens despite the fact that retinal measurements of acceleration rate-a variable related to tau-dot-should allow veridical perception. Here we address the question of whether additional depth cues (specifically binocular stereo, object occlusion, or constant average object size) help break the confusion between narrowing and acceleration. Using a forced-choice paradigm, the confusion is shown to persist even if unambiguous stereo information is provided. The confusion can also be demonstrated in an adjustment task in which subjects were asked to keep a constant speed in a tunnel with varying diameter: Subjects increased speed in widening sections and decreased speed in narrowing sections even though stereoscopic depth information was provided. If object-based depth information (stereo, occlusion, constant average object size) is added, the confusion between narrowing and acceleration still remains but may be slightly reduced. All experiments are consistent with a simple matched filter algorithm for ego-motion detection, neglecting both parallactic and stereoscopic depth information, but leave open the possibility of cue combination at a later stage.

Interaction between image and text during the process of biblical art reception.

In our exploratory study, we ask how naive observers, without a distinct religious background, approach biblical art that combines image and text. For this purpose, we choose the book 'New biblical figures of the Old and New Testament' published in 1569 as source of the stimuli. This book belongs to the genre of illustrated Bibles, which were very popular during the Reformation. Since there is no empirical knowledge regarding the interaction between image and text during the process of such biblical art reception, we selected four relevant images from the book and measured the eye movements of participants in order to characterize and quantify their scanning behavior related to such stimuli in terms of i) looking at text (text usage), ii) text vs. image interaction measures (semantic or contextual relevance of text), and iii) narration. We show that texts capture attention early in the process of inspection and that text and image interact. Moreover, semantics of texts are used to guide eye movements later through the image, supporting the formation of the narrative.

Head and eye movements and the role of memory limitations in a visual search paradigm.

The image information guiding visual behavior is acquired and maintained in an interplay of gaze shifts and visual short-term memory (VSTM). If storage capacity of VSTM is exhausted, gaze shifts can be used to regain information not currently represented in memory. By varying the separation between relevant image regions, S. Inamdar and M. Pomplun (2003) demonstrated a trade-off between VSTM storage and gaze shifts, which were performed as pure eye movements, that is, without a head movement component. Here we extend this paradigm to larger gaze shifts involving both eye and head movements. We use a comparative visual search paradigm with two relevant image regions and region separation as independent variable. Image regions were defined by two cupboards displaying colored geometrical objects in roughly equal arrangements. Subjects were asked to find differences in the arrangement of the objects in the two cupboards. Cupboard separation was varied between 30 degrees and 120 degrees . Images were presented with two projectors on a 150 degrees x 70 degrees curved screen. Head and eye movements were simultaneously recorded with an ART head tracker and an ASL mobile eye tracker, respectively. In the large separation conditions, the number of gaze shifts between the two cupboards was reduced, while fixation duration increased. Furthermore, the head movement proportions negatively correlated with the number of gaze shifts and positively correlated with fixation duration. We conclude that the visual system uses increased VSTM involvement to avoid gaze movements and in particular movements of the head. Scan path analysis revealed two subject-specific strategies (encode left, compare right, and vice versa), which were consistently used in all separation conditions.

How to Best Name a Place? Facilitation and Inhibition of Route Learning Due to Descriptive and Arbitrary Location Labels.

Establishing verbal memory traces for non-verbal stimuli was reported to facilitate or inhibit memory for the non-verbal stimuli. We show that these effects are also observed in a domain not indicated before-wayfinding. Fifty-three participants followed a guided route in a virtual environment. They were asked to remember half of the intersections by relying on the visual impression only. At the other 50% of the intersections, participants additionally heard a place name, which they were asked to memorize. For testing, participants were teleported to the intersections and were asked to indicate the subsequent direction of the learned route. In Experiment 1, intersections' names were arbitrary (i.e., not related to the visual impression). Here, participants performed more accurately at unnamed intersections. In Experiment 2, intersections' names were descriptive and participants' route memory was more accurate at named intersections. Results have implications for naming places in a city and for wayfinding aids.

Allocation of cognitive resources in comparative visual search--individual and task dependent effects.

Behaviors recruit multiple, mutually substitutable types of cognitive resources (e.g., data acquisition and memorization in comparative visual search), and the allocation of resources is performed in a cost-optimizing way. If costs associated with each type of resource are manipulated, e.g., by varying the complexity of the items studied or the visual separation of the arrays to be compared, according adjustments of resource allocation ("trade-offs") have been demonstrated. Using between-subject designs, previous studies showed overall trade-off behavior but neglected inter-individual variability of trade-off behavior. Here, we present a simplified paradigm for comparative visual search in which gaze-measurements are replaced by switching of a visual mask covering one stimulus array at a time. This paradigm allows for a full within-subject design. While overall trade-off curves could be reproduced, we found that each subject used a specific trade-off strategy which differ substantially between subjects. Still, task-dependent adjustment of resource allocation can be demonstrated but accounts only for a minor part of the overall trade-off range. In addition, we show that the individual trade-offs were adjusted in an unconscious and rather intuitive way, enabling a robust manifestation of the selected strategy space.

View-based organization and interplay of spatial working and long-term memories.

Space perception provides egocentric, oriented views of the environment from which working and long-term memories are constructed. "Allocentric" (i.e. position-independent) long-term memories may be organized as graphs of recognized places or views but the interaction of such cognitive graphs with egocentric working memories is unclear. Here we present a simple coherent model of view-based working and long-term memories, together with supporting evidence from behavioral experiments. The model predicts (i) that within a given place, memories for some views may be more salient than others, (ii) that imagery of a target square should depend on the location where the recall takes place, and (iii) that recall favors views of the target square that would be obtained when approaching it from the current recall location. In two separate experiments in an outdoor urban environment, pedestrians were approached at various interview locations and asked to draw sketch maps of one of two well-known squares. Orientations of the sketch map productions depended significantly on distance and direction of the interview location from the target square, i.e. different views were recalled at different locations. Further analysis showed that location-dependent recall is related to the respective approach direction when imagining a walk from the interview location to the target square. The results are consistent with a view-based model of spatial long-term and working memories and their interplay.

Gaze movements and spatial working memory in collision avoidance: a traffic intersection task.

Street crossing under traffic is an everyday activity including collision detection as well as avoidance of objects in the path of motion. Such tasks demand extraction and representation of spatio-temporal information about relevant obstacles in an optimized format. Relevant task information is extracted visually by the use of gaze movements and represented in spatial working memory. In a virtual reality traffic intersection task, subjects are confronted with a two-lane intersection where cars are appearing with different frequencies, corresponding to high and low traffic densities. Under free observation and exploration of the scenery (using unrestricted eye and head movements) the overall task for the subjects was to predict the potential-of-collision (POC) of the cars or to adjust an adequate driving speed in order to cross the intersection without collision (i.e., to find the free space for crossing). In a series of experiments, gaze movement parameters, task performance, and the representation of car positions within working memory at distinct time points were assessed in normal subjects as well as in neurological patients suffering from homonymous hemianopia. In the following, we review the findings of these experiments together with other studies and provide a new perspective of the role of gaze behavior and spatial memory in collision detection and avoidance, focusing on the following questions: (1) which sensory variables can be identified supporting adequate collision detection? (2) How do gaze movements and working memory contribute to collision avoidance when multiple moving objects are present and (3) how do they correlate with task performance? (4) How do patients with homonymous visual field defects (HVFDs) use gaze movements and working memory to compensate for visual field loss? In conclusion, we extend the theory of collision detection and avoidance in the case of multiple moving objects and provide a new perspective on the combined operation of external (bottom-up) and internal (top-down) cues in a traffic intersection task.

Gaze patterns predicting successful collision avoidance in patients with homonymous visual field defects.

Aim of the present study was to identify efficient compensatory gaze patterns applied by patients with homonymous visual field defects (HVFDs) under virtual reality (VR) conditions in a dynamic collision avoidance task. Thirty patients with HVFDs due to vascular brain lesions and 30 normal subjects performed a collision avoidance task with moving objects at an intersection under two difficulty levels. Based on their performance (i.e. the number of collisions), patients were assigned to either an "adequate" (HVFD(A)) or "inadequate" (HVFD(I)) subgroup by the median split method. Eye and head tracking data were available for 14 patients and 19 normal subjects. Saccades, fixations, mean number of gaze shifts, scanpath length and the mean gaze eccentricity, were compared between HVFD(A), HVFD(I) patients and normal subjects. For both difficulty levels, the gaze patterns of HVFD(A) patients (N=5) compared to HVFD(I) patients (N=9) were characterized by longer saccadic amplitudes towards both the affected and the intact side, larger mean gaze eccentricity, more gaze shifts, longer scanpaths and more fixations on vehicles but fewer fixations on the intersection. Both patient groups displayed more fixations in the affected compared to the intact hemifield. Fixation number, fixation duration, scanpath length, and number of gaze shifts were similar between HVFD(A) patients and normal subjects. Patients with HVFDs who adapt successfully to their visual deficit, display distinct gaze patterns characterized by increased exploratory eye and head movements, particularly towards moving objects of interest on their blind side. In the context of a dynamic environment, efficient compensation in patients with HVFDs is possible by means of gaze scanning. This strategy allows continuous update of the moving objects' spatial location and selection of the task-relevant ones, which will be represented in visual working memory.

The neural correlates of impaired collision avoidance in hemianopic patients.

PURPOSE: The aim of this study was to assess the brain regions associated with impaired performance in a virtual, dynamic collision avoidance task, in a group of patients with homonymous visual field defects (HVFDs) because of unilateral vascular brain lesions. METHODS: Overall task performance was quantitatively assessed as the number of collisions while crossing an intersection at two levels of traffic density. Twenty-six patients were divided into two subgroups using the median split method: patients with 'performance above average' (HVFD(A), i.e. lower number of collisions) and patients with 'performance below average' (HVFD(B), i.e. higher number of collisions). In order to identify the anatomical structures that might be specifically affected in HVFD(B) patients but spared in HVFD(A) patients, overlap, subtraction and voxel-based lesion-symptom mapping analyses were performed using the MRIcron software. RESULTS: No significant difference in collision avoidance between patients with left- and right-hemispheric lesions was revealed. Separate lesion analysis in 12 patients with right- and 14 patients with left-hemispheric lesions showed that the cortical structures associated with impaired collision avoidance were the parieto-occipital region and posterior cingulate gyrus in the right hemisphere and the inferior occipital cortex and parts of the fusiform (occipito-temporal) gyrus in the left hemisphere. CONCLUSION: In the present collision avoidance paradigm, impaired performance of patients with right-hemispheric lesions is associated with damage in the dorsal processing stream and potential impact on the visual spatial working memory (WM), while impaired performance of patients with left-hemispheric lesions is associated with damage in the ventral stream and potential impact on the visual object WM.

Collision avoidance in persons with homonymous visual field defects under virtual reality conditions.

The aim of the present study was to examine the effect of homonymous visual field defects (HVFDs) on collision avoidance of dynamic obstacles at an intersection under virtual reality (VR) conditions. Overall performance was quantitatively assessed as the number of collisions at a virtual intersection at two difficulty levels. HVFDs were assessed by binocular semi-automated kinetic perimetry within the 90° visual field, stimulus III4e and the area of sparing within the affected hemifield (A-SPAR in deg(2)) was calculated. The effect of A-SPAR, age, gender, side of brain lesion, time since brain lesion and presence of macular sparing on the number of collisions, as well as performance over time were investigated. Thirty patients (10 female, 20 male, age range: 19-71 years) with HVFDs due to unilateral vascular brain lesions and 30 group-age-matched subjects with normal visual fields were examined. The mean number of collisions was higher for patients and in the more difficult level they experienced more collisions with vehicles approaching from the blind side than the seeing side. Lower A-SPAR and increasing age were associated with decreasing performance. However, in agreement with previous studies, wide variability in performance among patients with identical visual field defects was observed and performance of some patients was similar to that of normal subjects. Both patients and healthy subjects displayed equal improvement of performance over time in the more difficult level. In conclusion, our results suggest that visual-field related parameters per se are inadequate in predicting successful collision avoidance. Individualized approaches which also consider compensatory strategies by means of eye and head movements should be introduced.

Acquisition vs. memorization trade-offs are modulated by walking distance and pattern complexity in a large-scale copying paradigm.

In a "block-copying paradigm", subjects were required to copy a configuration of colored blocks from a model area to a distant work area, using additional blocks provided at an equally distant resource area. Experimental conditions varied between the inter-area separation (walking distance) and the complexity of the block patterns to be copied. Two major behavioral strategies were identified: in the memory-intensive strategy, subjects memorize large parts of the pattern and rebuild them without intermediate visits at the model area. In the acquisition-intensive strategy, subjects memorize one block at a time and return to the model after having placed this block. Results show that the frequency of the memory-intensive strategy is increased for larger inter-area separations (larger walking distances) and for simpler block patterns. This strategy-shift can be interpreted as the result of an optimization process or trade-off, minimizing combined, condition-dependent costs of the two strategies. Combined costs correlate with overall response time. We present evidence that for the memory-intensive strategy, costs correlate with model visit duration, while for the acquisition-intensive strategy, costs correlate with inter-area transition (i.e., walking) times.

Task-dependent representation of moving objects within working memory in obstacle avoidance.

PURPOSE: The purpose of the study was the quantitative analysis of the working memory representation of dynamic objects related to gaze movement behavior. METHODS: Eighteen subjects participated in a virtual street-crossing paradigm. The primary task was collisions avoidance. To investigate the representation format, during a sub-task subjects were asked to reconstruct the traffic scene from memory. RESULTS: The distribution of cars positioned during the sub-task reveals a task-dependent (i.e., collision-relevant) representation of about four cars. In contrast, analysis of gaze behavior did not show a preference for collision-prone cars. CONCLUSION: Subjects avoided collisions efficiently by applying a gaze strategy adequate to create a representation that fulfills the demands of the task. Collision-prone cars are more likely to be represented in memory, but not more likely to be fixated.

Functional compensation of visual field deficits in hemianopic patients under the influence of different task demands.

We investigated the task-specific role of eye and head movements as a compensatory strategy in patients with homonymous visual field deficits (HVFDs) and in age-matched normal controls. All participants were tested in two tasks, i.e. a dot counting (DC) task requiring mostly simple visual scanning and a cognitively more demanding comparative visual search (CVS) task. The CVS task involved recognition and memory of geometrical objects and their configuration in two test fields. Based on task performance, patients were assigned to one of two groups, "adequate" (HVFD(A)) and "inadequate" (HVFD(I)); the group definitions based on either task turned out to be identical. With respect to the gaze related parameters in the DC task we obtained results in agreement with previous studies: the gaze pattern of HVFD(A) patients and normal controls did not differ significantly, while HVFD(I) patients showed increased gaze movement activity. In contrast, for the more complex CVS task we identified a deviating pattern of compensatory strategy use. Adequately performing subjects, who had used the same gaze strategies as normals in the DC task, now changed to increased gaze movement activity that allowed coping with the increasing task demands. Inadequately performing patients switched to a novel pattern of compensatory behavior in the CVS task. Different compensatory strategies are discussed with respect to the task-specific demands (in particular working memory involvement), the specific behavioral deficits of the patients, and the corresponding brain lesions.

Assessment of vision-related quality of life in patients with homonymous visual field defects.

BACKGROUND AND PURPOSE: Homonymous visual field defects (HVFDs) are among the most common disorders that occur in the elderly after vascular brain damage and can have a major impact on quality of life (QOL). Aims of this study were to describe the vision-targeted, health-related QOL in patients with HVFDs after cerebrovascular lesion, and to determine the relationship between patients' self-reported difficulties and the characteristics of HVFDs in the binocular visual field. METHODS: The German version of the 25-item National Eye Institute Visual Functioning Questionnaire (NEI-VFQ-25) was used. NEI-VFQ-25 scores for patients were compared to reference values of healthy German subjects from Franke (Z Med Psychol 7:178-184, 1999). Extent and location of absolute HVFDs were assessed by binocular semi-automated kinetic perimetry (SKP) within the 90 degrees visual field. Correlations of the NEI-VFQ-25 scores of patients with the area of sparing within the affected hemifield (A-SPAR) were estimated by Spearman's r (s). RESULTS: The mean NEI-VFQ-25 composite score for 33 patients (time span after brain injury at least 6 months) was 77.1, which was significantly lower (p < 0.0001) than the reference value for 360 healthy subjects (composite score = 90.6), and this was also the case for general vision, near activities, vision specific mental health, driving, colour, and peripheral vision. The score for general health was also significantly lower in patients than in reference subjects (p < 0.0001). A weak correlation of the composite score with A-SPAR (r (s) = 0.38) was observed. CONCLUSIONS: Our findings indicate that detectable decrements in vision-targeted, health-related QOL are observed in patients with homonymous visual field loss. A relationship of the perceived visual functioning with objective parameters is by definition difficult; however, understanding what components of visual function affect certain visual tasks, would help in developing more efficient, clinical assessment strategies. The results reveal a tendency for increasing QOL with advancing size of the area of sparing within the affected hemifield (A-SPAR). The lack of a strong correlation between NEI-VFQ-25 subscales and A-SPAR suggests that an assessment of the visual field may not accurately reflect patients' perceived difficulty in visual tasks. Additional consideration of visual exploration via eye and head movements may improve the correlation between visual function and its perception.