Image memorability influences memory for where the item was seen but not when.

Observers can determine whether they have previously seen hundreds of images with more than 80% accuracy. This "massive memory" for WHAT we have seen is accompanied by smaller but still massive memories for WHERE and WHEN the item was seen (spatial & temporal massive memory). Recent studies have shown that certain images are more easily remembered than others (higher "memorability"). Does memorability influence spatial massive memory and temporal massive memory? In two experiments, viewers saw 150 images presented twice in random order. These 300 images were sequentially presented at random locations in a 7 × 7 grid. If an image was categorized as old, observers clicked on the spot in the grid where they thought they had previously seen it. They also noted when they had seen it: Experiment 1-clicking on a timeline; Experiment 2-estimating the trial number when the item first appeared. Replicating prior work, data show that high-memorability images are remembered better than low-memorability images. Interestingly, in both experiments, spatial memory precision was correlated with image memorability, while temporal memory precision did not vary as a function of memorability. Apparently, properties that make images memorable help us remember WHERE but not WHEN those images were presented. The lack of correlation between memorability and temporal memory is, of course, a negative result and should be treated with caution.

In simple but challenging search tasks, most errors are stochastic.

In visual search tasks in the lab and in the real world, people routinely miss targets that are clearly visible: so-called look but fail to see (LBFTS) errors. If search displays are shown to the same observer twice, we can ask about the probability of joint errors, where the target is missed both times. If errors are "deterministic," then the probability of a second error on the same display-given that the target was missed the first time-should be high. If errors are "stochastic," the probability of joint errors should be the product of the error rate for first and second appearances. Here, we report on two versions of a T among Ls search with somewhat degraded letters to make search more difficult. In Experiment 1, Ts could either appear amidst crowded "clumps" of Ls or more in isolation. Observers made more errors when the T was in a clump, but these errors were mainly stochastic. In Experiment 2, the task was made harder by making Ts and Ls more similar. Again, errors were predominantly stochastic. If other, socially important errors are also stochastic, this would suggest that "double reading," where two observers (human or otherwise) look at each stimulus, could reduce overall error rates.

Research on re-searching: interrupted foraging is not disrupted foraging.

In classic visual search, observers typically search for the presence of a target in a scene or display. In foraging tasks, there may be multiple targets in the same display (or "patch"). Observers typically search for and collect these target items in one patch until they decide to leave that patch and move to the next one. This is a highly rule-governed behavior. The current study investigated whether these rules are disrupted when the foraging is interrupted in various manners. In Experiment 1, the foraging was briefly interrupted and then resumed in the same patch. In Experiments 2 and 3, the foraging in each patch either ended voluntarily or compulsorily after a fixed amount of time. In these cases, foraging resumed in a patch only after all patches were visited. Overall, the rules of foraging remained largely intact, though Experiment 2 shows that foraging rules can be overridden by the demand characteristics of the task. The results show that participants tended to perform approximately consistently despite interruptions. The results suggest that foraging behavior in a relatively simple foraging environment is resilient and not easily disrupted by interruption.

Errors in visual search: Are they stochastic or deterministic?

In any visual search task in the lab or in the world, observers will make errors. Those errors can be categorized as "deterministic": If you miss this target in this display once, you will definitely miss it again. Alternatively, errors can be "stochastic", occurring randomly with some probability from trial to trial. Researchers and practitioners have sought to reduce errors in visual search, but different types of errors might require different techniques for mitigation. To empirically categorize errors in a simple search task, our observers searched for the letter "T" among "L" distractors, with each display presented twice. When the letters were clearly visible (white letters on a gray background), the errors were almost completely stochastic (Exp 1). An error made on the first appearance of a display did not predict that an error would be made on the second appearance. When the visibility of the letters was manipulated (letters of different gray levels on a noisy background), the errors became a mix of stochastic and deterministic. Unsurprisingly, lower contrast targets produced more deterministic errors. (Exp 2). Using the stimuli of Exp 2, we tested whether errors could be reduced using cues that guided attention around the display but knew nothing about the content of that display (Exp3a, b). This had no effect, but cueing all item locations did succeed in reducing deterministic errors (Exp3c).

Toward viewing behavior for aerial scene categorization.

Previous work has demonstrated similarities and differences between aerial and terrestrial image viewing. Aerial scene categorization, a pivotal visual processing task for gathering geoinformation, heavily depends on rotation-invariant information. Aerial image-centered research has revealed effects of low-level features on performance of various aerial image interpretation tasks. However, there are fewer studies of viewing behavior for aerial scene categorization and of higher-level factors that might influence that categorization. In this paper, experienced subjects' eye movements were recorded while they were asked to categorize aerial scenes. A typical viewing center bias was observed. Eye movement patterns varied among categories. We explored the relationship of nine image statistics to observers' eye movements. Results showed that if the images were less homogeneous, and/or if they contained fewer or no salient diagnostic objects, viewing behavior became more exploratory. Higher- and object-level image statistics were predictive at both the image and scene category levels. Scanpaths were generally organized and small differences in scanpath randomness could be roughly captured by critical object saliency. Participants tended to fixate on critical objects. Image statistics included in this study showed rotational invariance. The results supported our hypothesis that the availability of diagnostic objects strongly influences eye movements in this task. In addition, this study provides supporting evidence for Loschky et al.'s (Journal of Vision, 15(6), 11, 2015) speculation that aerial scenes are categorized on the basis of image parts and individual objects. The findings were discussed in relation to theories of scene perception and their implications for automation development.

What is the basis of ensemble subset selection?

The visual system can rapidly calculate the ensemble statistics of a set of objects; for example, people can easily estimate an average size of apples on a tree. To accomplish this, it is not always useful to summarize all the visual information. If there are various types of objects, the visual system should select a relevant subset: only apples, not leaves and branches. Here, we ask what kind of visual information makes a "good" ensemble that can be selectively attended to provide an accurate summary estimate. We tested three candidate representations: basic features, preattentive object files, and full-fledged bound objects. In four experiments, we presented a target and several distractors' sets of differently colored objects. We found that conditions where a target ensemble had at least one unique color (basic feature) provided ensemble averaging performance comparable to the baseline displays without distractors. When the target subset was defined as a conjunction of two colors or color-shape partly shared with distractors (so that they could be differentiated only as preattentive object files), subset averaging was also possible but less accurate than in the baseline and feature conditions. Finally, performance was very poor when the target subset was defined by an exact feature relationship, such as in the spatial conjunction of two colors (spatially bound object). Overall, these results suggest that distinguishable features and, to a lesser degree, preattentive object files can serve as the representational basis of ensemble selection, while bound objects cannot.

Hybrid visual and memory search for scenes and objects with variable viewpoints.

In hybrid search, observers search visual arrays for any of several target types held in memory. The key finding in hybrid search is that response times (RTs) increase as a linear function of the number of items in a display (visual set size), but RTs increase linearly with the log of the memory set size. Previous experiments have shown this result for specific targets (find exactly this picture of a boot on a blank background) and for broad categorical targets (find any animal). Arguably, these are rather unnatural situations. In the real world, objects are parts of scenes and are seen from multiple viewpoints. The present experiments generalize the hybrid search findings to scenes (Experiment 1) and multiple viewpoints (Experiment 2). The results replicated the basic pattern of hybrid search results: RTs increased logarithmically with the number of scene photos/categories held in memory. Experiment 3 controls the experiment for which viewpoints were seen in an initial learning phase. The results replicate the findings of Experiment 2. Experiment 4 compares hybrid search for specific viewpoints, variable viewpoints, and categorical targets. Search difficulty increases from specific viewpoints to variable viewpoints and then to categorical targets. The results of the four experiments show the generality of logarithmic search through memory in hybrid search.

Reliability of radiologists' first impression when interpreting a screening mammogram.

Previous studies showed that radiologists can detect the gist of an abnormality in a mammogram based on a half-second image presentation through global processing of screening mammograms. This study investigated the intra- and inter-observer reliability of the radiologists' initial impressions about the abnormality (or "gist signal"). It also examined if a subset of radiologists produced more reliable and accurate gist signals. Thirty-nine radiologists provided their initial impressions on two separate occasions, viewing each mammogram for half a second each time. The intra-class correlation (ICC) values showed poor to moderate intra-reader reliability. Only 13 radiologists had an ICC of 0.6 or above, which is considered the minimum standard for reliability, and only three radiologists had an ICC exceeding 0.7. The median value for the weighted Cohen's Kappa was 0.478 (interquartile range = 0.419-0.555). The Mann-Whitney U-test showed that the "Gist Experts", defined as those who outperformed others, had significantly higher ICC values (p = 0.002) and weighted Cohen's Kappa scores (p = 0.026). However, even for these experts, the intra-radiologist agreements were not strong, as an ICC of at least 0.75 indicates good reliability and the signal from none of the readers reached this level of reliability as determined by ICC values. The inter-reader reliability of the gist signal was poor, with an ICC score of 0.31 (CI = 0.26-0.37). The Fleiss Kappa score of 0.106 (CI = 0.105-0.106), indicating only slight inter-reader agreement, confirms the findings from the ICC analysis. The intra- and inter-reader reliability analysis showed that the radiologists' initial impressions are not reliable signals. In particular, the absence of an abnormal gist does not reliably signal a normal case, so radiologists should keep searching. This highlights the importance of "discovery scanning," or coarse screening to detect potential targets before ending the visual search.

How do people find pairs?

Humans continuously scan their visual environment for relevant information. Such visual search behavior has typically been studied with tasks in which the search goal is constant and well-defined, requiring relatively little interplay between memory and orienting. Here we studied a situation in which the target is not known in advance, and instead, memory needs to be dynamically updated during the actual search. Observers compared two simultaneously presented arrays of objects for any matching pair of items-a task that requires continuous comparisons between what is seen now and what was seen a few moments ago. To manipulate the balance between memorizing and scanning, we ran two versions of the task. In an eye-tracking version, the objects were continuously available and could be scanned with relative ease. The results suggested that observers preferred scanning over memorizing. In a mouse-tracking version, perceptual availability was limited, and scanning was slowed. Now observers substantially increased their memory use. Thus, the results revealed a flexible and dynamic interplay between memory and perception. The findings aid in further bridging the research fields of attention and memory. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Mirror blindness: Our failure to recognize the target in search for mirror-reversed shapes.

It is well known that visual search for a mirror target (i.e., a horizontally flipped item) is more difficult than search for other-oriented items (e.g., vertically flipped items). Previous studies have typically attributed costs of mirror search to early, attention-guiding processes but could not rule out contributions from later processes. In the present study we used eye tracking to distinguish between early, attention-guiding processes and later target identification processes. The results of four experiments revealed a marked human weakness in identifying mirror targets: Observers appear to frequently fail to classify a mirror target as a target on first fixation and to continue with search even after having directly looked at the target. Awareness measures corroborated that the location of a mirror target could not be reported above chance level after it had been fixated once. This mirror blindness effect explained a large proportion (45-87%) of the overall costs of mirror search, suggesting that part of the difficulties with mirror search are rooted in later, object identification processes (not attentional guidance). Mirror blindness was significantly reduced but not completely eliminated when both the target and non-targets were held constant, which shows that perfect top-down knowledge can reduce mirror blindness, without completely eliminating it. The finding that non-target certainty reduced mirror blindness suggests that object identification is not solely achieved by comparing a selected item to a target template. These results demonstrate that templates that guide search toward targets are not identical to the templates used to conclusively identify those targets.

Spatial and temporal massive memory in humans.

It is well known that humans have a massive memory for pictures and scenes.1,2,3,4 They show an ability to encode thousands of images with only a few seconds of exposure to each. In addition to this massive memory for "what" observers have seen, three experiments reported here show that observers have a "spatial massive memory" (SMM) for "where" stimuli have been seen and a "temporal massive memory" (TMM) for "when" stimuli have been seen. The positions in time and space for at least dozens of items can be reported with good, if not perfect accuracy. Previous work has suggested that there might be good memory for stimulus location,5,6 but there do not seem to have been concerted efforts to measure the extent of this memory. Moreover, in our method, observers are recalling where items were located and not merely recognizing the correct location. This is interesting because massive memory is sometimes thought to be limited to recognition tasks based on sense of familiarity.

Taking prevalence effects on the road: Rare hazards are often missed.

Previous work has shown that, in many visual search and detection tasks, observers frequently miss rare but important targets, like weapons in bags or abnormalities in radiological images. These prior studies of the low-prevalence effect (LPE) use static stimuli and typically permitted observers to search at will. In contrast, many real-world tasks, like looking for dangerous behavior on the road, only afford observers a brief glimpse of a complex, changing scene before they must make a decision. Can the LPE be a factor in in dynamic, time-limited moments of real driving? To test this, we developed a novel hazard-detection task that preserves much of the perceptual richness and complexity of hazard detection in the real world, while allowing for experimental control over event prevalence. Observers viewed brief video clips of road scenes recorded from dashboard cameras and reported whether they saw a hazardous event. In separate sessions, the prevalence of these events was either high (50% of videos) or low (4%). Under low prevalence, observers missed hazards at more than twice the rate observed in the high-prevalence condition. Follow-up experiments demonstrate that this elevation of miss rate at low prevalence persists when participants were allowed to correct their responses, increases as hazards become increasingly rare (down to 1% prevalence) and is resistant to simple cognitive intervention (participant prebriefing). Together, our results demonstrate that the LPE generalizes to complex perceptual decisions in dynamic natural scenes, such as driving, where observers must monitor and respond to rare hazards.

What eye tracking can tell us about how radiologists use automated breast ultrasound.

Purpose: Automated breast ultrasound (ABUS) presents three-dimensional (3D) representations of the breast in the form of stacks of coronal and transverse plane images. ABUS is especially useful for the assessment of dense breasts. Here, we present the first eye tracking data showing how radiologists search and evaluate ABUS cases. Approach: Twelve readers evaluated single-breast cases in 20-min sessions. Positive findings were present in 56% of the evaluated cases. Eye position and the currently visible coronal and transverse slice were tracked, allowing for reconstruction of 3D "scanpaths." Results: Individual readers had consistent search strategies. Most readers had strategies that involved examination of all available images. Overall accuracy was 0.74 (sensitivity = 0.66 and specificity = 0.84). The 20 false negative errors across all readers can be classified using Kundel's (1978) taxonomy: 17 are "decision" errors (readers found the target but misclassified it as normal or benign). There was one recognition error and two "search" errors. This is an unusually high proportion of decision errors. Readers spent essentially the same proportion of time viewing coronal and transverse images, regardless of whether the case was positive or negative, correct or incorrect. Readers tended to use a "scanner" strategy when viewing coronal images and a "driller" strategy when viewing transverse images. Conclusions: These results suggest that ABUS errors are more likely to be errors of interpretation than of search. Further research could determine if readers' exploration of all images is useful or if, in some negative cases, search of transverse images is redundant following a search of coronal images.

Normal blindness: when we Look But Fail To See.

Humans routinely miss important information that is 'right in front of our eyes', from overlooking typos in a paper to failing to see a cyclist in an intersection. Recent studies on these 'Looked But Failed To See' (LBFTS) errors point to a common mechanism underlying these failures, whether the missed item was an unexpected gorilla, the clearly defined target of a visual search, or that simple typo. We argue that normal blindness is the by-product of the limited-capacity prediction engine that is our visual system. The processes that evolved to allow us to move through the world with ease are virtually guaranteed to cause us to miss some significant stimuli, especially in important tasks like driving and medical image perception.

Advancing Research on Medical Image Perception by Strengthening Multidisciplinary Collaboration.

Medical image interpretation is central to detecting, diagnosing, and staging cancer and many other disorders. At a time when medical imaging is being transformed by digital technologies and artificial intelligence, understanding the basic perceptual and cognitive processes underlying medical image interpretation is vital for increasing diagnosticians' accuracy and performance, improving patient outcomes, and reducing diagnostician burnout. Medical image perception remains substantially understudied. In September 2019, the National Cancer Institute convened a multidisciplinary panel of radiologists and pathologists together with researchers working in medical image perception and adjacent fields of cognition and perception for the "Cognition and Medical Image Perception Think Tank." The Think Tank's key objectives were to identify critical unsolved problems related to visual perception in pathology and radiology from the perspective of diagnosticians, discuss how these clinically relevant questions could be addressed through cognitive and perception research, identify barriers and solutions for transdisciplinary collaborations, define ways to elevate the profile of cognition and perception research within the medical image community, determine the greatest needs to advance medical image perception, and outline future goals and strategies to evaluate progress. The Think Tank emphasized diagnosticians' perspectives as the crucial starting point for medical image perception research, with diagnosticians describing their interpretation process and identifying perceptual and cognitive problems that arise. This article reports the deliberations of the Think Tank participants to address these objectives and highlight opportunities to expand research on medical image perception.

Top-down control of attention by stereoscopic depth.

Stereoscopic depth has a mixed record as a guiding attribute in visual attention. Visual search can be efficient if the target lies at a unique depth; whereas automatic segmentation of search arrays into different depth planes does not appear to be pre-attentive. These prior findings describe bottom-up, stimulus-driven depth guidance. Here, we ask about the top-down selection of depth information. To assess the ability to direct attention to specific depth planes, Experiment 1 used the centroid judgment paradigm which permits quantitative measures of selective processing of items of different depths or colors. Experiment 1 showed that a subset of observers could deploy specific attention filters for each of eight depth planes, suggesting that at least some observers can direct attention to a specific depth plane quite precisely. Experiment 2 used eight depth planes in a visual search experiment. Observers were encouraged to guide their attention to far or near depth planes with an informative but imperfect cue. The benefits of this probabilistic cue were small. However, this may not be a specific problem with guidance by stereoscopic depth. Equivalently poor results were obtained with color. To check and prove that depth guidance in search is possible, Experiment 3 presented items in only two depth planes. In this case, information about the target depth plane allows observers to search more efficiently, replicating earlier work. We conclude that top-down guidance by stereoscopic depth is possible but that it is hard to apply the full range of our stereoscopic ability in search.

Priming effects in inefficient visual search: Real, but transient.

In visual search tasks, responses to targets on one trial can influence responses on the next trial. Most typically, target repetition speeds response while switching to a different target slows response. Such "priming" effects have sometimes been given very significant roles in theories of search (e.g., Theeuwes, Philosophical Transactions of the Royal Society B: Biological Sciences, 368, 1628, 2013). Most work on priming has involved "singleton" or "popout" tasks. In non-popout priming tasks, observers must often perform a task-switching operation because the guiding template for one target (e.g., a red vertical target in a conjunction task) is incompatible with efficient search for the other target (green horizontal, in this example). We examined priming in inefficient search where the priming feature (Color: Experiments 1-3, Shape: Experiments 4-5) was irrelevant to the task of finding a T among Ls. We wished to determine if finding a red T on one trial helped observers to be more efficient if the next T was also red. In all experiments, we found additive priming effects. The reaction time (RT) for the second trial was shorter if the color of the T was repeated. However, there was no interaction with set size. The slope of the RT × Set Size function was not shallower for runs of the same target color, compared to trials where the target color switched. We propose that priming might produce transient guidance of the earliest deployments of attention on the next trial or it might speed decisions about a selected target. Priming does not appear to guide attention over the entire search.

How one block of trials influences the next: persistent effects of disease prevalence and feedback on decisions about images of skin lesions in a large online study.

Using an online, medical image labeling app, 803 individuals rated images of skin lesions as either "melanoma" (skin cancer) or "nevus" (a skin mole). Each block consisted of 80 images. Blocks could have high (50%) or low (20%) target prevalence and could provide full, accurate feedback or no feedback. As in prior work, with feedback, decision criteria were more conservative at low prevalence than at high prevalence and resulted in more miss errors. Without feedback, this low prevalence effect was reversed (albeit, not significantly). Participants could participate in up to four different conditions a day on each of 6 days. Our main interest was in the effect of Block N on Block N + 1. Low prevalence with feedback made participants more conservative on a subsequent block. High prevalence with feedback made participants more liberal on a subsequent block. Conditions with no feedback had no significant impact on the subsequent block. The delay between Blocks 1 and 2 had no significant effect. The effect on the second half of Block 2 was just as large as on the first half. Medical expertise (over the range available in the study) had no impact on these effects, though medical students were better at the task than other groups. Overall, these seem to be robust effects where feedback may be 'teaching' participants how to respond in the future. This might have application in, for example, training or re-training situations.

Hiding the Rabbit: Using a genetic algorithm to investigate shape guidance in visual search.

During visual search, attention is guided by specific features, including shape. Our understanding of shape guidance is limited to specific attributes (closures and line terminations) that do not fully explain the richness of preattentive shape processing. We used a novel genetic algorithm method to explore shape space and to stimulate hypotheses about shape guidance. Initially, observers searched for targets among 12 random distractors defined, in radial frequency space, by the amplitude and phase of 10 radial frequencies. Reaction time (RT) was the measure of "fitness." To evolve toward an easier search task, distractors with faster RTs survived to the next generation, "mated," and produced offspring (new distractors for the next generation of search). To evolve a harder search, surviving distractors were those yielding longer RTs. Within eight generations of evolution, the method succeeds in producing visual searches either harder or easier than the starting search. In radial frequency space, easy distractors evolve amplitude × frequency spectra that are dissimilar to the target, whereas hard distractors evolve spectra that are more similar to the target. This method also works with naturally shaped targets (e.g., rabbit silhouettes). Interestingly, the most inefficient distractors featured a combination of a body and ear distractors that did not resemble the rabbit (visually or in spectrum). Adding extra ears to these distractors did not impact the search spectrally and instead made it easier to confirm a rabbit, once it was found. In general, these experiments show that shapes that are clearly distinct when attended are similar to each other preattentively.

Right place, right time: Spatiotemporal predictions guide attention in dynamic visual search.

Visual search is a fundamental element of human behavior and is predominantly studied in a laboratory setting using static displays. However, real-life search is often an extended process taking place in dynamic environments. We have designed a dynamic-search task in order to incorporate the temporal dimension into visual search. Using this task, we tested how participants learn and utilize spatiotemporal regularities embedded within the environment to guide performance. Participants searched for eight instances of a target that faded in and out of a display containing similarly transient distractors. In each trial, four of the eight targets appeared in a temporally predictable fashion with one target appearing in each of four spatially separated quadrants. The other four targets were spatially and temporally unpredictable. Participants' performance was significantly better for spatiotemporally predictable compared to unpredictable targets (Experiments 1-4). The effects were reliable over different patterns of spatiotemporal predictability (Experiment 2) and primarily reflected long-term learning over trials (Experiments 3, 4), although single-trial priming effects also contributed (Experiment 4). Eye-movement recordings (Experiment 1) revealed that spatiotemporal regularities guide attention proactively and dynamically. Taken together, our results show that regularities across both space and time can guide visual search and this guidance can primarily be attributed to robust long-term representations of these regularities. (PsycInfo Database Record (c) 2022 APA, all rights reserved).