Electrophysiological Correlates of Visual Memory Search.

In everyday life, we frequently engage in 'hybrid' visual and memory search, where we look for multiple items stored in memory (e.g., a mental shopping list) in our visual environment. Across three experiments, we used event-related potentials to better understand the contributions of visual working memory (VWM) and long-term memory (LTM) during the memory search component of hybrid search. Experiments 1 and 2 demonstrated that the FN400 - an index of LTM recognition - and the CDA -an index of VWM load - increased with memory set size (target load), suggesting that both VWM and LTM are involved in memory search, even when target load exceeds capacity limitations of VWM. In Experiment 3, we used these electrophysiological indices to test how categorical similarity of targets and distractors affects memory search. The CDA and FN400 were modulated by memory set size only if items resembled targets. This suggests that dissimilar distractor items can be rejected before eliciting a memory search. Together, our findings demonstrate the interplay of VWM and LTM processes during memory search for multiple targets.

Dissociable online integration processes in visual working memory.

Visual working memory has severe capacity limits, creating a bottleneck for active processing. A key way of mitigating this limitation is by chunking, i.e. compressing several pieces of information into one visual working memory representation. However, despite decades of research, chunking efficiency remains debated because of mixed evidence. We propose that there are actually 2 integration mechanisms: Grouping combines several objects to one representation, and object-unification merges the parts of a single object. Critically, we argue that the fundamental distinction between the 2 processes is their differential use of the pointer system, the indexing process connecting visual working memory representations with perception. In grouping, the objects that are represented together still maintain independent pointers, making integration costly but highly flexible. Conversely, object-unification fuses the pointers as well as the representations, with the single pointer producing highly efficient integration but blocking direct access to individual parts. We manipulated integration cues via task-irrelevant movement, and monitored visual working memory's online electrophysiological marker. Uniquely colored objects were flexibly grouped and ungrouped via independent pointers (experiment 1). If objects turned uniformly black, object-integration could not be undone (experiment 2), requiring visual working memory to reset before re-individuation. This demonstrates 2 integration levels (representational-merging versus pointer-compression) and establishes the dissociation between visual working memory representations and their underlying pointers.

From Image to Diagnosis: Characterizing Sources of Error in Histopathologic Interpretation.

An accurate histopathologic diagnosis on surgical biopsy material is necessary for the clinical management of patients and has important implications for research, clinical trial design/enrollment, and public health education. This study used a mixed methods approach to isolate sources of diagnostic error while residents and attending pathologists interpreted digitized breast biopsy slides. Ninety participants, including pathology residents and attending physicians at major United States medical centers reviewed a set of 14 digitized whole-slide images of breast biopsies. Each case had a consensus-defined diagnosis and critical region of interest (cROI) representing the most significant pathology on the slide. Participants were asked to view unmarked digitized slides, draw their participant region of interest (pROI), describe its features, and render a diagnosis. Participants' review behavior was tracked using case viewer software and an eye-tracking device. Diagnostic accuracy was calculated in comparison to the consensus diagnosis. We measured the frequency of errors emerging during 4 interpretive phases: (1) detecting the cROI, (2) recognizing its relevance, (3) using the correct terminology to describe findings in the pROI, and (4) making a diagnostic decision. According to eye-tracking data, trainees and attending pathologists were very likely (∼94% of the time) to find the cROI when inspecting a slide. However, trainees were less likely to consider the cROI relevant to their diagnosis. Pathology trainees (41% of cases) were more likely to use incorrect terminology to describe pROI features than attending pathologists (21% of cases). Failure to accurately describe features was the only factor strongly associated with an incorrect diagnosis. Identifying where errors emerge in the interpretive and/or descriptive process and working on building organ-specific feature recognition and verbal fluency in describing those features are critical steps for achieving competency in diagnostic decision making.

More scanning, but not zooming, is associated with diagnostic accuracy in evaluating digital breast pathology slides.

Diagnoses of medical images can invite strikingly diverse strategies for image navigation and visual search. In computed tomography screening for lung nodules, distinct strategies, termed scanning and drilling, relate to both radiologists' clinical experience and accuracy in lesion detection. Here, we examined associations between search patterns and accuracy for pathologists (N = 92) interpreting a diverse set of breast biopsy images. While changes in depth in volumetric images reveal new structures through movement in the z-plane, in digital pathology changes in depth are associated with increased magnification. Thus, "drilling" in radiology may be more appropriately termed "zooming" in pathology. We monitored eye-movements and navigation through digital pathology slides to derive metrics of how quickly the pathologists moved through XY (scanning) and Z (zooming) space. Prior research on eye-movements in depth has categorized clinicians as either "scanners" or "drillers." In contrast, we found that there was no reliable association between a clinician's tendency to scan or zoom while examining digital pathology slides. Thus, in the current work we treated scanning and zooming as continuous predictors rather than categorizing as either a "scanner" or "zoomer." In contrast to prior work in volumetric chest images, we found significant associations between accuracy and scanning rate but not zooming rate. These findings suggest fundamental differences in the relative value of information types and review behaviors across two image formats. Our data suggest that pathologists gather critical information by scanning on a given plane of depth, whereas radiologists drill through depth to interrogate critical features.

Lost in the supermarket: Quantifying the cost of partitioning memory sets in hybrid search.

The items on a memorized grocery list are not relevant in every aisle; for example, it is useless to search for the cabbage in the cereal aisle. It might be beneficial if one could mentally partition the list so only the relevant subset was active, so that vegetables would be activated in the produce section. In four experiments, we explored observers' abilities to partition memory searches. For example, if observers held 16 items in memory, but only eight of the items were relevant, would response times resemble a search through eight or 16 items? In Experiments 1a and 1b, observers were not faster for the partition set; however, they suffered relatively small deficits when "lures" (items from the irrelevant subset) were presented, indicating that they were aware of the partition. In Experiment 2 the partitions were based on semantic distinctions, and again, observers were unable to restrict search to the relevant items. In Experiments 3a and 3b, observers attempted to remove items from the list one trial at a time but did not speed up over the course of a block, indicating that they also could not limit their memory searches. Finally, Experiments 4a, 4b, 4c, and 4d showed that observers were able to limit their memory searches when a subset was relevant for a run of trials. Overall, observers appear to be unable or unwilling to partition memory sets from trial to trial, yet they are capable of restricting search to a memory subset that remains relevant for several trials. This pattern is consistent with a cost to switching between currently relevant memory items.

One visual search, many memory searches: An eye-tracking investigation of hybrid search.

Suppose you go to the supermarket with a shopping list of 10 items held in memory. Your shopping expedition can be seen as a combination of visual search and memory search. This is known as "hybrid search." There is a growing interest in understanding how hybrid search tasks are accomplished. We used eye tracking to examine how manipulating the number of possible targets (the memory set size [MSS]) changes how observers (Os) search. We found that dwell time on each distractor increased with MSS, suggesting a memory search was being executed each time a new distractor was fixated. Meanwhile, although the rate of refixation increased with MSS, it was not nearly enough to suggest a strategy that involves repeatedly searching visual space for subgroups of the target set. These data provide a clear demonstration that hybrid search tasks are carried out via a "one visual search, many memory searches" heuristic in which Os examine items in the visual array once with a very low rate of refixations. For each item selected, Os activate a memory search that produces logarithmic response time increases with increased MSS. Furthermore, the percentage of distractors fixated was strongly modulated by the MSS: More items in the MSS led to a higher percentage of fixated distractors. Searching for more potential targets appears to significantly alter how Os approach the task, ultimately resulting in more eye movements and longer response times.

Clarifying the role of target similarity, task relevance and feature-based suppression during sustained inattentional blindness.

How is feature-based attention distributed when engaged in a challenging attentional task? Thanks to formative electrophysiological and psychophysical work, we know a great deal about the spatial distribution of attention, but much less is known about how feature-based attention is allocated. In a large-scale online study, we investigated the distribution of attention to color space using a sustained inattentional blindness task. In order to query what parts of color space were being attended or inhibited, we varied the color of an unexpected stimulus on the final trial. Noticing rates for this stimulus indicate that when engaged in a difficult task that involves tracking items of one color and ignoring items of two different colors, observers attend the target color and inhibit the to-be ignored colors. Further, similarity to the target drives detection such that colors more similar to the target are more likely to be detected. Finally, our data suggest that when possible, observers inhibit regions of color space rather than individuating specific colors and adjusting the level of inhibition for a particular color accordingly. Together, our data support the notion of feature-based suppression for task relevant (to-be ignored) information, but we found no evidence of an inhibitory surround based on target color similarity.

Memory search for the first target modulates the magnitude of the attentional blink.

The resolution of temporal attention is limited in a manner that makes it difficult to identify two targets in short succession. This limitation produces the phenomenon known as the attentional blink (AB), in which processing of a first target (T1) impairs identification of a second target (T2). In the AB literature, there is broad agreement that increasing the time it takes to process T1 leads to a larger AB. One might, therefore, predict that increasing the number of possible T1 identities, or target set, from 1 to 16 would lead to a larger AB. We were surprised to find that this manipulation of T1 difficulty had no influence on AB magnitude. In subsequent experiments, we found that AB magnitude interacts with T1 processing time only under certain circumstances. Specifically, when the T1 task was either well masked or had to be completed online, we found a reliable interaction between AB magnitude and the target set size. When neither of these conditions was fulfilled, there was no interaction between target set size and the AB. Previous research found that when the target set changes from trial to trial, trials with more possible targets elicited a larger AB. In the present study, the target set is held constant, reducing the demands on working memory. Nevertheless, AB magnitude still interacts with target set size, as long as the T1 task cannot be processed offline. Thus, the act of searching memory delays subsequent processing, even when the role of working memory has been minimized.

The effect of context on pointer allocation in visual working memory.

Visual working memory (VWM) can hold a limited amount of visual information and manipulate it. It encodes this information and forms representations of each one of the relevant objects. When an object changes, VWM can either update or reset its representation to account for this change. To access a specific representation VWM relies on a pointer system associating each representation with the corresponding object in the environment. While previous studies described these processes as reacting to a change in the object status, this study investigated the adaptability of the pointer system to the task context. We measured the contralateral delay activity (CDA; an electrophysiological marker of VWM) as a marker of updating and resetting. In two experiments we used a shape change detection task (similar to Balaban & Luria, 2017) and manipulated the proportion of the resetting and updating trials to create different task contexts. Experiment 1 indicated that VWM can adapt to a resetting mode in which it performs resetting in conditions that triggered updating in previous studies. However, Experiment 2 revealed that the pointer system cannot adapt to an updating mode and perform updating in conditions that trigger resetting. These results suggest that VWM can strategically perform resetting, but once a pointer is lost, it's impossible to update the representation and a resetting process is mandatory triggered regardless of the context.

Cue relevance drives early quitting in visual search.

Irrelevant salient distractors can trigger early quitting in visual search, causing observers to miss targets they might otherwise find. Here, we asked whether task-relevant salient cues can produce a similar early quitting effect on the subset of trials where those cues fail to highlight the target. We presented participants with a difficult visual search task and used two cueing conditions. In the high-predictive condition, a salient cue in the form of a red circle highlighted the target most of the time a target was present. In the low-predictive condition, the cue was far less accurate and did not reliably predict the target (i.e., the cue was often a false positive). These were contrasted against a control condition in which no cues were presented. In the high-predictive condition, we found clear evidence of early quitting on trials where the cue was a false positive, as evidenced by both increased miss errors and shorter response times on target absent trials. No such effects were observed with low-predictive cues. Together, these results suggest that salient cues which are false positives can trigger early quitting, though perhaps only when the cues have a high-predictive value. These results have implications for real-world searches, such as medical image screening, where salient cues (referred to as computer-aided detection or CAD) may be used to highlight potentially relevant areas of images but are sometimes inaccurate.

Neural measures reveal a fixed item limit in subitizing.

For centuries, it has been known that humans can rapidly and accurately enumerate small sets of items, a process referred to as subitizing. However, there is still active debate regarding the mechanisms that mediate this ability. For example, some have argued that subitizing reflects the operation of a fixed-capacity individuation mechanism that enables concurrent access to a small number of items. However, others have argued that subitizing reflects the operation of a continuous numerical estimation mechanism whose precision varies with numerosity in a manner consistent with Weber's law. Critically, quantitative models based on either of these predictions can provide a reasonable description of subitizing performance, making it difficult to discriminate between these alternatives solely on the basis of subjects' behavioral performance. Here, we attempted to discriminate between fixed-capacity and continuous estimation models of subitizing using neural measures. In two experiments, we recorded EEGs while subjects performed a demanding subitizing task and examined set-size-dependent changes in a neurophysiological marker of visual selection (the N2pc event-related potential component) evoked by an array of to-be-enumerated items. In both experiments, N2pc amplitudes increased monotonically within the subitizing range before reaching an asymptotic limit at approximately three items. Moreover, inter-participant differences in the location of this asymptote were strongly predictive of behavioral estimates of subitizing span derived from a fixed-capacity model. Thus, neural activity linked with subitizing ability shows evidence of an early and discrete limit in the number of items that can be concurrently apprehended, supporting a fixed-capacity model of this process.

The invisible gorilla strikes again: sustained inattentional blindness in expert observers.

Researchers have shown that people often miss the occurrence of an unexpected yet salient event if they are engaged in a different task, a phenomenon known as inattentional blindness. However, demonstrations of inattentional blindness have typically involved naive observers engaged in an unfamiliar task. What about expert searchers who have spent years honing their ability to detect small abnormalities in specific types of images? We asked 24 radiologists to perform a familiar lung-nodule detection task. A gorilla, 48 times the size of the average nodule, was inserted in the last case that was presented. Eighty-three percent of the radiologists did not see the gorilla. Eye tracking revealed that the majority of those who missed the gorilla looked directly at its location. Thus, even expert searchers, operating in their domain of expertise, are vulnerable to inattentional blindness.

Informatics in radiology: what can you see in a single glance and how might this guide visual search in medical images?

Diagnostic accuracy for radiologists is above that expected by chance when they are exposed to a chest radiograph for only one-fifth of a second, a period too brief for more than a single voluntary eye movement. How do radiologists glean information from a first glance at an image? It is thought that this expert impression of the gestalt of an image is related to the everyday, immediate visual understanding of the gist of a scene. Several high-speed mechanisms guide our search of complex images. Guidance by basic features (such as color) requires no learning, whereas guidance by complex scene properties is learned. It is probable that both hardwired guidance by basic features and learned guidance by scene structure become part of radiologists' expertise. Search in scenes may be best explained by a two-pathway model: Object recognition is performed via a selective pathway in which candidate targets must be individually selected for recognition. A second, nonselective pathway extracts information from global or statistical information without selecting specific objects. An appreciation of the role of nonselective processing may be particularly useful for understanding what separates novice from expert radiologists and could help establish new methods of physician training based on medical image perception.

Scanners and drillers: characterizing expert visual search through volumetric images.

Modern imaging methods like computed tomography (CT) generate 3-D volumes of image data. How do radiologists search through such images? Are certain strategies more efficient? Although there is a large literature devoted to understanding search in 2-D, relatively little is known about search in volumetric space. In recent years, with the ever-increasing popularity of volumetric medical imaging, this question has taken on increased importance as we try to understand, and ultimately reduce, errors in diagnostic radiology. In the current study, we asked 24 radiologists to search chest CTs for lung nodules that could indicate lung cancer. To search, radiologists scrolled up and down through a "stack" of 2-D chest CT "slices." At each moment, we tracked eye movements in the 2-D image plane and coregistered eye position with the current slice. We used these data to create a 3-D representation of the eye movements through the image volume. Radiologists tended to follow one of two dominant search strategies: "drilling" and "scanning." Drillers restrict eye movements to a small region of the lung while quickly scrolling through depth. Scanners move more slowly through depth and search an entire level of the lung before moving on to the next level in depth. Driller performance was superior to the scanners on a variety of metrics, including lung nodule detection rate, percentage of the lung covered, and the percentage of search errors where a nodule was never fixated.

Incidental recognition reveals attentional tradeoffs shaped by categorical similarity.

Search efficiency suffers when observers look for multiple targets or a single imprecisely defined target. These conditions prevent a narrow target template, resulting in improved delayed distractor recognition. In our first experiment with hybrid visual and memory search, we investigated the interaction of target variety and target number on search efficiency. Results supported the hypothesis that numerous targets impair search efficiency much more when targets are unrelated. These efficiency impairments were linked to distractor processing, indicated by increased delayed recognition. A second experiment manipulated target-distractor similarity to determine whether prioritization of target-defining features is totally eliminated in search for eight unrelated targets. For related and unrelated targets alike, recognition declined for distractors bearing less resemblance to targets and more to each other. This suggests templates for unrelated targets successfully prioritize relevant features at some stage of attention. Avoidance of random distractors was stronger when targets were related, at the price of slower, more error-prone identification of within-category distractors. Within-category processing difficulty for related targets likely stems from categorical interference as previously demonstrated in recognition memory. Thus, target variety versus homogeneity afforded different advantages and limitations depending on target number, target-distractor, and distractor-distractor resemblance. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Visual expertise is more than meets the eye: an examination of holistic visual processing in radiologists and architects.

Purpose: One of the dominant behavioral markers of visual-expert search strategy, holistic visual processing (HVP), suggests that experts process information from a larger region of space in conjunction with a more focused gaze pattern to improve search speed and accuracy. To date, extant literature suggests that visual search expertise is domain specific, including HVP and its associated behaviors. Approach: The current study is the first to use eye tracking to directly measure the HVP strategies of two expert groups, radiologists and architects, in comparison to one another and a novice control. Results: In doing so, we replicated and extended this prior research: visual expertise is domain specific. However, our eye-tracking data indicate that contrary to this prior work, HVP strategies are transferable across domains. Yet, despite the transfer of HVP strategies, there is neither reduced search time nor greater accuracy in visual search outside of an expert's domain. Conclusions: Therefore, our data suggest that HVP behaviors are a particular form of visual search mechanism that, outside of an expert's native search-ecology, are not necessarily conducive to more general visual search success. It is in addition to explicit knowledge of an expert's domain, how to search, and where to search, that HVP strategies are their most effective for visual search success.

Pathologist pupil dilation reflects experience level and difficulty in diagnosing medical images.

Purpose: Digital whole slide imaging allows pathologists to view slides on a computer screen instead of under a microscope. Digital viewing allows for real-time monitoring of pathologists' search behavior and neurophysiological responses during the diagnostic process. One particular neurophysiological measure, pupil diameter, could provide a basis for evaluating clinical competence during training or developing tools that support the diagnostic process. Prior research shows that pupil diameter is sensitive to cognitive load and arousal, and it switches between exploration and exploitation of a visual image. Different categories of lesions in pathology pose different levels of challenge, as indicated by diagnostic disagreement among pathologists. If pupil diameter is sensitive to the perceived difficulty in diagnosing biopsies, eye-tracking could potentially be used to identify biopsies that may benefit from a second opinion. Approach: We measured case onset baseline-corrected (phasic) and uncorrected (tonic) pupil diameter in 90 pathologists who each viewed and diagnosed 14 digital breast biopsy cases that cover the diagnostic spectrum from benign to invasive breast cancer. Pupil data were extracted from the beginning of viewing and interpreting of each individual case. After removing 122 trials ( < 10 % ) with poor eye-tracking quality, 1138 trials remained. We used multiple linear regression with robust standard error estimates to account for dependent observations within pathologists. Results: We found a positive association between the magnitude of phasic dilation and subject-centered difficulty ratings and between the magnitude of tonic dilation and untransformed difficulty ratings. When controlling for case diagnostic category, only the tonic-difficulty relationship persisted. Conclusions: Results suggest that tonic pupil dilation may indicate overall arousal differences between pathologists as they interpret biopsy cases and could signal a need for additional training, experience, or automated decision aids. Phasic dilation is sensitive to characteristics of biopsies that tend to elicit higher difficulty ratings and could indicate a need for a second opinion.

Are Pathologists Self-Aware of Their Diagnostic Accuracy? Metacognition and the Diagnostic Process in Pathology.

BACKGROUND: Metacognition is a cognitive process that involves self-awareness of thinking, understanding, and performance. This study assesses pathologists' metacognition by examining the association between their diagnostic accuracy and self-reported confidence levels while interpreting skin and breast biopsies. DESIGN: We studied 187 pathologists from the Melanoma Pathology Study (M-Path) and 115 pathologists from the Breast Pathology Study (B-Path). We measured pathologists' metacognitive ability by examining the area under the curve (AUC), the area under each pathologist's receiver operating characteristic (ROC) curve summarizing the association between confidence and diagnostic accuracy. We investigated possible relationships between this AUC measure, referred to as metacognitive sensitivity, and pathologist attributes. We also assessed whether higher metacognitive sensitivity affected the association between diagnostic accuracy and a secondary diagnostic action such as requesting a second opinion. RESULTS: We found no significant associations between pathologist clinical attributes and metacognitive AUC. However, we found that pathologists with higher AUC showed a stronger trend to request secondary diagnostic action for inaccurate diagnoses and not for accurate diagnoses compared with pathologists with lower AUC. LIMITATIONS: Pathologists reported confidence in specific diagnostic terms, rather than the broader classes into which the diagnostic terms were later grouped to determine accuracy. In addition, while there is no gold standard for the correct diagnosis to determine the accuracy of pathologists' interpretations, our studies achieved a high-quality reference diagnosis by using the consensus diagnosis of 3 experienced pathologists. CONCLUSIONS: Metacognition can affect clinical decisions. If pathologists have self-awareness that their diagnosis may be inaccurate, they can request additional tests or second opinions, providing the opportunity to correct inaccurate diagnoses. HIGHLIGHTS: Metacognitive sensitivity varied across pathologists, with most showing higher sensitivity than expected by chance.None of the demographic or clinical characteristics we examined was significantly associated with metacognitive sensitivity.Pathologists with higher metacognitive sensitivity were more likely to request additional tests or second opinions for their inaccurate diagnoses.

Zoom behavior during visual search modulates pupil diameter and reflects adaptive control states.

Adaptive gain theory proposes that the dynamic shifts between exploration and exploitation control states are modulated by the locus coeruleus-norepinephrine system and reflected in tonic and phasic pupil diameter. This study tested predictions of this theory in the context of a societally important visual search task: the review and interpretation of digital whole slide images of breast biopsies by physicians (pathologists). As these medical images are searched, pathologists encounter difficult visual features and intermittently zoom in to examine features of interest. We propose that tonic and phasic pupil diameter changes during image review may correspond to perceived difficulty and dynamic shifts between exploration and exploitation control states. To examine this possibility, we monitored visual search behavior and tonic and phasic pupil diameter while pathologists (N = 89) interpreted 14 digital images of breast biopsy tissue (1,246 total images reviewed). After viewing the images, pathologists provided a diagnosis and rated the level of difficulty of the image. Analyses of tonic pupil diameter examined whether pupil dilation was associated with pathologists' difficulty ratings, diagnostic accuracy, and experience level. To examine phasic pupil diameter, we parsed continuous visual search data into discrete zoom-in and zoom-out events, including shifts from low to high magnification (e.g., 1× to 10×) and the reverse. Analyses examined whether zoom-in and zoom-out events were associated with phasic pupil diameter change. Results demonstrated that tonic pupil diameter was associated with image difficulty ratings and zoom level, and phasic pupil diameter showed constriction upon zoom-in events, and dilation immediately preceding a zoom-out event. Results are interpreted in the context of adaptive gain theory, information gain theory, and the monitoring and assessment of physicians' diagnostic interpretive processes.

Delineating the neural signatures of tracking spatial position and working memory during attentive tracking.

In the attentive tracking task, observers track multiple objects as they move independently and unpredictably among visually identical distractors. Although a number of models of attentive tracking implicate visual working memory as the mechanism responsible for representing target locations, no study has ever directly compared the neural mechanisms of the two tasks. In the current set of experiments, we used electrophysiological recordings to delineate similarities and differences between the neural processing involved in working memory and attentive tracking. We found that the contralateral electrophysiological response to the two tasks was similarly sensitive to the number of items attended in both tasks but that there was also a unique contralateral negativity related to the process of monitoring target position during tracking. This signal was absent for periods of time during tracking tasks when objects briefly stopped moving. These results provide evidence that, during attentive tracking, the process of tracking target locations elicits an electrophysiological response that is distinct and dissociable from neural measures of the number of items being attended.