Advancing Research on Medical Image Perception by Strengthening Multidisciplinary Collaboration.

Medical image interpretation is central to detecting, diagnosing, and staging cancer and many other disorders. At a time when medical imaging is being transformed by digital technologies and artificial intelligence, understanding the basic perceptual and cognitive processes underlying medical image interpretation is vital for increasing diagnosticians' accuracy and performance, improving patient outcomes, and reducing diagnostician burnout. Medical image perception remains substantially understudied. In September 2019, the National Cancer Institute convened a multidisciplinary panel of radiologists and pathologists together with researchers working in medical image perception and adjacent fields of cognition and perception for the "Cognition and Medical Image Perception Think Tank." The Think Tank's key objectives were to identify critical unsolved problems related to visual perception in pathology and radiology from the perspective of diagnosticians, discuss how these clinically relevant questions could be addressed through cognitive and perception research, identify barriers and solutions for transdisciplinary collaborations, define ways to elevate the profile of cognition and perception research within the medical image community, determine the greatest needs to advance medical image perception, and outline future goals and strategies to evaluate progress. The Think Tank emphasized diagnosticians' perspectives as the crucial starting point for medical image perception research, with diagnosticians describing their interpretation process and identifying perceptual and cognitive problems that arise. This article reports the deliberations of the Think Tank participants to address these objectives and highlight opportunities to expand research on medical image perception.

How do we measure attention? Using factor analysis to establish construct validity of neuropsychological tests.

We investigated whether standardized neuropsychological tests and experimental cognitive paradigms measure the same cognitive faculties. Specifically, do neuropsychological tests commonly used to assess attention measure the same construct as attention paradigms used in cognitive psychology and neuroscience? We built on the "general attention factor", comprising several widely used experimental paradigms (Huang et al., 2012). Participants (n = 636) completed an on-line battery (TestMyBrain.org) of six experimental tests [Multiple Object Tracking, Flanker Interference, Visual Working Memory, Approximate Number Sense, Spatial Configuration Visual Search, and Gradual Onset Continuous Performance Task (Grad CPT)] and eight neuropsychological tests [Trail Making Test versions A & B (TMT-A, TMT-B), Digit Symbol Coding, Forward and Backward Digit Span, Letter Cancellation, Spatial Span, and Arithmetic]. Exploratory factor analysis in a subset of 357 participants identified a five-factor structure: (1) attentional capacity (Multiple Object Tracking, Visual Working Memory, Digit Symbol Coding, Spatial Span), (2) search (Visual Search, TMT-A, TMT-B, Letter Cancellation); (3) Digit Span; (4) Arithmetic; and (5) Sustained Attention (GradCPT). Confirmatory analysis in 279 held-out participants showed that this model fit better than competing models. A hierarchical model where a general cognitive factor was imposed above the five specific factors fit as well as the model without the general factor. We conclude that Digit Span and Arithmetic tests should not be classified as attention tests. Digit Symbol Coding and Spatial Span tap attentional capacity, while TMT-A, TMT-B, and Letter Cancellation tap search (or attention-shifting) ability. These five tests can be classified as attention tests.

Efficacy of intermittent exposure to bright light for treating maladaptation to night work on a counterclockwise shift work rotation.

OBJECTIVES: Rotating shift work is associated with adverse outcomes due to circadian misalignment, sleep curtailment, work-family conflicts, and other factors. We tested a bright light countermeasure to enhance circadian adaptation on a counterclockwise rotation schedule. METHODS: Twenty-nine adults (aged 20-40 years; 15 women) participated in a 4-week laboratory simulation with weekly counterclockwise transitions from day, to night, to evening, to day shifts. Each week consisted of five 8-hour workdays including psychomotor vigilance tests, two days off, designated 8-hour sleep episodes every day, and an assessment of circadian melatonin secretion. Participants were randomized to a treatment group (N=14), receiving intermittent bright light during work designed to facilitate circadian adaptation, or a control group (N=15) working in indoor light. Adaptation was measured by how much of the melatonin secretion episode overlapped with scheduled sleep timing. RESULTS: On the last night shift, there was a greater overlap between melatonin secretion and scheduled sleep time in the treatment group [mean 4.90, standard deviation (SD) 2.8 hours] compared to the control group (2.62, SD 2.8 hours; P=0.002), with night shift adaptation strongly influenced by baseline melatonin timing (r2=-0.71, P=0.01). While the control group exhibited cognitive deficits on the last night shift, the treatment group's cognitive deficits on the last night and evening shifts were minimized. CONCLUSIONS: In this laboratory setting, intermittent bright light during work hours enhanced adaptation to night work and subsequent readaptation to evening and day work. Light regimens scheduled to shift circadian timing should be tested in actual shift workers on counterclockwise schedules as a workplace intervention.

The Emerging Science of Interoception: Sensing, Integrating, Interpreting, and Regulating Signals within the Self.

Interoception refers to the representation of the internal states of an organism, and includes the processes by which it senses, interprets, integrates, and regulates signals from within itself. This review presents a unified research framework and attempts to offer definitions for key terms to describe the processes involved in interoception. We elaborate on these definitions through illustrative research findings, and provide brief overviews of central aspects of interoception, including the anatomy and function of neural and non-neural pathways, diseases and disorders, manipulations and interventions, and predictive modeling. We conclude with discussions about major research gaps and challenges.

Searching Through Alternating Sequences: Working Memory and Inhibitory Tagging Mechanisms Revealed Using the MILO Task.

We used the Multi-Item Localisation (MILO) task to examine search through two sequences. In Sequential blocks of trials, six letters and six digits were touched in order. In Mixed blocks, participants alternated between letters and digits. These conditions mimic the A and B variants of the Trail Making Test (TMT). In both block types, targets either vanished or remained visible after being touched. There were two key findings. First, in Mixed blocks, reaction times exhibited a saw-tooth pattern, suggesting search for successive pairs of targets. Second, reaction time patterns for vanish and remain conditions were identical in Sequential blocks-indicating that participants could ignore past targets-but diverged in Mixed blocks. This suggests a breakdown of inhibitory tagging. These findings may help explain the elevated completion times observed in TMT-B, relative to TMT-A.

MILO Mobile: An iPad App to Measure Search Performance in Multi-Target Sequences.

This article introduces a mobile app version of the Multi-Item Localization (MILO) task. The MILO task was designed to explore the temporal context of search through a sequence and has proven useful in both basic and applied research settings. Here, we describe the basic features of the app and how it can be obtained, installed, and modified. We also provide example data files and present two new sets of empirical data to verify that previous findings concerning prospective planning and retrospective memory (i.e., inhibitory tagging) are reproducible with the app. We conclude by discussing ongoing studies and future modifications that illustrate the flexibility and potential of the MILO Mobile app.

Rapid perceptual processing in two- and three-dimensional prostate images.

Radiologists can identify whether a radiograph is abnormal or normal at above chance levels in breast and lung images presented for half a second or less. This early perceptual processing has only been demonstrated in static two-dimensional images (e.g., mammograms). Can radiologists rapidly extract the "gestalt" from more complex imaging modalities? For example, prostate multiparametric magnetic resonance imaging (mpMRI) displays a series of images as a virtual stack and comprises multiple imaging sequences: anatomical information from the T2-weighted (T2W) sequence, functional information from diffusion-weighted imaging, and apparent diffusion coefficient sequences. We first tested rapid perceptual processing in static T2W images then among the two functional sequences. Finally, we examined whether this rapid radiological perception could be observed using T2W multislice imaging. Readers with experience in prostate mpMRI could detect and localize lesions in all sequences after viewing a 500-ms static image. Experienced prostate readers could also detect and localize lesions when viewing multislice image stacks presented as brief movies, with image slices presented at either 48, 96, or 144 ms. The ability to quickly extract the perceptual gestalt may be a general property of expert perception, even in complex imaging modalities.

Understanding the Profile of Cancer-Related Cognitive Impairments: A Critique of Meta-Analyses.

A large body of evidence indicates that cancer survivors who have undergone chemotherapy have cognitive impairments. Substantial disagreement exists regarding which cognitive domains are impaired in this population. We suggest that is in part due to inconsistency in how neuropsychological tests are assigned to cognitive domains. The purpose of this paper is to critically analyze the meta-analytic literature on cancer-related cognitive impairments (CRCI) to quantify this inconsistency. We identified all neuropsychological tests reported in seven meta-analyses of the CRCI literature. Although effect sizes were generally negative (indicating impairment), every domain was declared to be impaired in at least one meta-analysis and unimpaired in at least one other meta-analysis. We plotted summary effect sizes from all the meta-analyses and quantified disagreement by computing the observed and ideal distributions of the one-way χ2 statistic. The actual χ2 distributions were noticeably more peaked and shifted to the left than the ideal distributions, indicating substantial disagreement among the meta-analyses in how neuropsychological tests were categorized to domains. A better understanding of the profile of impairments in CRCI is essential for developing effective remediation methods. To accomplish this goal, the research field needs to promote better agreement on how to measure specific cognitive functions.

Measuring the time course of selection during visual search.

In visual search tasks, observers can guide their attention towards items in the visual field that share features with the target item. In this series of studies, we examined the time course of guidance toward a subset of items that have the same color as the target item. Landolt Cs were placed on 16 colored disks. Fifteen distractor Cs had gaps facing up or down while one target C had a gap facing left or right. Observers searched for the target C and reported which side contained the gap as quickly as possible. In the absence of other information, observers must search at random through the Cs. However, during the trial, the disks changed colors. Twelve disks were now of one color and four disks were of another color. Observers knew that the target C would always be in the smaller color set. The experimental question was how quickly observers could guide their attention to the smaller color set. Results indicate that observers could not make instantaneous use of color information to guide the search, even when they knew which two colors would be appearing on every trial. In each study, it took participants 200-300 ms to fully utilize the color information once presented. Control studies replicated the finding with more saturated colors and with colored C stimuli (rather than Cs on colored disks). We conclude that segregation of a display by color for the purposes of guidance takes 200-300 ms to fully develop.

A Call for a Neuroscience Approach to Cancer-Related Cognitive Impairment.

Cancer-related cognitive impairment (CRCI) is a widespread problem for the increasing population of cancer survivors. Our understanding of the nature, causes, and prevalence of CRCI is hampered by a reliance on clinical neuropsychological methods originally designed to detect focal lesions. Future progress will require collaboration between neuroscience and clinical neuropsychology.

Unique Views on Obesity-Related Behaviors and Environments: Research Using Still and Video Images.

OBJECTIVES: To document challenges to and benefits from research involving the use of images by capturing examples of such research to assess physical activity- or nutrition-related behaviors and/or environments. METHODS: Researchers (i.e., key informants) using image capture in their research were identified through knowledge and networks of the authors of this paper and through literature search. Twenty-nine key informants completed a survey covering the type of research, source of images, and challenges and benefits experienced, developed specifically for this study. RESULTS: Most respondents used still images in their research, with only 26.7% using video. Image sources were categorized as participant generated (n = 13; e.g., participants using smartphones for dietary assessment), researcher generated (n = 10; e.g., wearable cameras with automatic image capture), or curated from third parties (n = 7; e.g., Google Street View). Two of the major challenges that emerged included the need for automated processing of large datasets (58.8%) and participant recruitment/compliance (41.2%). Benefit-related themes included greater perspectives on obesity with increased data coverage (34.6%) and improved accuracy of behavior and environment assessment (34.6%). CONCLUSIONS: Technological advances will support the increased use of images in the assessment of physical activity, nutrition behaviors, and environments. To advance this area of research, more effective collaborations are needed between health and computer scientists. In particular development of automated data extraction methods for diverse aspects of behavior, environment, and food characteristics are needed. Additionally, progress in standards for addressing ethical issues related to image capture for research purposes is critical.

Five Factors that Guide Attention in Visual Search.

How do we find what we are looking for? Fundamental limits on visual processing mean that even when the desired target is in our field of view, we often need to search, because it is impossible to recognize everything at once. Searching involves directing attention to objects that might be the target. This deployment of attention is not random. It is guided to the most promising items and locations by five factors discussed here: Bottom-up salience, top-down feature guidance, scene structure and meaning, the previous history of search over time scales from msec to years, and the relative value of the targets and distractors. Modern theories of search need to specify how all five factors combine to shape search behavior. An understanding of the rules of guidance can be used to improve the accuracy and efficiency of socially-important search tasks, from security screening to medical image perception.

A soft handoff of attention between cerebral hemispheres.

Each cerebral hemisphere initially processes one half of the visual world. How are moving objects seamlessly tracked when they traverse visual hemifields? Covert tracking of lateralized objects evokes a difference between slow-wave electrophysiological activity observed from contralateral and ipsilateral electrodes in occipitoparietal regions. This event-related potentials (ERP) waveform, known as contralateral delay activity (CDA) [1, 2], is sensitive to the number of objects tracked [1, 2] and responds dynamically to changes in this quantity [3]. When a tracked object crosses the midline, an inversion in CDA polarity revealed the dropping of the object's representation by one hemisphere and its acquisition by the other. Importantly, our data suggest that the initially tracking hemisphere continues to represent the object for a period after that object crosses the midline. Meanwhile, the receiving hemisphere begins to represent the object before the object crosses the midline, leading to a period in which the object is represented by both hemispheres. Further, this overlap in representation is reduced if the midline crossing is unpredictable. Thus, this process is sensitive to observer expectations and does not simply reflect overlapping receptive fields near the midline.

Interactive multiple object tracking (iMOT).

We introduce a new task for exploring the relationship between action and attention. In this interactive multiple object tracking (iMOT) task, implemented as an iPad app, participants were presented with a display of multiple, visually identical disks which moved independently. The task was to prevent any collisions during a fixed duration. Participants could perturb object trajectories via the touchscreen. In Experiment 1, we used a staircase procedure to measure the ability to control moving objects. Object speed was set to 1°/s. On average participants could control 8.4 items without collision. Individual control strategies were quite variable, but did not predict overall performance. In Experiment 2, we compared iMOT with standard MOT performance using identical displays. Object speed was set to 2°/s. Participants could reliably control more objects (M = 6.6) than they could track (M = 4.0), but performance in the two tasks was positively correlated. In Experiment 3, we used a dual-task design. Compared to single-task baseline, iMOT performance decreased and MOT performance increased when the two tasks had to be completed together. Overall, these findings suggest: 1) There is a clear limit to the number of items that can be simultaneously controlled, for a given speed and display density; 2) participants can control more items than they can track; 3) task-relevant action appears not to disrupt MOT performance in the current experimental context.

Prevalence effects in newly trained airport checkpoint screeners: trained observers miss rare targets, too.

Many socially important search tasks are characterized by low target prevalence, meaning that targets are rarely encountered. For example, transportation security officers (TSOs) at airport checkpoints encounter very few actual threats in carry-on bags. In laboratory-based visual search experiments, low prevalence reduces the probability of detecting targets (Wolfe, Horowitz, & Kenner, 2005). In the lab, this "prevalence effect" is caused by changes in decision and response criteria (Wolfe & Van Wert, 2010) and can be mitigated by presenting a burst of high-prevalence search with feedback (Wolfe et al., 2007). The goal of this study was to see if these effects could be replicated in the field with TSOs. A total of 125 newly trained TSOs participated in one of two experiments as part of their final evaluation following training. They searched for threats in simulated bags across five blocks. The first three blocks were low prevalence (target prevalence ≤ .05) with no feedback; the fourth block was high prevalence (.50) with full feedback; and the final block was, again, low prevalence. We found that newly trained TSOs were better at detecting targets at high compared to low prevalence, replicating the prevalence effect. Furthermore, performance was better (and response criterion was more "liberal") in the low-prevalence block that took place after the high-prevalence block than in the initial three low-prevalence blocks, suggesting that a burst of high-prevalence trials may help alleviate the prevalence effect in the field.

Can we improve clinical prediction of at-risk older drivers?

OBJECTIVES: To conduct a pilot study to evaluate the predictive value of the Montreal Cognitive Assessment test (MoCA) and a brief test of multiple object tracking (MOT) relative to other tests of cognition and attention in identifying at-risk older drivers, and to determine which combination of tests provided the best overall prediction. METHODS: Forty-seven currently licensed drivers (58-95 years), primarily from a clinical driving evaluation program, participated. Their performance was measured on: (1) a screening test battery, comprising MoCA, MOT, Mini-Mental State Examination (MMSE), Trail-Making Test, visual acuity, contrast sensitivity, and Useful Field of View (UFOV) and (2) a standardized road test. RESULTS: Eighteen participants were rated at-risk on the road test. UFOV subtest 2 was the best single predictor with an area under the curve (AUC) of .84. Neither MoCA nor MOT was a better predictor of the at-risk outcome than either MMSE or UFOV, respectively. The best four-test combination (MMSE, UFOV subtest 2, visual acuity and contrast sensitivity) was able to identify at-risk drivers with 95% specificity and 80% sensitivity (.91 AUC). CONCLUSIONS: Although the best four-test combination was much better than a single test in identifying at-risk drivers, there is still much work to do in this field to establish test batteries that have both high sensitivity and specificity.

Apparent color-orientation bindings in the periphery can be influenced by feature binding in central vision.

A previous study reported the misbinding illusion in which visual features belonging to overlapping sets of items were erroneously integrated (Wu, Kanai, & Shimojo, 2004, Nature, 429, 262). In this illusion, central and peripheral portions of a transparent motion field combined color and motion in opposite fashions. When observers saw such stimuli, their perceptual color-motion bindings in the periphery were re-arranged in such a way as to accord with the bindings in the central region, resulting in erroneous color-motion pairings (misbinding) in peripheral vision. Here we show that this misbinding illusion is also seen in the binding of color and orientation. When the central field of a stimulus array was composed of objects that had coherent (regular) color-orientation pairings, subjective color-orientation bindings in the peripheral stimuli were automatically altered to match the coherent pairings of the central stimuli. Interestingly, the illusion was induced only when all items in the central field combined color and orientation in an orthogonal fashion (e.g. all red bars were horizontal and all green bars were vertical). If this orthogonality was disrupted (e.g. all red and green bars were horizontal), the central field lost its power to induce the misbinding illusion in the peripheral stimuli. The original misbinding illusion study proposed that the illusion stemmed from a perceptual extrapolation that resolved peripheral ambiguity with clear central vision. However, our present results indicate that visual analyses of the correlational structure between two features (color and orientation) are critical for the illusion to occur, suggesting a rapid integration of multiple featural cues in the human visual system.

Automatic feature-based grouping during multiple object tracking.

Contour interpolation automatically binds targets with distractors to impair multiple object tracking (Keane, Mettler, Tsoi, & Kellman, 2011). Is interpolation special in this regard or can other features produce the same effect? To address this question, we examined the influence of eight features on tracking: color, contrast polarity, orientation, size, shape, depth, interpolation, and a combination (shape, color, size). In each case, subjects tracked 4 of 8 objects that began as undifferentiated shapes, changed features as motion began (to enable grouping), and returned to their undifferentiated states before halting. We found that intertarget grouping improved performance for all feature types except orientation and interpolation (Experiment 1 and Experiment 2). Most importantly, target-distractor grouping impaired performance for color, size, shape, combination, and interpolation. The impairments were, at times, large (>15% decrement in accuracy) and occurred relative to a homogeneous condition in which all objects had the same features at each moment of a trial (Experiment 2), and relative to a "diversity" condition in which targets and distractors had different features at each moment (Experiment 3). We conclude that feature-based grouping occurs for a variety of features besides interpolation, even when irrelevant to task instructions and contrary to the task demands, suggesting that interpolation is not unique in promoting automatic grouping in tracking tasks. Our results also imply that various kinds of features are encoded automatically and in parallel during tracking.

Swapping or dropping? Electrophysiological measures of difficulty during multiple object tracking.

In the multiple object tracking task, participants are asked to keep targets separate from identical distractors as all items move randomly. It is well known that simple manipulations such as object speed and number of distractors dramatically alter the number of targets that are successfully tracked, but very little is known about what causes this variation in performance. One possibility is that participants tend to lose track of objects (dropping) more frequently under these conditions. Another is that the tendency to confuse a target with a distractor increases (swapping). These two mechanisms have very different implications for the attentional architecture underlying tracking. However, behavioral data alone cannot differentiate between these possibilities. In the current study, we used an electrophysiological marker of the number of items being actively tracked to assess which type of errors tended to occur during speed and distractor load manipulations. Our neural measures suggest that increased distractor load led to an increased likelihood of confusing targets with distractors while increased speed led to an increased chance of a target item being dropped. Behavioral experiments designed to test this novel prediction support this assertion.