Identifying inefficient strategies in automation-aided signal detection.

Automated diagnostic aids can assist human operators in signal detection tasks, providing alarms, warnings, or diagnoses. Operators often use decision aids poorly, though, falling short of best possible performance levels. Previous research has suggested that operators interact with binary signal detection aids using a sluggish contingent cutoff (CC) strategy (Robinson & Sorkin, 1985), shifting their response criterion in the direction stipulated by the aid's diagnosis each trial but making adjustments that are smaller than optimal. The present study tested this model by examining the efficiency of automation-aided signal detection under different levels of task difficulty. In a pair of experiments, participants performed a numeric decision-making task requiring them to make signal or noise judgments on the basis of probabilistic readings. The mean reading values of signal and noise states differed between groups of participants, producing two levels of task difficulty. Data were fit with the CC model and two alternative accounts of automation-aided strategy: a discrete deference (DD) model, which assumed participants defer to the aid on a subset of trials and a mixture model, which assumed that participants choose randomly between the CC and DD strategies every trial. Model fits favored the mixture model. The results indicate multiple forms of inefficiency in operators' strategies for using signal detection aids. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Simple manipulations of anthropomorphism fail to induce perceptions of humanness or improve trust in an automated agent.

Although automation is employed as an aid to human performance, operators often interact with automated decision aids inefficiently. The current study investigated whether anthropomorphic automation would engender higher trust and use, subsequently improving human-automation team performance. Participants performed a multi-element probabilistic signal detection task in which they diagnosed a hypothetical nuclear reactor as in a state of safety or danger. The task was completed unassisted and assisted by a 93%-reliable agent varying in anthropomorphism. Results gave no evidence that participants' perceptions of anthropomorphism differed between conditions. Further, anthropomorphic automation failed to bolster trust and automation-aided performance. Findings suggest that the benefits of anthropomorphism may be limited in some contexts.

Measuring the Efficiency of Automation-Aided Performance in a Simulated Baggage Screening Task.

OBJECTIVE: The present study replicated and extended prior findings of suboptimal automation use in a signal detection task, benchmarking automation-aided performance to the predictions of several statistical models of collaborative decision making. BACKGROUND: Though automated decision aids can assist human operators to perform complex tasks, operators often use the aids suboptimally, achieving performance lower than statistically ideal. METHOD: Participants performed a simulated security screening task requiring them to judge whether a target (a knife) was present or absent in a series of colored X-ray images of passenger baggage. They completed the task both with and without assistance from a 93%-reliable automated decision aid that provided a binary text diagnosis. A series of three experiments varied task characteristics including the timing of the aid's judgment relative to the raw stimuli, target certainty, and target prevalence. RESULTS AND CONCLUSION: Automation-aided performance fell closest to the predictions of the most suboptimal model under consideration, one which assumes the participant defers to the aid's diagnosis with a probability of 50%. Performance was similar across experiments. APPLICATION: Results suggest that human operators' performance when undertaking a naturalistic search task falls far short of optimal and far lower than prior findings using an abstract signal detection task.

The effect of cognitive load on horizontal and vertical spatial asymmetries.

Healthy individuals typically show a leftward attentional bias in the allocation of spatial attention along the horizontal plane, a phenomenon known as pseudoneglect, which relies on a right hemispheric dominance for visuospatial processing. Also, healthy individuals tend to overestimate the upper hemispace when orienting attention along the vertical plane, a phenomenon that may depend on asymmetric ventral and dorsal visual streams activation. Previous research has demonstrated that when attentional resources are reduced due to increased cognitive load, pseudoneglect is attenuated (or even reversed), due to decreased right-hemispheric activations. Critically, whether and how the reduction of attentional resources under load modulates vertical spatial asymmetries has not been addressed before. We asked participants to perform a line bisection task both with and without the addition of a concurrent auditory working memory task with lines oriented either horizontally or vertically. Results showed that increasing cognitive load reduced the typical leftward/upward bias with no difference between orientations. Our data suggest that the degree of cognitive load affects spatial attention not only in the horizontal but also in the vertical plane. Lastly, the similar effect of load on horizontal and vertical judgements suggests these biases may be related to only partially independent mechanisms.

Ironic efficiency in automation-aided signal detection.

Decision makers often make poor use of the information provided by an automated signal detection aid; recent studies have found that participants assisted by an automated aid fell well short of best-possible sensitivity levels. The present study tested the generalisability of this finding over varying levels of aid reliability. Participants performed a binary signal detection task either unaided or with assistance from a decision aid that was 60%, 85%, or 96%-reliable. Assistance from a highly reliable aid (85% or 96%) improved discrimination performance, while assistance from a low-reliability aid (60%) did not. Because their ideal strategy is to place less weight on less reliable cues, however, the decision makers' tendency to disuse the aid became more appropriate as the aid's reliability declined. Automation-aided efficiency was thus near to optimal when the aid was close to chance but became highly inefficient, ironically, as the aid's reliability increased. Practitioner Summary: Investigating operators' automation-aided information integration strategies allows human factors practitioners to predict the level of performance the operator will attain. Ironically, in an aided signal detection task, performance when assisted by a highly reliable aid is far less efficient than that obtained when assisted by a far less reliable aid. Abbreviations: OW: optimal weighting; UW: uniform weighting; CC: contingent criterion; BD: best decides; CF: coin flip; PM: probability matching; HDI: highest density interval; MCMC: markov chain monte carlo; HR: hit rate; FAR: false alarm rate.

Through Doorways and Down Corridors: Investigating Asymmetries During Computer Maze Navigation.

Pseudoneglect causes neurologically intact individuals to bias their attention to the left in near space, and to the right in far space. These attentional asymmetries impact both ambulatory and non-ambulatory activities, causing individuals to deviate rightward. While most studies investigating real-world navigation have found a rightward deviation when passing through a door, some have found the opposite pattern for corridors. To explore this dissociation, the current experiment explicitly compared navigation through doorways and corridors. To allow for a direct comparison between these two environments, the navigation task was undertaken in a simulated environment. Dextral participants (n = 98) completed several trials in either the doorway or corridor condition and their mean lateral position and variance was analysed. A rightward deviation was observed for doorways, consistent with previous research. Rightward biases were also observed for corridors, irrespective of the position within the corridor. The results argue against an explanation based on near/far space for the leftward bias in corridors. An explanation based on elevation of view is proposed as an alternative. The study also demonstrates that simulated environments provide an efficient means of investigating asymmetries in navigation.

Cognitive load exacerbates rightward biases during computer maze navigation.

Neurologically healthy individuals exhibit subtle attentional asymmetries, such that attention is preferentially directed leftwards for objects in near space and rightwards for objects in far space. These attentional biases also affect navigation and cause people to deviate to the right when passing through an aperture. The current study examined whether the rightward deviations observed in real-world environments translate to simulated environments. As proof of concept and to determine whether rightward biases could be further exacerbated, the degree of cognitive load imposed on participants was manipulated. Experiment 1 asked participants to navigate through the centre of a computer-based doorway. In one block of trials, participants completed the task by itself (baseline condition), while in another block of trials they also completed a simple auditory discrimination task (load condition). While analyses revealed rightward biases for both conditions, the difference between conditions was not significant. Experiment 2 therefore increased the difficulty of the auditory task. Analyses revealed a significant difference between conditions, suggesting that the degree of cognitive load further exacerbates rightward biases, demonstrating that the rightward asymmetries in navigation observed in the real world generalises to a simulated environment and that this phenomenon behaves in a way that is consistent with pseudoneglect.

No Effect of Cue Format on Automation Dependence in an Aided Signal Detection Task.

OBJECTIVE: To investigate whether manipulating the format of an automated decision aid's cues can improve participants' information integration strategies in a signal detection task. BACKGROUND: Automation-aided decision making is often suboptimal, falling well short of statistically ideal levels. The choice of format in which the cues from the aid are displayed may help users to better understand and integrate the aid's judgments with their own. METHOD: Participants performed a signal detection task that asked them to classify random dot images as either blue or orange dominant. They made their judgments either unaided or with assistance from a 93% reliable automated decision aid. The aid provided a binary judgment, along with an estimate of signal strength in the form of either a raw value, a likelihood ratio, or a confidence rating (Experiments 1 and 2) or a binary judgment along with either a verbal or verbal-visuospatial expression of confidence (Experiment 3). Aided sensitivity was benchmarked to the predictions of various statistical models of collaborative decision making. RESULTS AND CONCLUSION: Aided performance was suboptimal, matching the predictions of some of the least efficient models. Most importantly, performance was similar across cue formats. APPLICATION: Results indicate that changes to the format in which cues from a signal detection aid are rendered are unlikely to dramatically improve the efficiency of automation-aided decision making.

Benchmarking Aided Decision Making in a Signal Detection Task.

OBJECTIVE: A series of experiments examined human operators' strategies for interacting with highly (93%) reliable automated decision aids in a binary signal detection task. BACKGROUND: Operators often interact with automated decision aids in a suboptimal way, achieving performance levels lower than predicted by a statistically ideal model of information integration. To better understand operators' inefficient use of decision aids, we compared participants' automation-aided performance levels with the predictions of seven statistical models of collaborative decision making. METHOD: Participants performed a binary signal detection task that asked them to classify random dot images as either blue or orange dominant. They made their judgments either unaided or with assistance from a 93% reliable automated decision aid that provided either graded (Experiments 1 and 3) or binary (Experiment 2) cues. We compared automation-aided performance with the predictions of seven statistical models of collaborative decision making, including a statistically optimal model and Robinson and Sorkin's contingent criterion model. RESULTS AND CONCLUSION: Automation-aided sensitivity hewed closest to the predictions of the two least efficient collaborative models, well short of statistically ideal levels. Performance was similar whether the aid provided graded or binary judgments. Model comparisons identified potential strategies by which participants integrated their judgments with the aid's. APPLICATION: Results lend insight into participants' automation-aided decision strategies and provide benchmarks for predicting automation-aided performance levels.