The (Im)perfect Automation Schema: Who Is Trusted More, Automated or Human Decision Support?

OBJECTIVE: This study's purpose was to better understand the dynamics of trust attitude and behavior in human-agent interaction. BACKGROUND: Whereas past research provided evidence for a perfect automation schema, more recent research has provided contradictory evidence. METHOD: To disentangle these conflicting findings, we conducted an online experiment using a simulated medical X-ray task. We manipulated the framing of support agents (i.e., artificial intelligence (AI) versus expert versus novice) between-subjects and failure experience (i.e., perfect support, imperfect support, back-to-perfect support) within subjects. Trust attitude and behavior as well as perceived reliability served as dependent variables. RESULTS: Trust attitude and perceived reliability were higher for the human expert than for the AI than for the human novice. Moreover, the results showed the typical pattern of trust formation, dissolution, and restoration for trust attitude and behavior as well as perceived reliability. Forgiveness after failure experience did not differ between agents. CONCLUSION: The results strongly imply the existence of an imperfect automation schema. This illustrates the need to consider agent expertise for human-agent interaction. APPLICATION: When replacing human experts with AI as support agents, the challenge of lower trust attitude towards the novel agent might arise.

Understanding the Impact of Time Pressure and Automation Support in a Visual Search Task.

OBJECTIVE: To understand the impact of time pressure and automated decision support systems (DSS) in a simulated medical visual search task. BACKGROUND: Time pressure usually impairs manual performance in visual search tasks, but DSS support might neutralize this negative effect. Moreover, understanding the impact of time pressure and DSS support seems relevant for many real-world applications of visual search. METHOD: We used a visual search paradigm where participants had to search for target letters in a simulated medical image. Participants performed the task either manually or with support of a highly reliable DSS. Time pressure was varied within-subjects by either a trialwise time-pressure manipulation (Experiment 1) or a blockwise manipulation (Experiment 2). Performance was assessed based on signal detection measures. To further analyze visual search behavior, a mouse-over approach was used. RESULTS: In both experiments, results showed impaired sensitivity under high compared to low time pressure in the manual condition, but no negative effect of time pressure when working with a highly reliable DSS. Moreover, participants searched less under time pressure and when receiving DSS support, indicating participants followed the automation without thoroughly checking recommendations. However, the human-DSS team's sensitivity was always worse than that of the DSS alone, independent of the strength of time pressure. CONCLUSION: Negative effects of time pressure can be ameliorated when receiving support by a DSS, but joint overall performance remains below DSS-alone performance. APPLICATION: Highly reliable DSS seem capable of ameliorating the negative impact of time pressure in complex detection tasks.

Human Performance Consequences of Automated Decision Aids: The Impact of Time Pressure.

OBJECTIVE: The study addresses the impact of time pressure on human interactions with automated decision support systems (DSSs) and related performance consequences. BACKGROUND: When humans interact with DSSs, this often results in worse performance than could be expected from the automation alone. Previous research has suggested that time pressure might make a difference by leading humans to rely more on a DSS. METHOD: In two laboratory experiments, participants performed a luggage screening task either manually, supported by a highly reliable DSS, or by a low reliable DSS. Time provided for inspecting the X-rays was 4.5 s versus 9 s varied within-subjects as the time pressure manipulation. Participants in the automation conditions were either shown the automation's advice prior (Experiment 1) or following (Experiment 2) their own inspection, before they made their final decision. RESULTS: In Experiment 1, time pressure compromised performance independent of whether the task was performed manually or with automation support. In Experiment 2, the negative impact of time pressure was only found in the manual but not in the two automation conditions. However, neither experiment revealed any positive impact of time pressure on overall performance, and the joint performance of human and automation was mostly worse than the performance of the automation alone. CONCLUSION: Time pressure compromises the quality of decision-making. Providing a DSS can reduce this effect, but only if the automation's advice follows the assessment of the human. APPLICATION: The study provides suggestions for the effective implementation of DSSs in addition to supporting concerns that highly reliable DSSs are not used optimally by human operators.

Knowing When to Pass: The Effect of AI Reliability in Risky Decision Contexts.

OBJECTIVE: This study manipulates the presence and reliability of AI recommendations for risky decisions to measure the effect on task performance, behavioral consequences of trust, and deviation from a probability matching collaborative decision-making model. BACKGROUND: Although AI decision support improves performance, people tend to underutilize AI recommendations, particularly when outcomes are uncertain. As AI reliability increases, task performance improves, largely due to higher rates of compliance (following action recommendations) and reliance (following no-action recommendations). METHODS: In a between-subject design, participants were assigned to a high reliability AI, low reliability AI, or a control condition. Participants decided whether to bet that their team would win in a series of basketball games tying compensation to performance. We evaluated task performance (in accuracy and signal detection terms) and the behavioral consequences of trust (via compliance and reliance). RESULTS: AI recommendations improved task performance, had limited impact on risk-taking behavior, and were under-valued by participants. Accuracy, sensitivity (d'), and reliance increased in the high reliability AI condition, but there was no effect on response bias (c) or compliance. Participant behavior was only consistent with a probability matching model for compliance in the low reliability condition. CONCLUSION: In a pay-off structure that incentivized risk-taking, the primary value of the AI recommendations was in determining when to perform no action (i.e., pass on bets). APPLICATION: In risky contexts, designers need to consider whether action or no-action recommendations will be more influential to design appropriate interventions.

Investigating limits of task prioritization in dual-tasking: evidence from the prioritized processing and the psychological refractory period paradigms.

Dual-tasking often requires prioritizing one task over the other. For example, in the psychological refractory period (PRP) paradigm, participants are instructed to initially respond to Task 1 (T1) and only then to Task 2 (T2). Furthermore, in the prioritized processing paradigm (PP), participants are instructed to perform T2 only if T1 was a no-go trial-requiring even more prioritization. The present study investigated the limits of task prioritization. Two experiments compared performance in the PRP paradigm and the PP paradigm. To manipulate task prioritization, tasks were rewarded differently (e.g., high reward for T1, low reward for T2, and vice versa). We hypothesized (a) that performance will improve for the highly rewarded task (Experiments 1 and 2) and (b) that there are stronger reward effects for T1 in the PRP than in the PP paradigm (Experiment 2). Results showed an influence of reward on task prioritization: For T1, high reward (compared to low reward) caused a speed-up of responses that did not differ between the two paradigms. However, for T2, reward influenced response speed selectively in the PP paradigm, but not in the PRP paradigm. Based on paradigm-specific response demands, we propose that the coordination of two motor responses plays a crucial role in prioritizing tasks and might limit the flexibility of the allocation of preparatory capacity.

Are model parameters linked to processing stages? An empirical investigation for the ex-Gaussian, ex-Wald, and EZ diffusion models.

In previous research, the parameters of the ex-Gaussian distribution have been subject to a wide variety of interpretations. The present study investigated whether the ex-Gaussian model is capable of distinguishing effects on separate processing stages (i.e., pre-motor vs. motor). In order to do so, we used datasets where the locus of effect was quite clear. Specifically, we analyzed data from experiments comparing hand vs. foot responses-presumably differing in the motor stage-and from experiments in which the lateralized readiness potential was used to localize experimental effects into premotor vs. motor processes. Moreover, we broadened the scope to two other descriptive RT models: the ex-Wald and EZ diffusion models. To the extent possible with each of these models, we reanalyzed the RT data of 19 clearly localized experimental effects from 12 separate experiments reported in seven previously published articles. Unfortunately, we did not find a clear pattern of results for any of the models, with no clear link between effects on one of the model's parameters and effects on different processing stages. The present results suggest that one should resist the temptation to associate specific processing stages with individual parameters of the ex-Gaussian, ex-Wald, and EZ diffusion models.

Challenging presumed technological superiority when working with (artificial) colleagues.

Technological advancements are ubiquitously supporting or even replacing humans in all areas of life, bringing the potential for human-technology symbiosis but also novel challenges. To address these challenges, we conducted three experiments in different task contexts ranging from loan assignment over X-Ray evaluation to process industry. Specifically, we investigated the impact of support agent (artificial intelligence, decision support system, or human) and failure experience (one vs. none) on trust-related aspects of human-agent interaction. This included not only the subjective evaluation of the respective agent in terms of trust, reliability, and responsibility, when working together, but also a change in perspective to the willingness to be assessed oneself by the agent. In contrast to a presumed technological superiority, we show a general advantage with regard to trust and responsibility of human support over both technical support systems (i.e., artificial intelligence and decision support system), regardless of task context from the collaborative perspective. This effect reversed to a preference for technical systems when switching the perspective to being assessed. These findings illustrate an imperfect automation schema from the perspective of the advice-taker and demonstrate the importance of perspective when working with or being assessed by machine intelligence.

A global and local perspective of interruption frequency in a visual search task.

We investigated the impact of frequency of interruptions in a simulated medical visual search task. Participants (N = 150) performed the visual search task during which they were interrupted by a number-classification task in 25, 50, or 75% of all trials, respectively, reflecting the frequency conditions (i.e., low, mid, high). Target presence (i.e., present vs. absent) and interruption (i.e., uninterrupted vs. interrupted) were varied within-subjects, and interruption frequency was varied between-subjects. Globally, on a frequency condition level, participants in the low frequency condition had longer mean response times (RT) for the primary visual search task than in the high condition, but there were no other performance differences between the three frequency conditions. Locally, on the level of specific interruption effects, accuracy decreased directly after interruptions for target present but not for target absent trials. Furthermore, interruptions caused significant interruption costs, reflected in slower overall RTs in interrupted than in uninterrupted trials. The combined findings show that especially for critical visual search tasks as in the medical field, interruptions-regardless of frequency-should be avoided.

Determinants of Laypersons' Trust in Medical Decision Aids: Randomized Controlled Trial.

BACKGROUND: Symptom checker apps are patient-facing decision support systems aimed at providing advice to laypersons on whether, where, and how to seek health care (disposition advice). Such advice can improve laypersons' self-assessment and ultimately improve medical outcomes. Past research has mainly focused on the accuracy of symptom checker apps' suggestions. To support decision-making, such apps need to provide not only accurate but also trustworthy advice. To date, only few studies have addressed the question of the extent to which laypersons trust symptom checker app advice or the factors that moderate their trust. Studies on general decision support systems have shown that framing automated systems (anthropomorphic or emphasizing expertise), for example, by using icons symbolizing artificial intelligence (AI), affects users' trust. OBJECTIVE: This study aims to identify the factors influencing laypersons' trust in the advice provided by symptom checker apps. Primarily, we investigated whether designs using anthropomorphic framing or framing the app as an AI increases users' trust compared with no such framing. METHODS: Through a web-based survey, we recruited 494 US residents with no professional medical training. The participants had to first appraise the urgency of a fictitious patient description (case vignette). Subsequently, a decision aid (mock symptom checker app) provided disposition advice contradicting the participants' appraisal, and they had to subsequently reappraise the vignette. Participants were randomized into 3 groups: 2 experimental groups using visual framing (anthropomorphic, 160/494, 32.4%, vs AI, 161/494, 32.6%) and a neutral group without such framing (173/494, 35%). RESULTS: Most participants (384/494, 77.7%) followed the decision aid's advice, regardless of its urgency level. Neither anthropomorphic framing (odds ratio 1.120, 95% CI 0.664-1.897) nor framing as AI (odds ratio 0.942, 95% CI 0.565-1.570) increased behavioral or subjective trust (P=.99) compared with the no-frame condition. Even participants who were extremely certain in their own decisions (ie, 100% certain) commonly changed it in favor of the symptom checker's advice (19/34, 56%). Propensity to trust and eHealth literacy were associated with increased subjective trust in the symptom checker (propensity to trust b=0.25; eHealth literacy b=0.2), whereas sociodemographic variables showed no such link with either subjective or behavioral trust. CONCLUSIONS: Contrary to our expectation, neither the anthropomorphic framing nor the emphasis on AI increased trust in symptom checker advice compared with that of a neutral control condition. However, independent of the interface, most participants trusted the mock app's advice, even when they were very certain of their own assessment. Thus, the question arises as to whether laypersons use such symptom checkers as substitutes rather than as aids in their own decision-making. With trust in symptom checkers already high at baseline, the benefit of symptom checkers depends on interface designs that enable users to adequately calibrate their trust levels during usage. TRIAL REGISTRATION: Deutsches Register Klinischer Studien DRKS00028561; https://tinyurl.com/rv4utcfb (retrospectively registered).

Visual search behavior and performance in luggage screening: effects of time pressure, automation aid, and target expectancy.

Visual inspection of luggage using X-ray technology at airports is a time-sensitive task that is often supported by automated systems to increase performance and reduce workload. The present study evaluated how time pressure and automation support influence visual search behavior and performance in a simulated luggage screening task. Moreover, we also investigated how target expectancy (i.e., targets appearing in a target-often location or not) influenced performance and visual search behavior. We used a paradigm where participants used the mouse to uncover a portion of the screen which allowed us to track how much of the stimulus participants uncovered prior to their decision. Participants were randomly assigned to either a high (5-s time per trial) or a low (10-s time per trial) time-pressure condition. In half of the trials, participants were supported by an automated diagnostic aid (85% reliability) in deciding whether a threat item was present. Moreover, within each half, in target-present trials, targets appeared in a predictable location (i.e., 70% of targets appeared in the same quadrant of the image) to investigate effects of target expectancy. The results revealed better detection performance with low time pressure and faster response times with high time pressure. There was an overall negative effect of automation support because the automation was only moderately reliable. Participants also uncovered a smaller amount of the stimulus under high time pressure in target-absent trials. Target expectancy of target location improved accuracy, speed, and the amount of uncovered space needed for the search.Significance Statement Luggage screening is a safety-critical real-world visual search task which often has to be done under time pressure. The present research found that time pressure compromises performance and increases the risk to miss critical items even with automation support. Moreover, even highly reliable automated support may not improve performance if it does not exceed the manual capabilities of the human screener. Lastly, the present research also showed that heuristic search strategies (e.g., areas where targets appear more often) seem to guide attention also in luggage screening.

Disentangling stimulus and response compatibility as potential sources of backward crosstalk.

In two experiments (N= 60 each), we investigated the locus of backward crosstalk effects in dual tasking. Specifically, we embedded the typical flanker task within a dual-task paradigm by assigning stimulus-response (S-R) rules to the flankers. In Experiment 1, participants were instructed to first respond to the center letter and only respond to the flanker if the center was a no-go stimulus (i.e., prioritized processing paradigm). Mapping condition was varied between-subjects to be either matched (i.e., same S-R rule for flankers as for center letters), reversed (i.e., opposite S-R rule for flankers), or neutral (i.e., different letters for flankers with separate S-R rules). The results indicated that the backward crosstalk effect was mainly driven by a stimulus-based compatibility, as indicated by a significant S2-R1 compatibility effect in the matched and reversed conditions, with little change in this effect between the matched and reversed conditions. Experiment 2 replicated and extended these findings to a psychological refractory period paradigm. The present findings suggest that in the matched and reversed conditions, there was only one S-R rule active at a time.