Human Error as an Emergent Property of Action Selection and Task Place-Holding.

OBJECTIVE: A computational process model could explain how the dynamic interaction of human cognitive mechanisms produces each of multiple error types. BACKGROUND: With increasing capability and complexity of technological systems, the potential severity of consequences of human error is magnified. Interruption greatly increases people's error rates, as does the presence of other information to maintain in an active state. METHOD: The model executed as a software-instantiated Monte Carlo simulation. It drew on theoretical constructs such as associative spreading activation for prospective memory, explicit rehearsal strategies as a deliberate cognitive operation to aid retrospective memory, and decay. RESULTS: The model replicated the 30% effect of interruptions on postcompletion error in Ratwani and Trafton's Stock Trader task, the 45% interaction effect on postcompletion error of working memory capacity and working memory load from Byrne and Bovair's Phaser Task, as well as the 5% perseveration and 3% omission effects of interruption from the UNRAVEL Task. CONCLUSION: Error classes including perseveration, omission, and postcompletion error fall naturally out of the theory. APPLICATION: The model explains post-interruption error in terms of task state representation and priming for recall of subsequent steps. Its performance suggests that task environments providing more cues to current task state will mitigate error caused by interruption. For example, interfaces could provide labeled progress indicators or facilities for operators to quickly write notes about their task states when interrupted.

Validating and Refining Cognitive Process Models Using Probabilistic Graphical Models.

We describe a new approach for developing and validating cognitive process models. In our methodology, graphical models (specifically, hidden Markov models) are developed both from human empirical data on a task and synthetic data traces generated by a cognitive process model of human behavior on the task. Differences between the two graphical models can then be used to drive model refinement. We show that iteratively using this methodology can unveil substantive and nuanced imperfections of cognitive process models that can then be addressed to increase their fidelity to empirical data.

Understanding Is a Process.

How do we gauge understanding? Tests of understanding, such as Turing's imitation game, are numerous; yet, attempts to achieve a state of understanding are not satisfactory assessments. Intelligent agents designed to pass one test of understanding often fall short of others. Rather than approaching understanding as a system state, in this paper, we argue that understanding is a process that changes over time and experience. The only window into the process is through the lens of natural language. Usefully, failures of understanding reveal breakdowns in the process. We propose a set of natural language-based probes that can be used to map the degree of understanding a human or intelligent system has achieved through combinations of successes and failures.

A perceptual process approach to selecting color scales for complex visualizations.

Previous research has shown that multicolored scales are superior to ordered brightness scales for supporting identification tasks on complex visualizations (categorization, absolute numeric value judgments, etc.), whereas ordered brightness scales are superior for relative comparison tasks (greater/less). We examined the processes by which such tasks are performed. By studying eye movements and by comparing performance on scales of different sizes, we argued that (a) people perform identification tasks by conducting a serial visual search of the legend, whose speed is sensitive to the number of scale colors and the discriminability of the colors; and (b) people perform relative comparison tasks using different processes for multicolored versus brightness scales. With multicolored scales, they perform a parallel search of the legend, whose speed is relatively insensitive to the size of the scale, whereas with brightness scales, people usually directly compare the target colors in the visualization, while making little reference to the legend. Performance of comparisons was relatively robust against increases in scale size, whereas performance of identifications deteriorated markedly, especially with brightness scales, once scale sizes reached 10 colors or more.

Shedding light on the graph schema: perceptual features versus invariant structure.

Most theories of graph comprehension posit the existence of a graph schema to account for people's prior knowledge of how to understand different graph types. The graph schema is, however, a purely theoretical construct: No empirical studies have explicitly examined the nature of the graph schema. We sought to determine whether graph schemas are based on perceptual features or on a common invariant structure shared between certain graph types. We isolated the process of activating the graph schema by presenting the graphs to participants in pure and mixed blocks. Any differences in reaction time between the blocks could be attributed to loading the appropriate schema. Results from a series of experiments using five types of graphs suggest that graph schemas are based on the graphical framework, a common invariant structure among certain types of graphs. These results provide insight into the comprehension of novel graphs.

The effect of interruption duration and demand on resuming suspended goals.

The time to resume task goals after an interruption varied depending on the duration and cognitive demand of interruptions, as predicted by the memory for goals model (Altmann & Trafton, 2002). Three experiments using an interleaved tasks interruption paradigm showed that longer and more demanding interruptions led to longer resumption times in a hierarchical, interactive task. The resumption time profile for durations up to 1 min supported the role of decay in defining resumption costs, and the interaction between duration and demand supported the importance of goal rehearsal in mitigating decay. These findings supported the memory for goals model, and had practical implications for context where tasks are frequently interleaved such as office settings, driving, emergency rooms, and aircraft cockpits.

Thinking graphically: Connecting vision and cognition during graph comprehension.

Task analytic theories of graph comprehension account for the perceptual and conceptual processes required to extract specific information from graphs. Comparatively, the processes underlying information integration have received less attention. We propose a new framework for information integration that highlights visual integration and cognitive integration. During visual integration, pattern recognition processes are used to form visual clusters of information; these visual clusters are then used to reason about the graph during cognitive integration. In 3 experiments, the processes required to extract specific information and to integrate information were examined by collecting verbal protocol and eye movement data. Results supported the task analytic theories for specific information extraction and the processes of visual and cognitive integration for integrative questions. Further, the integrative processes scaled up as graph complexity increased, highlighting the importance of these processes for integration in more complex graphs. Finally, based on this framework, design principles to improve both visual and cognitive integration are described.

Examining the Role of Task Requirements in the Magnitude of the Vigilance Decrement.

The vigilance decrement in sustained attention tasks is a prevalent example of cognitive fatigue in the literature. A critical challenge for current theories is to account for differences in the magnitude of the vigilance decrement across tasks that involve memory (successive tasks) and those that do not (simultaneous tasks). The empirical results described in this paper examine this issue by comparing performance, including eye movement data, between successive and simultaneous tasks that require multiple fixations to encode the stimulus for each trial. The findings show that differences in the magnitude of the vigilance decrement between successive and simultaneous tasks were observed only when a response deadline was imposed in the analysis of reaction times. This suggests that memory requirements did not exacerbate the deleterious impacts of time on task on the ability to accurately identify the critical stimuli. At the same time, eye tracking data collected during the study provided evidence for disruptions in cognitive processing that manifested as increased delays between fixations on stimulus elements and between encoding the second stimulus element and responding. These delays were particularly pronounced in later stages of encoding and responding. The similarity of the findings for both tasks suggests that the vigilance decrement may arise from common mechanisms in both cases. Differences in the magnitude of the decrement arise as a function of how degraded cognitive processing interacts with differences in the information processing requirements and other task characteristics. The findings are consistent with recent accounts of the vigilance decrement, which integrate features of prior theoretical perspectives.

Timecourse of recovery from task interruption: data and a model.

Interruption of a complex cognitive task can entail, for the "interruptee", a sense of having to recover afterward. We examined this recovery process by measuring the timecourse of responses following an interruption, sampling over 13,000 interruptions to obtain stable data. Response times dropped in a smooth curvilinear pattern for the first 10 responses (15 sec or so) of postinterruption performance. We explain this pattern in terms of the cognitive system retrieving a displaced mental context from memory incrementally, with each retrieved element adding to the set of primes facilitating the next retrieval. The model explains a learning effect in our data in which the timecourse of recovery changes over blocks, and is generally consistent with current representational theories of expertise.

"What if ": The Use of Conceptual Simulations in Scientific Reasoning.

The term conceptual simulation refers to a type of everyday reasoning strategy commonly called "what if" reasoning. It has been suggested in a number of contexts that this type of reasoning plays an important role in scientific discovery; however, little direct evidence exists to support this claim. This article proposes that conceptual simulation is likely to be used in situations of informational uncertainty, and may be used to help scientists resolve that uncertainty. We conducted two studies to investigate the relationship between conceptual simulation and informational uncertainty. Study 1 was an in vivo study of expert scientists; the results suggest that scientists do use conceptual simulation in situations of informational uncertainty, and that they use conceptual simulation to make inferences from their data using the analogical reasoning process of alignment by similarity detection. Study 2 experimentally manipulated experts' level of uncertainty and provides further support for the hypothesis that conceptual simulation is more likely to be used in situations of informational uncertainty. Finally, we discuss the relationship between conceptual simulation and other types of reasoning using qualitative mental models.

Familiarity, Priming, and Perception in Similarity Judgments.

We present a novel way of accounting for similarity judgments. Our approach posits that similarity stems from three main sources-familiarity, priming, and inherent perceptual likeness. Here, we explore each of these constructs and demonstrate their individual and combined effectiveness in explaining similarity judgments. Using these three measures, our account of similarity explains ratings of simple, color-based perceptual stimuli that display asymmetry effects, as well as more complicated perceptual stimuli with structural properties; more traditional approaches to similarity solve one or the other and have difficulty accounting for both. Overall, our work demonstrates the importance of each of these components of similarity in explaining similarity judgments, both individually and together, and suggests important implications for other similarity approaches.

Effects of interruption length on procedural errors.

We investigated effects of task interruption on procedural performance, focusing on the effect of interruption length on the rates of different categories of error at the point of task resumption. Interruption length affected errors involving loss of place in the procedure (sequence errors) but not errors involving incorrect execution of a correct step (nonsequence errors), implicating memory for past performance, rather than generalized attentional resources, as the disrupted cognitive process. Within the category of sequence errors, interruption length produced a complex pattern of effects, with repetitions of the preinterruption step showing different effects than errors at other offsets from the correct step. A cognitive model we developed previously accounts for the results in terms of decay and rehearsal of memory for past performance and activation spreading through a procedural representation of task knowledge. The model links different types of errors to different cognitive processes, informs potential interventions, and predicts interruption effects for sequential tasks like problem solving and counting. (PsycINFO Database Record

An Account of Interference in Associative Memory: Learning the Fan Effect.

Associative learning is an essential feature of human cognition, accounting for the influence of priming and interference effects on memory recall. Here, we extend our account of associative learning that learns asymmetric item-to-item associations over time via experience (Thomson, Pyke, Trafton, & Hiatt, 2015) by including link maturation to balance associations between longer-term stability while still accounting for short-term variability. This account, combined with an existing account of activation strengthening and decay, predicts both human response times and error rates for the fan effect (Anderson, 1974; Anderson & Reder, 1999) for both target and foil stimuli.

Immediate inferences from quantified assertions.

We propose a theory of immediate inferences from assertions containing a single quantifier, such as: All of the artists are bakers; therefore, some of the bakers are artists. The theory is based on mental models and is implemented in a computer program, mReasoner. It predicts three main levels of increasing difficulty: (a) immediate inferences in which the premise and conclusion have identical meanings; (b) those in which the initial mental model of the premise yields the correct conclusion; and (c) those in which only an alternative to the initial model establishes the correct conclusion. These levels of difficulty were corroborated for inferences to necessary conclusions (in a reanalysis of data from Newstead, S. E., & Griggs, R. A. (1983). Drawing inferences from quantified statements: A study of the square of opposition. Journal of Verbal Learning and Verbal Behavior, 22, 535-546), for inferences to modal conclusions, such as, it is possible that all of the bakers are artists (Experiment 1), for inferences with unorthodox quantifiers, such as, most of the artists (Experiment 2), and for inferences about the consistency of pairs of quantified assertions (Experiment 3). The theory also includes three parameters in a stochastic system that predicted quantitative differences in accuracy within the three main sorts of inference.

Episodes, events, and models.

We describe a novel computational theory of how individuals segment perceptual information into representations of events. The theory is inspired by recent findings in the cognitive science and cognitive neuroscience of event segmentation. In line with recent theories, it holds that online event segmentation is automatic, and that event segmentation yields mental simulations of events. But it posits two novel principles as well: first, discrete episodic markers track perceptual and conceptual changes, and can be retrieved to construct event models. Second, the process of retrieving and reconstructing those episodic markers is constrained and prioritized. We describe a computational implementation of the theory, as well as a robotic extension of the theory that demonstrates the processes of online event segmentation and event model construction. The theory is the first unified computational account of event segmentation and temporal inference. We conclude by demonstrating now neuroimaging data can constrain and inspire the construction of process-level theories of human reasoning.

Momentary interruptions can derail the train of thought.

We investigated the effect of short interruptions on performance of a task that required participants to maintain their place in a sequence of steps each with their own performance requirements. Interruptions averaging 4.4 s long tripled the rate of sequence errors on post-interruption trials relative to baseline trials. Interruptions averaging 2.8 s long--about the time to perform a step in the interrupted task--doubled the rate of sequence errors. Nonsequence errors showed no interruption effects, suggesting that global attentional processes were not disrupted. Response latencies showed smaller interruption effects than sequence errors, a difference we interpret in terms of high levels of interference generated by the primary task. The results are consistent with an account in which activation spreading from the focus of attention allows control processes to navigate task-relevant representations and in which momentary interruptions are disruptive because they shift the focus and thereby cut off the flow.

Visualizing uncertainty: the impact on performance.

OBJECTIVE: This work investigated the impact of uncertainty representation on performance in a complex authentic visualization task, submarine localization. BACKGROUND: Because passive sonar does not provide unique course, speed, and range information on a contact, the submarine operates under significant uncertainty. There are many algorithms designed to address this problem, but all are subject to uncertainty. The extent of this solution uncertainty can be expressed in several ways, including a table of locations (course, speed, range) or a graphical area of uncertainty. METHOD: To test the hypothesis that the representation of uncertainty that more closely matches the experts' preferred representation of the problem would better support performance, even for the nonexpert, performance data were collected using displays that were either stripped of the spatial or the tabular representation. RESULTS: Performance was more accurate when uncertainty was displayed spatially. This effect was only significant for the nonexperts for whom the spatial displays supported almost expert-like performance. This effect appears to be due to reduced mental effort. CONCLUSION: These results suggest that when the representation of uncertainty for this spatial task better matches the expert's preferred representation of the problem even a nonexpert can show expert-like performance. APPLICATION: These results could apply to any domain where performance requires working with highly uncertain information.

Embodied spatial cognition.

We present a spatial system called Specialized Egocentrically Coordinated Spaces embedded in an embodied cognitive architecture (ACT-R Embodied). We show how the spatial system works by modeling two different developmental findings: gaze-following and Level 1 perspective taking. The gaze-following model is based on an experiment by Corkum and Moore (1998), whereas the Level 1 visual perspective-taking model is based on an experiment by Moll and Tomasello (2006). The models run on an embodied robotic system.

The perception of humanness from the movements of synthetic agents.

As technology develops, social robots and synthetic avatars might begin to play more of a role in our lives. An influential theory of the perception of synthetic agents states that as they begin to look and move in a more human-like way, they elicit profound discomfort in the observer--an effect known as the Uncanny Valley. Previous attempts to examine the existence of the Uncanny Valley have not adequately manipulated movement parameters that contribute to perceptions of the humanness or eeriness. Here we parametrically manipulated three different kinematic features of two walking avatars and found that, contrary to the Uncanny Valley hypothesis, ratings of the humanness, familiarity, and eeriness of these avatars changed monotonically. Our results indicate that, when a full gradient of motion parameter changes is examined, ratings of synthetic agents by human observers do not show an Uncanny Valley.