Validating and Refining Cognitive Process Models Using Probabilistic Graphical Models.

We describe a new approach for developing and validating cognitive process models. In our methodology, graphical models (specifically, hidden Markov models) are developed both from human empirical data on a task and synthetic data traces generated by a cognitive process model of human behavior on the task. Differences between the two graphical models can then be used to drive model refinement. We show that iteratively using this methodology can unveil substantive and nuanced imperfections of cognitive process models that can then be addressed to increase their fidelity to empirical data.

Understanding Is a Process.

How do we gauge understanding? Tests of understanding, such as Turing's imitation game, are numerous; yet, attempts to achieve a state of understanding are not satisfactory assessments. Intelligent agents designed to pass one test of understanding often fall short of others. Rather than approaching understanding as a system state, in this paper, we argue that understanding is a process that changes over time and experience. The only window into the process is through the lens of natural language. Usefully, failures of understanding reveal breakdowns in the process. We propose a set of natural language-based probes that can be used to map the degree of understanding a human or intelligent system has achieved through combinations of successes and failures.

Examining the Role of Task Requirements in the Magnitude of the Vigilance Decrement.

The vigilance decrement in sustained attention tasks is a prevalent example of cognitive fatigue in the literature. A critical challenge for current theories is to account for differences in the magnitude of the vigilance decrement across tasks that involve memory (successive tasks) and those that do not (simultaneous tasks). The empirical results described in this paper examine this issue by comparing performance, including eye movement data, between successive and simultaneous tasks that require multiple fixations to encode the stimulus for each trial. The findings show that differences in the magnitude of the vigilance decrement between successive and simultaneous tasks were observed only when a response deadline was imposed in the analysis of reaction times. This suggests that memory requirements did not exacerbate the deleterious impacts of time on task on the ability to accurately identify the critical stimuli. At the same time, eye tracking data collected during the study provided evidence for disruptions in cognitive processing that manifested as increased delays between fixations on stimulus elements and between encoding the second stimulus element and responding. These delays were particularly pronounced in later stages of encoding and responding. The similarity of the findings for both tasks suggests that the vigilance decrement may arise from common mechanisms in both cases. Differences in the magnitude of the decrement arise as a function of how degraded cognitive processing interacts with differences in the information processing requirements and other task characteristics. The findings are consistent with recent accounts of the vigilance decrement, which integrate features of prior theoretical perspectives.

Effects of interruption length on procedural errors.

We investigated effects of task interruption on procedural performance, focusing on the effect of interruption length on the rates of different categories of error at the point of task resumption. Interruption length affected errors involving loss of place in the procedure (sequence errors) but not errors involving incorrect execution of a correct step (nonsequence errors), implicating memory for past performance, rather than generalized attentional resources, as the disrupted cognitive process. Within the category of sequence errors, interruption length produced a complex pattern of effects, with repetitions of the preinterruption step showing different effects than errors at other offsets from the correct step. A cognitive model we developed previously accounts for the results in terms of decay and rehearsal of memory for past performance and activation spreading through a procedural representation of task knowledge. The model links different types of errors to different cognitive processes, informs potential interventions, and predicts interruption effects for sequential tasks like problem solving and counting. (PsycINFO Database Record

An Account of Interference in Associative Memory: Learning the Fan Effect.

Associative learning is an essential feature of human cognition, accounting for the influence of priming and interference effects on memory recall. Here, we extend our account of associative learning that learns asymmetric item-to-item associations over time via experience (Thomson, Pyke, Trafton, & Hiatt, 2015) by including link maturation to balance associations between longer-term stability while still accounting for short-term variability. This account, combined with an existing account of activation strengthening and decay, predicts both human response times and error rates for the fan effect (Anderson, 1974; Anderson & Reder, 1999) for both target and foil stimuli.

Human Error as an Emergent Property of Action Selection and Task Place-Holding.

OBJECTIVE: A computational process model could explain how the dynamic interaction of human cognitive mechanisms produces each of multiple error types. BACKGROUND: With increasing capability and complexity of technological systems, the potential severity of consequences of human error is magnified. Interruption greatly increases people's error rates, as does the presence of other information to maintain in an active state. METHOD: The model executed as a software-instantiated Monte Carlo simulation. It drew on theoretical constructs such as associative spreading activation for prospective memory, explicit rehearsal strategies as a deliberate cognitive operation to aid retrospective memory, and decay. RESULTS: The model replicated the 30% effect of interruptions on postcompletion error in Ratwani and Trafton's Stock Trader task, the 45% interaction effect on postcompletion error of working memory capacity and working memory load from Byrne and Bovair's Phaser Task, as well as the 5% perseveration and 3% omission effects of interruption from the UNRAVEL Task. CONCLUSION: Error classes including perseveration, omission, and postcompletion error fall naturally out of the theory. APPLICATION: The model explains post-interruption error in terms of task state representation and priming for recall of subsequent steps. Its performance suggests that task environments providing more cues to current task state will mitigate error caused by interruption. For example, interfaces could provide labeled progress indicators or facilities for operators to quickly write notes about their task states when interrupted.

Familiarity, Priming, and Perception in Similarity Judgments.

We present a novel way of accounting for similarity judgments. Our approach posits that similarity stems from three main sources-familiarity, priming, and inherent perceptual likeness. Here, we explore each of these constructs and demonstrate their individual and combined effectiveness in explaining similarity judgments. Using these three measures, our account of similarity explains ratings of simple, color-based perceptual stimuli that display asymmetry effects, as well as more complicated perceptual stimuli with structural properties; more traditional approaches to similarity solve one or the other and have difficulty accounting for both. Overall, our work demonstrates the importance of each of these components of similarity in explaining similarity judgments, both individually and together, and suggests important implications for other similarity approaches.

Episodes, events, and models.

We describe a novel computational theory of how individuals segment perceptual information into representations of events. The theory is inspired by recent findings in the cognitive science and cognitive neuroscience of event segmentation. In line with recent theories, it holds that online event segmentation is automatic, and that event segmentation yields mental simulations of events. But it posits two novel principles as well: first, discrete episodic markers track perceptual and conceptual changes, and can be retrieved to construct event models. Second, the process of retrieving and reconstructing those episodic markers is constrained and prioritized. We describe a computational implementation of the theory, as well as a robotic extension of the theory that demonstrates the processes of online event segmentation and event model construction. The theory is the first unified computational account of event segmentation and temporal inference. We conclude by demonstrating now neuroimaging data can constrain and inspire the construction of process-level theories of human reasoning.

Immediate inferences from quantified assertions.

We propose a theory of immediate inferences from assertions containing a single quantifier, such as: All of the artists are bakers; therefore, some of the bakers are artists. The theory is based on mental models and is implemented in a computer program, mReasoner. It predicts three main levels of increasing difficulty: (a) immediate inferences in which the premise and conclusion have identical meanings; (b) those in which the initial mental model of the premise yields the correct conclusion; and (c) those in which only an alternative to the initial model establishes the correct conclusion. These levels of difficulty were corroborated for inferences to necessary conclusions (in a reanalysis of data from Newstead, S. E., & Griggs, R. A. (1983). Drawing inferences from quantified statements: A study of the square of opposition. Journal of Verbal Learning and Verbal Behavior, 22, 535-546), for inferences to modal conclusions, such as, it is possible that all of the bakers are artists (Experiment 1), for inferences with unorthodox quantifiers, such as, most of the artists (Experiment 2), and for inferences about the consistency of pairs of quantified assertions (Experiment 3). The theory also includes three parameters in a stochastic system that predicted quantitative differences in accuracy within the three main sorts of inference.

Situation awareness recovery.

OBJECTIVE: We describe a novel concept, situation awareness recovery (SAR), and we identify perceptual and cognitive processes that characterize SAR. BACKGROUND: Situation awareness (SA) is typically described in terms of perceiving relevant elements of the environment, comprehending how those elements are integrated into a meaningful whole, and projecting that meaning into the future. Yet SA fluctuates during the time course of a task making it important to understand the process by which SA is recovered after it is degraded. METHOD: We investigated SAR using different types of interruptions to degrade SA. In Experiment I, participants watched short videos of an operator performing a supervisory control task, and then the participants were either interrupted or not interrupted, after which SA was assessed using a questionnaire. In Experiment 2, participants performed a supervisory control task in which they guided vehicles to their respective targets and either experienced an interruption, during which they performed a visual search task in a different panel, or were not interrupted. RESULTS: The SAR processes we identified included shorter fixation durations, increased number of objects scanned, longer resumption lags, and a greater likelihood of refixating on objects that were previously looked at. CONCLUSIONS: We interpret these findings in terms of the memory-for-goals model, which suggests that SAR consists of increased scanning in order to compensate for decay, and previously viewed cues act as associative primes that reactivate memory traces of goals and plans.

Visualizing uncertainty: the impact on performance.

OBJECTIVE: This work investigated the impact of uncertainty representation on performance in a complex authentic visualization task, submarine localization. BACKGROUND: Because passive sonar does not provide unique course, speed, and range information on a contact, the submarine operates under significant uncertainty. There are many algorithms designed to address this problem, but all are subject to uncertainty. The extent of this solution uncertainty can be expressed in several ways, including a table of locations (course, speed, range) or a graphical area of uncertainty. METHOD: To test the hypothesis that the representation of uncertainty that more closely matches the experts' preferred representation of the problem would better support performance, even for the nonexpert, performance data were collected using displays that were either stripped of the spatial or the tabular representation. RESULTS: Performance was more accurate when uncertainty was displayed spatially. This effect was only significant for the nonexperts for whom the spatial displays supported almost expert-like performance. This effect appears to be due to reduced mental effort. CONCLUSION: These results suggest that when the representation of uncertainty for this spatial task better matches the expert's preferred representation of the problem even a nonexpert can show expert-like performance. APPLICATION: These results could apply to any domain where performance requires working with highly uncertain information.

Momentary interruptions can derail the train of thought.

We investigated the effect of short interruptions on performance of a task that required participants to maintain their place in a sequence of steps each with their own performance requirements. Interruptions averaging 4.4 s long tripled the rate of sequence errors on post-interruption trials relative to baseline trials. Interruptions averaging 2.8 s long--about the time to perform a step in the interrupted task--doubled the rate of sequence errors. Nonsequence errors showed no interruption effects, suggesting that global attentional processes were not disrupted. Response latencies showed smaller interruption effects than sequence errors, a difference we interpret in terms of high levels of interference generated by the primary task. The results are consistent with an account in which activation spreading from the focus of attention allows control processes to navigate task-relevant representations and in which momentary interruptions are disruptive because they shift the focus and thereby cut off the flow.

The perception of humanness from the movements of synthetic agents.

As technology develops, social robots and synthetic avatars might begin to play more of a role in our lives. An influential theory of the perception of synthetic agents states that as they begin to look and move in a more human-like way, they elicit profound discomfort in the observer--an effect known as the Uncanny Valley. Previous attempts to examine the existence of the Uncanny Valley have not adequately manipulated movement parameters that contribute to perceptions of the humanness or eeriness. Here we parametrically manipulated three different kinematic features of two walking avatars and found that, contrary to the Uncanny Valley hypothesis, ratings of the humanness, familiarity, and eeriness of these avatars changed monotonically. Our results indicate that, when a full gradient of motion parameter changes is examined, ratings of synthetic agents by human observers do not show an Uncanny Valley.

Mitigating disruptive effects of interruptions through training: what needs to be practiced?

It is generally accepted that, with practice, people improve on most tasks. However, when tasks have multiple parts, it is not always clear what aspects of the tasks practice or training should focus on. This research explores the features that allow training to improve the ability to resume a task after an interruption, specifically focusing on task-specific versus general interruption/resumption-process mechanisms that could account for improved performance. Three experiments using multiple combinations of primary tasks and interruptions were conducted with undergraduate psychology students. The first experiment showed that for one primary and interruption task-pair, people were able to resume the primary task faster when they had previous practice with the interruption. The second experiment replicated this finding for two other sets of primary and interruption task-pairs. Finally, the third experiment showed that people were able to resume a primary task faster only when they had previous practice with that specific primary and interruption task-pair. Experience with other primary and interruption task-pairs, or practice on the primary task alone, did not facilitate resumption. This suggests that a critical component in resuming after an interruption is the relationship between two tasks. These findings are in line with a task-specific mechanism of resumption and incompatible with a general-process mechanism. These findings have practical implications for developing training programs and mitigation strategies to lessen the disruptive effects of interruptions which plague both our personal and professional environments.

Embodied spatial cognition.

We present a spatial system called Specialized Egocentrically Coordinated Spaces embedded in an embodied cognitive architecture (ACT-R Embodied). We show how the spatial system works by modeling two different developmental findings: gaze-following and Level 1 perspective taking. The gaze-following model is based on an experiment by Corkum and Moore (1998), whereas the Level 1 visual perspective-taking model is based on an experiment by Moll and Tomasello (2006). The models run on an embodied robotic system.

An eye movement analysis of the effect of interruption modality on primary task resumption.

OBJECTIVE: We examined the effect of interruption modality (visual or auditory) on primary task (visual) resumption to determine which modality was the least disruptive. BACKGROUND: Theories examining interruption modality have focused on specific periods of the interruption timeline. Preemption theory has focused on the switch from the primary task to the interrupting task. Multiple resource theory has focused on interrupting tasks that are to be performed concurrently with the primary task. Our focus was on examining how interruption modality influences task resumption.We leverage the memory-for-goals theory, which suggests that maintaining an associative link between environmental cues and the suspended primary task goal is important for resumption. METHOD: Three interruption modality conditions were examined: auditory interruption with the primary task visible, auditory interruption with a blank screen occluding the primary task, and a visual interruption occluding the primary task. Reaction time and eye movement data were collected. RESULTS: The auditory condition with the primary task visible was the least disruptive. Eye movement data suggest that participants in this condition were actively maintaining an associative link between relevant environmental cues on the primary task interface and the suspended primary task goal during the interruption. CONCLUSION: These data suggest that maintaining cue association is the important factor for reducing the disruptiveness of interruptions, not interruption modality. APPLICATION: Interruption-prone computing environments should be designed to allow for the user to have access to relevant primary task cues during an interruption to minimize disruptiveness.

Measuring search efficiency in complex visual search tasks: global and local clutter.

Set size and crowding affect search efficiency by limiting attention for recognition and attention against competition; however, these factors can be difficult to quantify in complex search tasks. The current experiments use a quantitative measure of the amount and variability of visual information (i.e., clutter) in highly complex stimuli (i.e., digital aeronautical charts) to examine limits of attention in visual search. Undergraduates at a large southern university searched for a target among 4, 8, or 16 distractors in charts with high, medium, or low global clutter. The target was in a high or low local-clutter region of the chart. In Experiment 1, reaction time increased as global clutter increased, particularly when the target was in a high local-clutter region. However, there was no effect of distractor set size, supporting the notion that global clutter is a better measure of attention against competition in complex visual search tasks. As a control, Experiment 2 demonstrated that increasing the number of distractors leads to a typical set size effect when there is no additional clutter (i.e., no chart). In Experiment 3, the effects of global and local clutter were minimized when the target was highly salient. When the target was nonsalient, more fixations were observed in high global clutter charts, indicating that the number of elements competing with the target for attention was also high. The results suggest design techniques that could improve pilots' search performance in aeronautical charts.

A generalized model for predicting postcompletion errors.

A postcompletion error is a type of procedural error that occurs after the main goal of a task has been accomplished. There is a strong theoretical foundation accounting for postcompletion errors (Altmann & Trafton, 2002; Byrne & Bovair, 1997). This theoretical foundation has been leveraged to develop a logistic regression model of postcompletion errors based on reaction time and eye movement measures (Ratwani, McCurry, & Trafton, 2008). This study further develops and extends this predictive model by (a) validating the model and the general set of predictors on a new task to test the robustness of the model, and (b) determining which specific theoretical components are most important to postcompletion error prediction.

Cognitive models of the influence of color scale on data visualization tasks.

OBJECTIVE: Computational models of identification and relative comparison tasks performed on color-coded data visualizations were presented and evaluated against two experiments. In this context, the possibility of a dual-use color scale, useful for both tasks, was explored, and the use of the legend was a major focus. BACKGROUND: Multicolored scales are superior to ordered brightness scales for identification tasks, such as determining the absolute numeric value of a represented item, whereas ordered brightness scales are superior for relative comparison tasks, such as determining which of two represented items has a greater value. METHOD: Computational models were constructed for these tasks, and their predictions were compared with the results of two experiments. RESULTS: The models fit the experimental results well. A multicolored, brightness-ordered dual-use scale supported high accuracy on both tasks and fast responses on a comparison task but relatively slower responses on the identification task. CONCLUSION: Identification tasks are solved by a serial visual search of the legend, whose speed and accuracy are a function of the discriminability of the color scales. Comparison tasks with multicolored scales are performed by a parallel search of the legend; with brightness scales, comparison tasks are generally solved by a direct comparison between colors on the visualization, without reference to the legend. Finally, it is possible to provide users a dual-use color scale effective on both tasks. APPLICATION: Trade-offs that must typically be made in the design of color-coded visualizations between speed and accuracy or between identification and comparison tasks may be mitigated.