Effects of machine learning errors on human decision-making: manipulations of model accuracy, error types, and error importance.

This study addressed the cognitive impacts of providing correct and incorrect machine learning (ML) outputs in support of an object detection task. The study consisted of five experiments that manipulated the accuracy and importance of mock ML outputs. In each of the experiments, participants were given the T and L task with T-shaped targets and L-shaped distractors. They were tasked with categorizing each image as target present or target absent. In Experiment 1, they performed this task without the aid of ML outputs. In Experiments 2-5, they were shown images with bounding boxes, representing the output of an ML model. The outputs could be correct (hits and correct rejections), or they could be erroneous (false alarms and misses). Experiment 2 manipulated the overall accuracy of these mock ML outputs. Experiment 3 manipulated the proportion of different types of errors. Experiments 4 and 5 manipulated the importance of specific types of stimuli or model errors, as well as the framing of the task in terms of human or model performance. These experiments showed that model misses were consistently harder for participants to detect than model false alarms. In general, as the model's performance increased, human performance increased as well, but in many cases the participants were more likely to overlook model errors when the model had high accuracy overall. Warning participants to be on the lookout for specific types of model errors had very little impact on their performance. Overall, our results emphasize the importance of considering human cognition when determining what level of model performance and types of model errors are acceptable for a given task.

Studying visual search without an eye tracker: an assessment of artificial foveation.

Eye tracking is a useful tool for studying human cognition, both in the laboratory and in real-world applications. However, there are cases in which eye tracking is not possible, such as in high-security environments where recording devices cannot be introduced. After facing this challenge in our own work, we sought to test the effectiveness of using artificial foveation as an alternative to eye tracking for studying visual search performance. Two groups of participants completed the same list comparison task, which was a computer-based task designed to mimic an inventory verification process that is commonly performed by international nuclear safeguards inspectors. We manipulated the way in which the items on the inventory list were ordered and color coded. For the eye tracking group, an eye tracker was used to assess the order in which participants viewed the items and the number of fixations per trial in each list condition. For the artificial foveation group, the items were covered with a blurry mask except when participants moused over them. We tracked the order in which participants viewed the items by moving their mouse and the number of items viewed per trial in each list condition. We observed the same overall pattern of performance for the various list display conditions, regardless of the method. However, participants were much slower to complete the task when using artificial foveation and had more variability in their accuracy. Our results indicate that the artificial foveation method can reveal the same pattern of differences across conditions as eye tracking, but it can also impact participants' task performance.

Where are we going and where have we been? Examining the effects of maps on spatial learning in an indoor guided navigation task.

BACKGROUND: International Atomic Energy Agency (IAEA) safeguards inspectors are faced with the difficult task of learning the layout of complex nuclear facilities while being escorted through the facilities. This study addresses a gap in the literature regarding how to best support the development of inspectors' spatial knowledge, given the constraint that they cannot bring digital devices into most nuclear facilities. We tested whether viewing a map before learning a guided route or carrying a map along the route enabled better spatial learning than having no exposure to a map. Moreover, we tested the impact of carrying maps with different levels of detail (simple 2D, simple 3D, or complex 2D maps) on spatial learning outcomes, as well as interactions between map type and individual differences in sense of direction. RESULTS: The results showed nearly opposite patterns of performance for participants with good and poor sense of direction scores. Participants with a good sense of direction showed higher levels of spatial knowledge when studying or carrying simple maps, whether 2D or 3D, but they did not benefit from using a complex map. Participants with a poor sense of direction showed lower levels of spatial knowledge when using a simple map relative to using no map or a complex map, suggesting that they did not attempt to use the complex map. For both groups of participants, referring to a simple map while learning a route decreased their awareness of their environment, as measured by response times on a memory test that included incidentally learned items.

Event-related brain potentials reveal how multiple aspects of semantic processing unfold across parafoveal and foveal vision during sentence reading.

Recent event-related brain potential (ERP) experiments have demonstrated parafoveal N400 expectancy and congruity effects, showing that semantic information can be accessed from words in parafoveal vision (a conclusion also supported by some eye-tracking work). At the same time, it is unclear how higher-order integrative aspects of language comprehension unfold across the visual field during reading. In the current study, we recorded ERPs in a parafoveal flanker paradigm, while readers were instructed to read passively for comprehension or to judge the plausibility of sentences in which target words varied in their semantic expectancy and congruity. We directly replicated prior work showing graded N400 effects for parafoveal viewing, which are then not duplicated when the target words are processed foveally. Critically, although N400 effects were not modulated by task goals, a posteriorly distributed late positive component thought to reflect semantic integration processes was observed to semantic incongruities only in the plausibility judgment task. However, this effect was observed at a considerable delay, appearing only after words had moved into foveal vision. Our findings thus suggest that semantic access can be initiated in parafoveal vision, whereas central foveal vision may be necessary to enact higher-order (and task-dependent) integrative processing.

Time will tell: A longitudinal investigation of brain-behavior relationships during reading development.

ERPs are a powerful tool for the study of reading, as they are both temporally precise and functionally specific. These are essential characteristics for studying a process that unfolds rapidly and consists of multiple, interactive subprocesses. In work with adults, clear, specific models exist linking components of the ERP with individual subprocesses of reading including orthographic decoding, phonological processing, and semantic access (e.g., Grainger & Holcomb, 2009). The relationships between ERP components and reading subprocesses are less clear in development; here, we address two questions regarding these relationships. First, we ask whether there are ERP markers that predict future reading behaviors across a longitudinal year. Second, we ask whether any relationships observed between ERP components and reading behavior across time map onto the better-established relationships between ERPs and reading subprocesses in adults. To address these questions, we acquired ERPs from children engaging in a silent reading task and then, a year later, collected behavioral assessments of their reading ability. We find that ERPs collected in Year 1 do predict reading behaviors a year later. Further, we find that these relationships do conform, at least to some extent, to relationships between ERP components and reading subprocesses observed in adults, with, for example, N250 amplitude in Year 1 predicting phonological awareness in Year 2, and N400 amplitude in Year 1 predicting vocabulary in Year 2.

Getting ahead of yourself: Parafoveal word expectancy modulates the N400 during sentence reading.

An important question in the reading literature regards the nature of the semantic information readers can extract from the parafovea (i.e., the next word in a sentence). Recent eye-tracking findings have found a semantic parafoveal preview benefit under many circumstances, and findings from event-related brain potentials (ERPs) also suggest that readers can at least detect semantic anomalies parafoveally (Barber, Van der Meij, & Kutas, Psychophysiology, 50(1), 48-59, 2013). We use ERPs to ask whether fine-grained aspects of semantic expectancy can affect the N400 elicited by a word appearing in the parafovea. In an RSVP-with-flankers paradigm, sentences were presented word by word, flanked 2° bilaterally by the previous and upcoming words. Stimuli consisted of high constraint sentences that were identical up to the target word, which could be expected, unexpected but plausible, or anomalous, as well as low constraint sentences that were always completed with the most expected ending. Findings revealed an N400 effect to the target word when it appeared in the parafovea, which was graded with respect to the target's expectancy and congruency within the sentence context. Furthermore, when targets appeared at central fixation, this graded congruency effect was mitigated, suggesting that the semantic information gleaned from parafoveal vision functionally changes the semantic processing of those words when foveated.

Out of the corner of my eye: Foveal semantic load modulates parafoveal processing in reading.

In 2 experiments, we examined the impact of foveal semantic expectancy and congruity on parafoveal word processing during reading. Experiment 1 utilized an eye-tracking gaze-contingent display change paradigm, and Experiment 2 measured event-related brain potentials (ERPs) in a modified flanker rapid serial visual presentation (RSVP) paradigm. Eye-tracking and ERP data converged to reveal graded effects of foveal load on parafoveal processing. In Experiment 1, when word n was highly expected, and thus foveal load was low, there was a large parafoveal preview benefit to word n + 1. When word n was unexpected but still plausible, preview benefits to n + 1 were reduced in magnitude, and when word n was semantically incongruent, the preview benefit to n + 1 was unreliable in early pass measures. In Experiment 2, ERPs indicated that when word n was expected, and thus foveal load was low, readers successfully discriminated between valid and orthographically invalid previews during parafoveal perception. However, when word n was unexpected, parafoveal processing of n + 1 was reduced, and it was eliminated when word n was semantically incongruent. Taken together, these findings suggest that sentential context modulates the allocation of attention in the parafovea, such that covert allocation of attention to parafoveal processing is disrupted when foveal words are inconsistent with expectations based on various contextual constraints. (PsycINFO Database Record

Do Morphemes Matter when Reading Compound Words with Transposed Letters? Evidence from Eye-Tracking and Event-Related Potentials.

The current study investigates the online processing consequences of encountering compound words with transposed letters (TLs), to determine if TLs that cross morpheme boundaries are more disruptive to reading than those within a single morpheme, as would be predicted by accounts of obligatory morpho-orthopgrahic decomposition. Two measures of online processing, eye movements and event-related potentials (ERPs), were collected in separate experiments. Participants read sentences containing correctly spelled compound words (cupcake), or compounds with TLs occurring either across morpheme boundaries (cucpake) or within one morpheme (cupacke). Results showed that between- and within-morpheme transpositions produced equal processing costs in both measures, in the form of longer reading times (Experiment 1) and a late posterior positivity (Experiment 2) that did not differ between conditions. Findings converge to suggest that within- and between-morpheme TLs are equally disruptive to recognition, providing evidence against obligatory morpho-orthographic processing and in favor of whole-word access of English compound words during sentence reading.

Relationships between event-related potentials and behavioral and scholastic measures of reading ability: A large-scale, cross-sectional study.

In the cognitive, computational, neuropsychological, and educational literatures, it is established that children approach text in unique ways, and that even adult readers can differ in the strategies they bring to reading. In the developmental event-related potential (ERP) literature, however, children with differing degrees of reading ability are, the majority of the time, placed in monolithic groups such as 'normal' and 'dyslexic' (e.g. Araújo et al., 2012) and analyzed only at the group level. This is likely done due to methodological concerns - such as low sample size or a lack of statistical power - that can make it difficult to perform analysis at the individual level. Here, we collected ERPs and behavior from > 100 children in grades pre-K-7, as they read unconnected text silently to themselves. This large sample, combined with the statistical power of the Linear Mixed Effects Regression (LMER) technique, enables us to address individual differences in ERP component effects due to reading ability at an unprecedented level of detail. Results indicate that it is possible to predict reading-related report card scores from ERP component amplitudes - especially that of the N250, a component pertaining to sublexical processing (including phonological decoding). Results also reveal relationships between behavioral measures of reading ability and ERP component effects that have previously been elusive, such as the relationship between vocabulary and N400 mean amplitude (cf. Henderson et al., 2011). We conclude that it is possible to meaningfully examine reading-related ERP effects at the single subject level in developing readers, and that this type of analysis can provide novel insights into both behavior and scholastic achievement.

Subsequent to suppression: Downstream comprehension consequences of noun/verb ambiguity in natural reading.

We used eye tracking to investigate the downstream processing consequences of encountering noun/verb (NV) homographs (i.e., park) in semantically neutral but syntactically constraining contexts. Target words were followed by a prepositional phrase containing a noun that was plausible for only 1 meaning of the homograph. Replicating previous work, we found increased first fixation durations on NV homographs compared with unambiguous words, which persisted into the next sentence region. At the downstream noun, we found plausibility effects following ambiguous words that were correlated with the size of a reader's first fixation effect, suggesting that this effect reflects the recruitment of processing resources necessary to suppress the homograph's context-inappropriate meaning. Using these same stimuli, Lee and Federmeier (2012) found a sustained frontal negativity to the NV homographs, and, on the downstream noun, found a plausibility effect that was also positively correlated with the size of a reader's ambiguity effect. Together, these findings suggest that when only syntactic constraints are available, meaning selection recruits inhibitory mechanisms that can be measured in both first fixation slowdown and event-related potential ambiguity effects.

Cross-age comparisons reveal multiple strategies for lexical ambiguity resolution during natural reading.

Eye tracking was used to investigate how younger and older (60 or more years) adults use syntactic and semantic information to disambiguate noun/verb (NV) homographs (e.g., park). In event-related potential (ERP) work using the same materials, Lee and Federmeier (2009, 2011) found that young adults elicited a sustained frontal negativity to NV homographs when only syntactic cues were available (i.e., in syntactic prose); this effect was eliminated by semantic constraints. The negativity was only present in older adults with high verbal fluency. The current study shows parallel findings: Young adults exhibit inflated first fixation durations to NV homographs in syntactic prose, but not semantically congruent sentences. This effect is absent in older adults as a group. Verbal fluency modulates the effect in both age groups: High fluency is associated with larger first fixation effects in syntactic prose. Older, but not younger, adults also show significantly increased rereading of the NV homographs in syntactic prose. Verbal fluency modulates this effect as well: High fluency is associated with a reduced tendency to reread, regardless of age. This relationship suggests a trade-off between initial and downstream processing costs for ambiguity during natural reading. Together the eye-tracking and ERP data suggest that effortful meaning selection recruits mechanisms important for suppressing contextually inappropriate meanings, which also slow eye movements. Efficacy of frontotemporal circuitry, as captured by verbal fluency, predicts the success of engaging these mechanisms in both young and older adults. Failure to recruit these processes requires compensatory rereading or leads to comprehension failures (Lee & Federmeier, 2012).

The psychologist said quickly, "dialogue descriptions modulate reading speed!".

In the present study, we investigated whether the semantic content of a dialogue description can affect reading times on an embedded quote, to determine whether the speed at which a character is described as saying a quote influences how quickly it is read. Yao and Scheepers (Cognition, 121:447-453, 2011) previously found that readers were faster to read direct quotes when the preceding context implied that the talker generally spoke quickly, an effect attributed to perceptual simulation of talker speed. For the present study, we manipulated the speed of a physical action performed by the speaker independently from character talking rate to determine whether these sources have separable effects on perceptual simulation of a direct quote. The results showed that readers spent less time reading direct quotes described as being said quickly, as compared to those described as being said slowly (e.g., John walked/bolted into the room and said energetically/nonchalantly, "I finally found my car keys."), an effect that was not present when a nearly identical phrase was presented as an indirect quote (e.g., John . . . said energetically that he finally found his car keys.). The speed of the character's movement did not affect direct-quote reading times. Furthermore, fast adverbs were themselves read significantly faster than slow adverbs, an effect that we attribute to implicit effects on the eye movement program stemming from automatically activated semantic features of the adverbs. Our findings add to the literature on perceptual simulation by showing that these effects can be instantiated with only a single adverb and are strong enough to override the effects of global sentence speed.