Detecting gender as a moderator in meta-analysis: The problem of restricted between-study variance.

Meta-analyses in the psychological sciences typically examine moderators that may explain heterogeneity in effect sizes. One of the most commonly examined moderators is gender. Overall, tests of gender as a moderator are rarely significant, which may be because effects rarely differ substantially between men and women. While this may be true in some cases, we also suggest that the lack of significant findings may be attributable to the way in which gender is examined as a meta-analytic moderator, such that detecting moderating effects is very unlikely even when such effects are substantial in magnitude. More specifically, we suggest that lack of between-primary study variance in gender composition makes it exceedingly difficult to detect moderation. That is, because primary studies tend to have similar male-to-female ratios, there is very little variance in gender composition between primaries, making it nearly impossible to detect between-study differences in the relationship of interest as a function of gender. In the present article, we report results from two studies: (a) a meta-meta-analysis in which we demonstrate the magnitude of this problem by computing the between-study variance in gender composition across 286 meta-analytic moderation tests from 50 meta-analyses, and (b) a Monte Carlo simulation study in which we show that this lack of variance results in near-zero moderator effects even when male-female differences in correlations are quite large. Our simulations are also used to show the value of single-gender studies for detecting moderating effects. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Near-Perfect Automation: Investigating Performance, Trust, and Visual Attention Allocation.

OBJECTIVE: Assess performance, trust, and visual attention during the monitoring of a near-perfect automated system. BACKGROUND: Research rarely attempts to assess performance, trust, and visual attention in near-perfect automated systems even though they will be relied on in high-stakes environments. METHODS: Seventy-three participants completed a 40-min supervisory control task where they monitored three search feeds. All search feeds were 100% reliable with the exception of two automation failures: one miss and one false alarm. Eye-tracking and subjective trust data were collected. RESULTS: Thirty-four percent of participants correctly identified the automation miss, and 67% correctly identified the automation false alarm. Subjective trust increased when participants did not detect the automation failures and decreased when they did. Participants who detected the false alarm had a more complex scan pattern in the 2 min centered around the automation failure compared with those who did not. Additionally, those who detected the failures had longer dwell times in and transitioned to the center sensor feed significantly more often. CONCLUSION: Not only does this work highlight the limitations of the human when monitoring near-perfect automated systems, it begins to quantify the subjective experience and attentional cost of the human. It further emphasizes the need to (1) reevaluate the role of the operator in future high-stakes environments and (2) understand the human on an individual level and actively design for the given individual when working with near-perfect automated systems. APPLICATION: Multiple operator-level measures should be collected in real-time in order to monitor an operator's state and leverage real-time, individualized assistance.

Detecting automation failures in a simulated supervisory control environment.

The goal of this research was to determine how individuals perform and allocate their visual attention when monitoring multiple automated displays that differ in automation reliability. Ninety-six participants completed a simulated supervisory control task where each automated display had a different level of reliability (namely 70%, 85% and 95%). In addition, participants completed a high and low workload condition. The performance data revealed that (1) participants' failed to detect automation misses approximately 2.5 times more than automation false alarms, (2) participants' had worse automation failure detection in the high workload condition and (3) participant automation failure detection remained mostly static across reliability. The eye tracking data revealed that participants spread their attention relatively equally across all three of the automated displays for the duration of the experiment. Together, these data support a system-wide trust approach as the default position of an individual monitoring multiple automated displays. Practitioner Summary: Given the rapid growth of automation throughout the workforce, there is an immediate need to better understand how humans monitor multiple automated displays concurrently. The data in this experiment support a system-wide trust approach as the default position of an individual monitoring multiple automated displays. Abbreviations: DoD: Department of Defense; UA: unmanned aircraft; SCOUT: Supervisory Control Operations User Testbed; UAV: unmanned aerial vehicle; AOI: areas of interest.

Pupil size as a measure of within-task learning.

Pupillometry is commonly used in research to determine how much mental effort an individual is exerting while completing tasks. Traditionally, larger pupils are associated with increased mental effort when completing more difficult tasks. However, little research has investigated how pupils change as individuals learn a new task. In theory, as one repeatedly completes a task, the task demands should reduce, reliance on working memory should decrease, and the task should become more automatic. This should translate to faster completion times and smaller peak pupil dilations. We tested this hypothesis by having participants complete multiple trials of a cognitive task that requires individuals to orient themselves in space relative to a target. We found that trial completion times and maximum pupil size significantly reduced across trials. These data suggest that measuring changes in pupil dilation may help researchers determine whether individuals have shifted from a learned procedure to an automatic processing of information when learning a new task.

Multiple interpretations of long-term working memory theory: Reply to Delaney and Ericsson (2016).

This reply is in response to Delaney and Ericsson (2016), who argue that the results of our recent research (Foroughi, Werner, Barragán, & Boehm-Davis, 2015) can be explained by Ericsson and Kintsch's (1995) long-term working memory (LTWM) theory. Our original work was designed to test the prediction made by LTWM theory that interruptions of up to 30 s in duration would not disrupt reading performance. We conducted the work following the method and outcome measures recommended by Ericsson and Kintsch (1995). Our data were clear: interruptions disrupted reading comprehension. We believe that these data do not support predictions made by LTWM theory. Although we appreciate Delaney and Ericsson's (2016) comments, we are unsure how best to move forward because it appears that some of their comments are not consistent with the published work on LTWM theory. Because of the inconsistent and contradictory claims surrounding LTWM theory, the theory does not appear to be falsifiable, or is in danger of becoming unfalsifiable. Creating and testing theory is vital for the advancement of psychological science, but it appears that testing predictions made by LTWM would be very difficult, if not impossible, given the fluid state of the theory. (PsycINFO Database Record

Placebo effects in cognitive training.

Although a large body of research shows that general cognitive ability is heritable and stable in young adults, there is recent evidence that fluid intelligence can be heightened with cognitive training. Many researchers, however, have questioned the methodology of the cognitive-training studies reporting improvements in fluid intelligence: specifically, the role of placebo effects. We designed a procedure to intentionally induce a placebo effect via overt recruitment in an effort to evaluate the role of placebo effects in fluid intelligence gains from cognitive training. Individuals who self-selected into the placebo group by responding to a suggestive flyer showed improvements after a single, 1-h session of cognitive training that equates to a 5- to 10-point increase on a standard IQ test. Controls responding to a nonsuggestive flyer showed no improvement. These findings provide an alternative explanation for effects observed in the cognitive-training literature and the brain-training industry, revealing the need to account for confounds in future research.

Individual differences in working-memory capacity and task resumption following interruptions.

Previous research has shown that there is a time cost (i.e., a resumption lag) associated with resuming a task following an interruption and that the longer the duration of the interruption, the greater the time cost (i.e., resumption lag increases as interruption duration increases). The memory-for-goals model (Altmann & Trafton, 2002) suggests that this greater time cost is a result of increased interference caused by longer duration interruptions. Therefore, the goal for this research was to determine whether individuals who can better manage interference, i.e., individuals with higher working-memory capacity (WMC), can resume tasks more quickly following interruptions than those who cannot manage interference as well (i.e., individuals with lower WMC). A procedural interruption task with 3 different interruption durations and a measure of WMC were completed by 229 students. In line with previous research, we found a strong positive relationship between interruption duration and resumption lag. We found a strong negative effect of WMC on resumption lag (i.e., increases in WMC reduced resumption lags). Notably, WMC moderated the effect of interruption duration on resumption lag (i.e., increases in WMC attenuated the positive relationship between interruption duration and resumption lag). Specifically, individuals with high WMC experienced small increases in resumption lag as interruption duration increased, whereas individuals with low WMC experienced substantial increases in resumption lag as interruption duration increased. Our data suggest that individuals with higher WMC are less susceptible to interference caused by interruptions. (PsycINFO Database Record

Reducing the Disruptive Effects of Interruptions With Noninvasive Brain Stimulation.

OBJECTIVE: The authors determine whether transcranial direct current stimulation (tDCS) can reduce resumption time when an ongoing task is interrupted. BACKGROUND: Interruptions are common and disruptive. Working memory capacity has been shown to predict resumption lag (i.e., time to successfully resume a task after interruption). Given that tDCS applied to brain areas associated with working memory can enhance performance, tDCS has the potential to improve resumption lag when a task is interrupted. METHOD: Participants were randomly assigned to one of four groups that received anodal (active) stimulation of 2 mA tDCS to one of two target brain regions, left and right dorsolateral prefrontal cortex (DLPFC), or to one of two control areas, active stimulation of the left primary motor cortex or sham stimulation of the right DLPFC, while completing a financial management task that was intermittently interrupted with math problem solving. RESULTS: Anodal stimulation to the right and left DLPFC significantly reduced resumption lags compared to the control conditions (sham and left motor cortex stimulation). Additionally, there was no speed-accuracy tradeoff (i.e., the improvement in resumption time was not accompanied by an increased error rate). CONCLUSION: Noninvasive brain stimulation can significantly decrease resumption lag (improve performance) after a task is interrupted. APPLICATION: Noninvasive brain stimulation offers an easy-to-apply tool that can significantly improve interrupted task performance.

Interruptions disrupt reading comprehension.

Previous research suggests that being interrupted while reading a text does not disrupt the later recognition or recall of information from that text. This research is used as support for Ericsson and Kintsch's (1995) long-term working memory (LT-WM) theory, which posits that disruptions while reading (e.g., interruptions) do not impair subsequent text comprehension. However, to fully comprehend a text, individuals may need to do more than recognize or recall information that has been presented in the text at a later time. Reading comprehension often requires individuals to connect and synthesize information across a text (e.g., successfully identifying complex topics such as themes and tones) and not just make a familiarity-based decision (i.e., recognition). The goal for this study was to determine whether interruptions while reading disrupt reading comprehension when the questions assessing comprehension require participants to connect and synthesize information across the passage. In Experiment 1, interruptions disrupted reading comprehension. In Experiment 2, interruptions disrupted reading comprehension but not recognition of information from the text. In Experiment 3, the addition of a 15-s time-out prior to the interruption successfully removed these negative effects. These data suggest that the time it takes to process the information needed to successfully comprehend text when reading is greater than that required for recognition. Any interference (e.g., an interruption) that occurs during the comprehension process may disrupt reading comprehension. This evidence supports the need for transient activation of information in working memory for successful text comprehension and does not support LT-WM theory.

Do interruptions affect quality of work?

OBJECTIVE: The aim of this study was to determine if interruptions affect the quality of work. BACKGROUND: Interruptions are commonplace at home and in the office. Previous research in this area has traditionally involved time and errors as the primary measures of disruption. Little is known about the effect interruptions have on quality of work. METHOD: Fifty-four students outlined and wrote three essays using a within-subjects design. During Condition 1, interruptions occurred while participants were outlining. During Condition 2, interruptions occurred while they were writing. No interruptions occurred in Condition 3. RESULTS: Quality of work was significantly reduced in both interruption conditions when compared to the non-interruption condition. The number of words produced was significantly reduced when participants were interrupted while writing the essay but not when outlining the essay. CONCLUSION: This research represents a crucial first step in understanding the effect interruptions have on quality of work. Our research suggests that interruptions negatively impact quality of work during a complex, creative writing task. Since interruptions are such a prevalent part of daily life, more research needs to be conducted to determine what other tasks are negatively impacted. Moreover, the underlying mechanism(s) causing these decrements needs to be identified. Finally, strategies and systems need to be designed and put in place to help counteract the decline in quality of work caused by interruptions.

Activation and inhibition of posterior parietal cortex have bi-directional effects on spatial errors following interruptions.

Interruptions to ongoing mental activities are omnipresent in our modern digital world, but the brain networks involved in interrupted performance are not known, nor have the activation of those networks been modulated. Errors following interruptions reflect failures in spatial memory, whose maintenance is supported by a brain network including the right posterior parietal cortex (PPC). The present study therefore used bi-directional transcranial Direct Current Stimulation (tDCS) of right PPC to examine the neuromodulation of spatial errors following interruptions, as well as performance on another PPC-dependent task, mental rotation. Anodal stimulation significantly reduced the number of interruption-based errors and increased mental rotation accuracy whereas cathodal stimulation significantly increased errors and reduced mental rotation accuracy. The results provide evidence for a causal role of the PPC in the maintenance of spatial representations during interrupted task performance.