Return-to-Manual Performance can be Predicted Before Automation Fails.

OBJECTIVE: This study aimed to examine operator state variables (workload, fatigue, and trust in automation) that may predict return-to-manual (RTM) performance when automation fails in simulated air traffic control. BACKGROUND: Prior research has largely focused on triggering adaptive automation based on reactive indicators of performance degradation or operator strain. A more direct and effective approach may be to proactively engage/disengage automation based on predicted operator RTM performance (conflict detection accuracy and response time), which requires analyses of within-person effects. METHOD: Participants accepted and handed-off aircraft from their sector and were assisted by imperfect conflict detection/resolution automation. To avoid aircraft conflicts, participants were required to intervene when automation failed to detect a conflict. Participants periodically rated their workload, fatigue and trust in automation. RESULTS: For participants with the same or higher average trust than the sample average, an increase in their trust (relative to their own average) slowed their subsequent RTM response time. For participants with lower average fatigue than the sample average, an increase in their fatigue (relative to own average) improved their subsequent RTM response time. There was no effect of workload on RTM performance. CONCLUSIONS: RTM performance degraded as trust in automation increased relative to participants' own average, but only for individuals with average or high levels of trust. APPLICATIONS: Study outcomes indicate a potential for future adaptive automation systems to detect vulnerable operator states in order to predict subsequent RTM performance decrements.

Compromises in career-related decisions: examining the role of compromise severity.

This study tested L. S. Gottfredson's (1996) revised compromise theory by examining whether the relative importance of job sex type, job prestige, and person-job interest congruence for predicting job choice changed as the level of compromise required changed. The fully within-persons design had participants engage in a simulated occupational choice task where job sex type and job prestige were manipulated to be experimentally independent. Participants 1st categorized jobs as unacceptable, acceptable, or preferred. Then, within each category, they made further pairwise choices among jobs in that category. In Study 1, participants were 168 college seniors (124 women, 44 men) from a large Midwestern university. In Study 2, participants were 262 (146 women, 116 men) individuals residing in the United States and recruited via Amazon's Mechanical Turk platform. Across both studies, job sex type predicted choice when large compromises were required. Across both studies, job prestige did not predict choice when moderate compromises were required. In Study 2 but not Study 1, person-job interest congruence predicted choice when minimal compromises were required.

Operator selection for human-automation teaming: The role of manual task skill in predicting automation failure intervention.

Humans working in modern work systems are increasingly required to supervise task automation. We examined whether manual aircraft conflict detection skill predicted participants' ability to respond to conflict detection automation failures in simulated air traffic control. In a conflict discrimination task (to assess manual skill), participants determined whether pairs of aircraft were in conflict or not by judging their relative-arrival time at common intersection points. Then in a simulated air traffic control task, participants supervised automation which either partially or fully detected and resolved conflicts on their behalf. Automation supervision required participants to detect when automation may have failed and effectively intervene. When automation failed, participants who had better manual conflict detection skill were faster and more accurate to intervene. However, a substantial proportion of variance in failure intervention was not explained by manual conflict detection skill, potentially reflecting that future research should consider other cognitive skills underlying automation supervision.

Adverse impact reduction and job performance optimization via pareto-optimal weighting: A shrinkage formula and regularization technique using machine learning.

In personnel selection practice, one useful technique for reducing adverse impact and enhancing diversity is the Pareto-optimal weighting approach of De Corte et al. (2007). This approach produces a series of hiring solutions that characterize a diversity-job performance trade-off and can lead to more optimal selection outcomes (sometimes doubling the number of job offers for minority applicants without changing the job performance outcomes of personnel selection). Despite these advantages, recent research has identified a potential problem with the Pareto-weighting technique-Pareto solutions suffer from shrinkage upon cross-validation. To address the problem of shrinkage in the Pareto trade-off curve (i.e., diversity shrinkage and validity shrinkage), we offer two contributions. First, a shrinkage approximation formula is introduced (similar to a formula for adjusted R², but applicable to the entire Pareto trade-off curve). Second, we derive a novel technique for the regularization of Pareto-optimal predictor weights (borrowed from the field of machine learning), which is designed to produce predictor weights that are less vulnerable to shrinkage (similar to ridge regression and adapted from the elastic net technique). Both approaches-the proposed Pareto shrinkage formula and the proposed regularization technique-are then evaluated using Monte Carlo simulation. Recommendations are provided for approximating potential diversity-performance trade-off curves in personnel selection, while accounting for shrinkage. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Are Individual Differences in Information-Processing Styles Related to Transformational Leadership? A Test of the Cognitive Experiential Leadership Model.

The recently proposed Cognitive Experiential Leadership Model (CELM) states that leaders' preference for rational thinking and behavioral coping will be related to their level of transformational leadership. The CELM was based on research that principally used cross-sectional self-report methods. Study 1 compared both self-ratings and follower-ratings of leadership styles with leaders' self-rated thinking styles in 160 leader-follower dyads. Study 2 compared both self-ratings and coworker-ratings of leadership styles with leaders' self-rated thinking styles for 74 leaders rated by 607 coworkers. In both Studies, leaders' rational thinking, imaginative thinking, and behavioral coping correlated positively with their self-rated transformational leadership. However, only behavioral coping, but not rational thinking, was correlated with follower-rated (FR) transformational leadership in Study 1, and thinking styles were unrelated to other-rated transformational leadership in Study 2. These results partly support and partly challenge the CELM. Practically, this study suggests that leadership may be improved by leaders developing their capacity for behavioral coping.

Signal detection with criterion noise: applications to recognition memory.

A tacit but fundamental assumption of the theory of signal detection is that criterion placement is a noise-free process. This article challenges that assumption on theoretical and empirical grounds and presents the noisy decision theory of signal detection (ND-TSD). Generalized equations for the isosensitivity function and for measures of discrimination incorporating criterion variability are derived, and the model's relationship with extant models of decision making in discrimination tasks is examined. An experiment evaluating recognition memory for ensembles of word stimuli revealed that criterion noise is not trivial in magnitude and contributes substantially to variance in the slope of the isosensitivity function. The authors discuss how ND-TSD can help explain a number of current and historical puzzles in recognition memory, including the inconsistent relationship between manipulations of learning and the isosensitivity function's slope, the lack of invariance of the slope with manipulations of bias or payoffs, the effects of aging on the decision-making process in recognition, and the nature of responding in remember-know decision tasks. ND-TSD poses novel, theoretically meaningful constraints on theories of recognition and decision making more generally, and provides a mechanism for rapprochement between theories of decision making that employ deterministic response rules and those that postulate probabilistic response rules.

Aligning Predictor-Criterion Bandwidths: Specific Abilities as Predictors of Specific Performance.

The purpose of the current study is to compare the extent to which general and specific abilities predict academic performances that are also varied in breadth (i.e., general performance and specific performance). The general and specific constructs were assumed to vary only in breadth, not order, and two data analytic approaches (i.e., structural equation modeling [SEM] and relative weights analysis) consistent with this theoretical assumption were compared. Conclusions regarding the relative importance of general and specific abilities differed based on data analytic approaches. The SEM approach identified general ability as the strongest and only significant predictor of general academic performance, with neither general nor specific abilities predicting any of the specific subject grade residuals. The relative weights analysis identified verbal reasoning as contributing more than general ability, or other specific abilities, to the explained variance in general academic performance. Verbal reasoning also contributed to most of the explained variance in each of the specific subject grades. These results do not provide support for the utility of predictor-criterion alignment, but they do provide evidence that both general and specific abilities can serve as useful predictors of performance.

Diversity shrinkage: Cross-validating pareto-optimal weights to enhance diversity via hiring practices.

To reduce adverse impact potential and improve diversity outcomes from personnel selection, one promising technique is De Corte, Lievens, and Sackett's (2007) Pareto-optimal weighting strategy. De Corte et al.'s strategy has been demonstrated on (a) a composite of cognitive and noncognitive (e.g., personality) tests (De Corte, Lievens, & Sackett, 2008) and (b) a composite of specific cognitive ability subtests (Wee, Newman, & Joseph, 2014). Both studies illustrated how Pareto-weighting (in contrast to unit weighting) could lead to substantial improvement in diversity outcomes (i.e., diversity improvement), sometimes more than doubling the number of job offers for minority applicants. The current work addresses a key limitation of the technique-the possibility of shrinkage, especially diversity shrinkage, in the Pareto-optimal solutions. Using Monte Carlo simulations, sample size and predictor combinations were varied and cross-validated Pareto-optimal solutions were obtained. Although diversity shrinkage was sizable for a composite of cognitive and noncognitive predictors when sample size was at or below 500, diversity shrinkage was typically negligible for a composite of specific cognitive subtest predictors when sample size was at least 100. Diversity shrinkage was larger when the Pareto-optimal solution suggested substantial diversity improvement. When sample size was at least 100, cross-validated Pareto-optimal weights typically outperformed unit weights-suggesting that diversity improvement is often possible, despite diversity shrinkage. Implications for Pareto-optimal weighting, adverse impact, sample size of validation studies, and optimizing the diversity-job performance tradeoff are discussed. (PsycINFO Database Record

Capturing attention when attention blinks.

Four experiments addressed the question of whether attention may be captured when the visual system is in the midst of an attentional blink (AB). Participants identified 2 target letters embedded among distractor letters in a rapid serial visual presentation sequence. In some trials, a square frame was inserted between the targets; as the only geometric object in the sequence, it constituted a singleton. Capture effects obtained when the AB was most severe and when it was over were compared. There were 3 main results. First, capture occurred even when the AB was crippling, suggesting that a singleton exogenously engaged attention even when processing of a previous target was continuing apace. Second, when the singleton contained the key target feature, capture effects were clearly manifest. Third, even when the singleton did not possess the key target feature, it still succeeded in capturing attention, although the effects were both feeble and fleeting.

More than g: selection quality and adverse impact implications of considering second-stratum cognitive abilities.

When using cognitive tests, personnel selection practitioners typically face a trade-off between the expected job performance and diversity of new hires. We review the increasingly mainstream evidence that cognitive ability is a multidimensional and hierarchically ordered set of concepts, and examine the implications for both composite test validity and subgroup differences. Ultimately, we recommend a strategy for differentially weighting cognitive subtests (i.e., second-stratum abilities) in a way that minimizes overall subgroup differences without compromising composite test validity. Using data from 2 large validation studies that included a total of 15 job families, we demonstrate that this strategy could lead to substantial improvement in diversity hiring (e.g., doubling the number of job offers extended to minority applicants) and to at least 8% improvement in job offers made to minority applicants, without decrements in expected selection quality compared to a unit-weighted cognitive test composite. Finally, we conduct a sensitivity analysis to examine whether the technique continues to perform well when applied to applicant pools of smaller size. We discuss prerequisites for the application of this strategy, potential limitations, and extensions.

Outcomes assessment of role-play scenarios for teaching responsible conduct of research.

We describe the summative assessment of role-play scenarios that we previously developed to teach central topics in the responsible conduct of research (RCR) to graduate students in science and engineering. Interviews with role-play participants, with participants in a case discussion training session, and with untrained students suggested that role-playing might promote a deeper appreciation of RCR by shifting the focus away from wanting to simply "know the rules." We also present the results of a think-aloud case analysis study and describe the development of a behaviorally-anchored rating scale (BARS) to assess participants' case analysis performance.