Designing pareto-optimal selection systems for multiple minority subgroups and multiple criteria.

Currently used Pareto-optimal (PO) approaches for balancing diversity and validity goals in selection can deal only with one minority group and one criterion. These are key limitations because the workplace and society at large are getting increasingly diverse and because selection system designers often have interest in multiple criteria. Therefore, the article extends existing methods for designing PO selection systems to situations involving multiple criteria and multiple minority groups (i.e., multiobjective PO selection systems). We first present a hybrid multiobjective PO approach for computing selection systems that are PO with respect to (a) a set of quality objectives (i.e., criteria) and (b) a set of diversity objectives where each diversity objective relates to a different minority group. Next, we propose three two-dimensional subspace procedures that aid selection designers in choosing between the PO systems in case of a high number of quality and diversity objectives. We illustrate our novel multiobjective PO approaches via several example applications, thereby demonstrating that they are the first to reveal the complete gamut of eligible PO selection designs and to faithfully capture the Pareto trade-off front in case of more than two objectives. In addition, a small-scale cross-validation study confirms that the resulting PO selection designs retain an advantage over alternative designs when applied in new validation samples. Finally, the article provides a link to an executable code to perform the new multiobjective PO approaches. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

A comprehensive examination of the cross-validity of pareto-optimal versus fixed-weight selection systems in the biobjective selection context.

The article presents evidence for the cross-validity potential of fixed-weight (FW) versus Pareto-Optimal (PO) selection systems in biobjective selection situations where both the goals of diversity and quality are valued and the importance of the goals is undecided a priori. The article extends previous research by also studying the cross-validity potential of selection systems in the practically most important sample-to-sample cross-validity scenario. We address three research questions: (a) Do different PO systems show comparable levels of relative (i.e., proportional) achievement upon cross-validation? (b) Do PO systems achieve higher levels of relative achievement upon cross-validation than FW selection systems?, and (c) How does the achievement of PO and FW systems, in terms of adverse impact ratios and average performance of the selected applicants, evolve under cross-validation? As a key result, in case of sufficiently large applicant pools (typically 100 applicants or more), PO systems had on average a higher cross-validity potential than the corresponding FW systems. Yet, even for applicant pools as large as 500, FW systems may match the merits of PO systems and we present a straightforward procedure to decide which FW systems may offer a comparable cross-validation potential than the PO systems. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Weighting admission scores to balance predictiveness-diversity: The Pareto-optimization approach.

CONTEXT: Although many medical schools seek to improve diversity, they grapple with the challenge of how to weight the scores of different admission methods to achieve a balance between obtaining high predictiveness and ensuring diversity in the selected student pool. Yet, in large-scale employment settings, substantial progress has been made on this front: Pareto-optimization has been introduced as an elegant statistical tool to assist decision makers in determining the weights assigned to selection methods in advance (before the selection has taken place) so that a selection system is designed to achieve an optimal balance as reflected by the trade-off that one outcome (e.g., predictiveness) cannot be improved without harm to the other outcome (e.g., diversity). AIMS: This paper reviews the theory and research evidence about Pareto-optimization and explains how Pareto-optimization permits medical schools to better balance predictiveness and diversity in medical admission systems. METHODS: After reviewing common weighting schemes (unit, regression-based and ad hoc weighting) and their drawbacks, we introduce the theory and logic of Pareto-optimization for better balancing predictiveness and diversity. To this end, we also offer an illustrative example. Next, we review the mathematical basis and available research evidence regarding Pareto-optimization. Finally, we discuss potential criticisms (i.e., complexity and legal concerns). CONCLUSIONS: Compared to traditional unit weighting, regression-based weighting and ad hoc weighting, Pareto-optimization leads to substantial increases in diversity intake (up to three times more), while keeping the predictiveness of the selection methods at the same level. Moreover, the Pareto-optimization is robust to sampling variability and variability of the input selection parameters.

Designing Pareto-optimal selection systems: formalizing the decisions required for selection system development.

The article presents an analytic method for designing Pareto-optimal selection systems where the applicants belong to a mixture of candidate populations. The method is useful in both applied and research settings. In an applied context, the present method is the first to assist the selection practitioner when deciding on 6 major selection design issues: (1) the predictor subset, (2) the selection rule, (3) the selection staging, (4) the predictor sequencing, (5) the predictor weighting, and (6) the stage retention decision issue. From a research perspective, the method offers a unique opportunity for studying the impact and relative importance of different strategies for reducing adverse impact.

A closer look at the frame-of-reference effect in personality scale scores and validity.

This article contributes to the understanding of why the use of a frame-of-reference leads to increased criterion-related validity of personality inventories. Two competing explanations are described and tested. A between-subjects (N = 337) and a within-subject (N = 105) study are conducted to test the hypothesized effects of use of a frame of reference on reliability and validity. Regarding the effects on reliability, use of a frame of reference reduces within-person inconsistency (instead of between-person variability) in responding to generic items. Use of a frame of reference further leads to higher validity as a result of the reduction of between-person variability and within-person inconsistency. Yet, reducing these inconsistencies is not enough. It is also important to use a frame of reference that is conceptually relevant to the criterion. Besides implications for contextualized personality inventories, these results provide an explanation for the moderate validities of generic personality inventories.

Combining predictors to achieve optimal trade-offs between selection quality and adverse impact.

The authors propose a procedure to determine (a) predictor composites that result in a Pareto-optimal trade-off between the often competing goals in personnel selection of quality and adverse impact and (b) the relative importance of the quality and impact objectives that correspond to each of these trade-offs. They also investigated whether the obtained Pareto-optimal composites continue to perform well under variability of the selection parameters that characterize the intended selection decision. The results of this investigation indicate that this is indeed the case. The authors suggest that the procedure be used as one of a number of potential strategies for addressing the quality-adverse impact problem in settings where estimates of the selection parameters (e.g., validity estimates, predictor intercorrelations, subgroup mean differences on the predictors and criteria) are available from either a local validation study or meta-analytic research.

Predicting adverse impact and mean criterion performance in multistage selection.

The authors present an analytical method to assess the average criterion performance of the selected candidates as well as the adverse impact and the cost of general multistage selection decisions. The method extends previous work on the analytical estimation of multistage selection outcomes to the case in which the applicant pool is a mixture of applicant populations that differ in their average performance on the selection predictors. Next, the method was used to conduct 3 studies of important issues practitioners and researchers have with multistage selection processes. Finally, the authors indicate how the method can be integrated into a broader analytical framework to design multistage selection decisions that achieve intended levels of selection cost, workforce quality, and workforce diversity.

The mean and variance of the selection differential after top-down selection from a mixture of normal samples.

A procedure is presented for determining the mean and variance of the selection differential for top-down selections in which the candidates come from populations that have a different average score on the selection measure. Although the procedure is based on the same stochastic model and requires identical data to the currently available method for estimating the mean selection differential, it has the advantage that the resulting expressions are valid for finite-sample selection decisions and that the variance of the selection differential can also be assessed. The difference between the two procedures is illustrated by means of an example application, and it is shown how the present results are particularly helpful in determining the expected utility of personnel selection decisions.

Acceptance of the unpleasant reality of chronic pain: effects upon attention to pain and engagement with daily activities.

This paper investigates whether acceptance was related to less attention to pain, and to more engagement with daily activities. The results of two studies are reported. In a first cross-sectional study, 501 chronic pain patients completed self-report instruments on pain severity, attention to pain and acceptance. In a second diary study, 62 patients with chronic pain reported pain intensity, attention to pain and characteristics of goal-directed behaviour 8 times a day using an experience sampling method. Acceptance was measured using a self-report instrument. It was found that acceptance was related to less attention to pain (study 1 and study 2), more engagement with daily activities, a higher motivation to complete activities and a better efficacy to perform daily activities (study 2). Results are discussed in terms of how a positive life despite pain may be preserved by a flexible adjustment of personal goals to current limitations and adversities.

Acceptance of pain is an independent predictor of mental well-being in patients with chronic pain: empirical evidence and reappraisal.

This paper reports upon: (1) the value of acceptance of pain in predicting well-being in patients suffering from chronic pain and (2) the construct validity of acceptance by comparing two questionnaires designed to measure acceptance (the Chronic Pain Acceptance Questionnaire, CPAQ, unpublished doctoral dissertation, University of Nevada, Reno, NV, 1992 and the Illness Cognitions Questionnaire, ICQ, J Consult Clin Psychol 69 (2001) 1026). The results of two independent cross-sectional studies are reported. Study 1 included 120 patients seeking help in tertiary care settings. In Study 2, 66 patients were recruited from a self-support group for fibromyalgia patients and from a pain clinic. Both studies revealed that acceptance of pain predicted mental well-being beyond pain severity and pain catastrophizing, but did not account for physical functioning. In both instruments, it was found that acceptance of pain was strongly related to engagement in normal life activities and the recognition that pain may not change. Acceptance in both instruments was strongly related to a cognitive control over pain. Study 2 further revealed that the correlation between the CPAQ and the ICQ is moderate, indicating that both instruments measured different aspects of acceptance. It is concluded that acceptance of chronic pain is best conceived of as the shift away from pain to non-pain aspects of life, and the shift away from a search for a cure with an acknowledgement that pain may not change.

Sampling variability of the success ratio in predictor-based selection.

In applied settings, the expected benefits of a predictor-based selection are often expressed in terms of the expected success ratio of the selection. Although Taylor and Russell proposed a formula to estimate its value, simple simulations show that their formula is often inadequate. Also, little is known at present about the sampling variability of the success ratio and, hence, about the accuracy with which the success ratio of any particular selection can be predicted. The above deficiencies are addressed for the three most popular single-stage selection scenarios: the restricted (or fixed) quota, the threshold (mastery or fixed cut-off) and the mixed quota/threshold decision. For each scenario it is shown how the sampling distribution of the success rate statistic can be derived. The problems related to the numerical evaluation of the distribution are also discussed, and it is shown by means of example applications that the sampling variability of the success ratio may be quite substantial in many real-world predictor-based selections.