Promoting inclusive recruiting and selection into military training schools: Admission waivers versus retesting.

There is high-level interest in diversifying workforces, which has led organizations-including the U.S. Armed Forces-to reevaluate recruiting and selection practices. The U.S. Coast Guard (USCG) has encountered particular difficulties in diversifying its workforce, and it relies mainly on the Armed Services Vocational Aptitude Battery (ASVAB) for assigning active-duty recruits to one of 19 specialized training schools. When recruits' scores fall below ASVAB entrance standards, the USCG sometimes offers admission waivers. Alternatively, recruits can retest until their ASVAB scores meet the entrance standard. Retesting has shown mixed results in the personnel selection literature, so our main interest is to determine whether retesting or waivers best support USCG recruits' training school outcomes, especially for recruits identifying as an underrepresented minority (URM). We use data from 16,624 USCG recruits entering between 2013 and 2021 and fit augmented inverse propensity weighted models to assess differences in training outcomes by pathway to admission while accounting for self-selection into pathways. Our analyses found (a) no difference in training outcomes between recruits who qualified from their initial scores and recruits who retested, (b) recruits who received waivers were less likely to complete training school on time and spent more time in remedial training when they failed training school compared to those who retested, and (c) improvement in training outcomes for retesting over waivers was larger for recruits identifying as an URM. Results suggest that retesting may be an effective strategy for workforce diversification and for improving outcomes among recruits identifying as an URM. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Modeling individual differences in the timing of change onset and offset.

Individual differences in the timing of developmental processes are often of interest in longitudinal studies, yet common statistical approaches to modeling change cannot directly estimate the timing of when change occurs. The time-to-criterion framework was recently developed to incorporate the timing of a prespecified criterion value; however, this framework has difficulty accommodating contexts where the criterion value differs across people or when the criterion value is not known a priori, such as when the interest is in individual differences in when change starts or stops. This article combines aspects of reparameterized quadratic models and multiphase models to provide information on the timing of change. We first consider the more common situation of modeling decelerating change to an offset point, defined as the point in time at which change ceases. For increasing trajectories, the offset occurs when the criterion attains its maximum ("inverted J-shaped" trajectories). For decreasing trajectories, offset instead occurs at the minimum. Our model allows for individual differences in both the timing of offset and ultimate level of the outcome. The same model, reparameterized slightly, captures accelerating change from a point of onset ("J-shaped" trajectories). We then extend the framework to accommodate "S-shaped" curves where both the onset and offset of change are within the observation window. We provide demonstrations that span neuroscience, educational psychology, developmental psychology, and cognitive science, illustrating the applicability of the modeling framework to a variety of research questions about individual differences in the timing of change. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Quantity and Quality in Scientific Productivity: The Tilted Funnel Goes Bayesian.

The equal odds baseline model of creative scientific productivity proposes that the number of high-quality works depends linearly on the number of total works. In addition, the equal odds baseline implies that the percentage of high-quality works and total number of works are uncorrelated. The tilted funnel hypothesis proposes that the linear regression implied by the equal odds baseline is heteroscedastic with residual variance in the quality of work increasing as a function of quantity. The aim of the current research is to leverage Bayesian statistical modeling of the equal odds baseline. Previous work has examined the tilted funnel by means of frequentist quantile regression, but Bayesian quantile regression based on the asymmetric Laplace model allows for only one conditional quantile at a time. Hence, we propose additional Bayesian methods, including Poisson modeling to study conditional variance as a function of quantity. We use a classical small sample of eminent neurosurgeons, as well as the brms Bayesian R package, to accomplish this work. In addition, we provide open code and data to allow interested researchers to extend our work and utilize the proposed modeling alternatives.

Longitudinal measurement invariance of the Behavioral Health Measure in a clinical sample.

The practice of routine outcome monitoring (ROM) has grown in popularity and become a fixture in feedback-supported clinical practice and research. However, if the interpretation of an ROM measure changes over time, treatment outcome scores may be inaccurate and produce erroneous or misguided interpretations of client progress and therapist efficacy. The current study examined whether factorial invariance held when using the Behavioral Health Measure (BHM-20) longitudinally in a clinical sample (n = 12,467). Using multidimensional item response theory-based models for the investigation of the BHM-20 factor structure, at a single time point and then longitudinally. Based on the original factor structure of the BHM-20 a unidimensional model, a three-factor orthogonal model, and a three-factor correlated model were fit to the data, indicating poor model fit with the proposed three-factor or unidimensional models. Next, using exploratory factor analysis and subsequent multidimensional item response theory procedures, a new 4-factor (General Distress, Life Functioning, Anxiety, and Alcohol/Drug Use) model was proposed with improved model-fit statistics. Finally, when testing the longitudinal invariance of the BHM-17 over 10 sessions of treatment, it was found to be fully consistent. The current study proposes the use of a 17-item, 4-factor model for a new understanding of the BHM-17. Implications for use in ROM and limitations are discussed. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Measuring flexibility: A text-mining approach.

In creativity research, ideational flexibility, the ability to generate ideas by shifting between concepts, has long been the focus of investigation. However, psychometric work to develop measurement procedures for flexibility has generally lagged behind other creativity-relevant constructs such as fluency and originality. Here, we build from extant research to theoretically posit, and then empirically validate, a text-mining based method for measuring flexibility in verbal divergent thinking (DT) responses. The empirical validation of this method is accomplished in two studies. In the first study, we use the verbal form of the Torrance Test of Creative Thinking (TTCT) to demonstrate that our novel flexibility scoring method strongly and positively correlates with traditionally used TTCT flexibility scores. In the second study, we conduct a confirmatory factor analysis using the Alternate Uses Task to show reliability and construct validity of our text-mining based flexibility scoring. In addition, we also examine the relationship between personality facets and flexibility of ideas to provide criterion validity of our scoring methodology. Given the psychometric evidence presented here and the practicality of automated scores, we recommend adopting this new method which provides a less labor-intensive and less costly objective measurement of flexibility.

Understanding ideational fluency as a survival process.

BACKGROUND: When students generate ideas, important inter-individual variance exists both in the quantity and the quality of ideas they are able to produce (e.g., perfectionists who have few highly creative ideas or mass producers who produce a lot of uncreative ideas). In educational psychology research on creativity, the relation between the quantity and quality of ideas has not been well understood, limiting progress in this area. AIMS: We conceptualized Ideational Fluency as a phenomenon that requires participants to 'survive' to produce more ideas, and where dropping out of the ideational process was analogous to 'dying'. Using this novel paradigm, we aimed to test the relations among Fluency (as a dependent variable); and creative Expertise, Originality and self-reported Personality attributes (as independent variables). SAMPLE AND METHOD: Participants were drawn from three groups: those with demonstrated expertise in stage or screen acting (n = 104); undergraduates being trained in the same domain (n = 100), and adults with no acting training or experience (n = 92). Participants responded to the Alternate Uses Task; Non-parametric and semi-parametric survival models were fit to their Ideational Fluency; and average and maximum Originality scores, as well as self-reported Personality attributes, were used as covariates. RESULTS: Across all participants, the Ideational Fluency survival function showed an S-shape, but the Expertise grouping interacted with that pattern. The survival rate of professional actors decreased more rapidly during the first few ideas, but after the 5th idea, professional actors displayed a clear advantage in survival rate. Participants who were less original on average but who showed a high maximum Originality, as well as those participants who reported more Assertiveness and less Industriousness, also survived further into the Ideational process. CONCLUSIONS: Contrary to our hypothesis, professional actors' advantage in Fluency did not manifest in the survival model until after the 5th idea generated. A quantity-quality trade-off was observed with average Originality being associated with shorter survival, but that trade-off was not observed with maximum Originality, which was associated with longer survival.

The psychology of professional and student actors: Creativity, personality, and motivation.

As a profession, acting is marked by a high-level of economic and social riskiness concomitantly with the possibility for artistic satisfaction and/or public admiration. Current understanding of the psychological attributes that distinguish professional actors is incomplete. Here, we compare samples of professional actors (n = 104), undergraduate student actors (n = 100), and non-acting adults (n = 92) on 26 psychological dimensions and use machine-learning methods to classify participants based on these attributes. Nearly all of the attributes measured here displayed significant univariate mean differences across the three groups, with the strongest effect sizes being on Creative Activities, Openness, and Extraversion. A cross-validated Least Absolute Shrinkage and Selection Operator (LASSO) classification model was capable of identifying actors (either professional or student) from non-actors with a 92% accuracy and was able to sort professional from student actors with a 96% accuracy when age was included in the model, and a 68% accuracy with only psychological attributes included. In these LASSO models, actors in general were distinguished by high levels of Openness, Assertiveness, and Elaboration, but professional actors were specifically marked by high levels of Originality, Volatility, and Literary Activities.

Scrutinizing the basis of originality in divergent thinking tests: On the measurement precision of response propensity estimates.

BACKGROUND: The originality of divergent thinking (DT) production is one of the most critical indicators of creative potential. It is commonly scored using the statistical infrequency of responses relative to all responses provided in a given sample. AIMS: Response frequency estimates vary in terms of measurement precision. This issue has been widely overlooked and is addressed in the current study. SAMPLE AND METHOD: Secondary data analysis of 202 participants was performed. A total of 900 uniquely identified responses were generated on three DT tasks and subjected to a 1-parameter logistic model with a response as the unit of measurement which allowed for the calculation of response-level conditional reliability (and marginal reliability as an overall summary of measurement precision). RESULTS: Marginal reliability of response propensity estimates ranged from .62 to .67 across the DT tasks. Unique responses in the sample (the basis for the classic uniqueness scoring) displayed the lowest conditional reliability (across tasks: ≈ .50). Reliability increased nonlinearly as a function of both the frequency of occurrence predicted by the model (conditional reliability) and sample size (conditional and marginal reliability). CONCLUSIONS: This study indicates that the common practice of frequency-based originality scoring with typical sample sizes (e.g., N = 100 to N = 200) yields unacceptable levels of measurement precision (i.e., in particular for highly original responses). We further offer recommendations to mitigate the lack of measurement precision of frequency-based originality scores for DT research.

Estimating New Quantities from Longitudinal Test Scores to Improve Forecasts of Future Performance.

Psychometric models for longitudinal test scores typically estimate quantities associated with single-administration tests, like ability at each time-point. However, models for longitudinal tests have not considered opportunities to estimate new quantities that are unavailable from single-administration tests. Specifically, we discuss dynamic measurement models - which combine aspects of longitudinal IRT, nonlinear growth models, and dynamic assessment - to directly estimate capacity, defined as the expected future score once the construct has fully developed. After discussing the history and connecting these areas into a single framework, we apply the model to verbal test scores from the Intergenerational Studies, which follow 494 people from 3 to 72 years old. The goal is to predict adult verbal scores (Age ≥ 34) from adolescent scores (Age ≤ 20). We held-out the adult data for prediction and compared predictions from traditional longitudinal IRT ability scores and proposed dynamic measurement capacity scores from models fit to the adolescent data. Results showed that the R2 from capacity scores were 2.5 times larger than the R2 from longitudinal IRT ability scores (43% vs. 16%), providing some evidence that exploring new quantities available from longitudinal testing could be worthwhile when an interest in testing is forecasting future performance.

Dynamic Measurement in Health Professions Education: Rationale, Application, and Possibilities.

Dynamic measurement modeling (DMM) is a psychometric paradigm that uses longitudinal data to estimate individual students' growth in measured skills over the course of an educational program (i.e., growth scores). DMM represents a more formal way of assessing learning progress across the health professions education continuum. In this article, the authors provide justification for this approach in health professions education and demonstrate its proof-of-concept use with three time points of United States Medical Licensing Examination Step exams to generate growth scores for 454 current and recent medical learners. The authors demonstrate that learners vary substantially on their growth scores, and those growth scores exhibit psychometric reliability. In addition, growth scores significantly and positively correlated with indicators of medical learner readiness (e.g., undergraduate grade point average and Medical College Admission Test scores). These growth scores were also capable of significantly and positively correlating with future ratings of clinical competencies during internship as assessed through a survey sent to their program directors at the end of the first postgraduate year (e.g., patient care, interpersonal skills). These preliminary findings of reliability and validity for DMM growth scores provide initial evidence for further investigation into the suitability of a dynamic measurement paradigm in health professions education.

Using Relational Reasoning Strategies to Help Improve Clinical Reasoning Practice.

Clinical reasoning-the steps up to and including establishing a diagnosis and/or therapy-is a fundamentally important mental process for physicians. Unfortunately, mounting evidence suggests that errors in clinical reasoning lead to substantial problems for medical professionals and patients alike, including suboptimal care, malpractice claims, and rising health care costs. For this reason, cognitive strategies by which clinical reasoning may be improved-and that many expert clinicians are already using-are highly relevant for all medical professionals, educators, and learners.In this Perspective, the authors introduce one group of cognitive strategies-termed relational reasoning strategies-that have been empirically shown, through limited educational and psychological research, to improve the accuracy of learners' reasoning both within and outside of the medical disciplines. The authors contend that relational reasoning strategies may help clinicians to be metacognitive about their own clinical reasoning; such strategies may also be particularly well suited for explicitly organizing clinical reasoning instruction for learners. Because the particular curricular efforts that may improve the relational reasoning of medical students are not known at this point, the authors describe the nature of previous research on relational reasoning strategies to encourage the future design, implementation, and evaluation of instructional interventions for relational reasoning within the medical education literature. The authors also call for continued research on using relational reasoning strategies and their role in clinical practice and medical education, with the long-term goal of improving diagnostic accuracy.

Nonlinear Growth Models as Measurement Models: A Second-Order Growth Curve Model for Measuring Potential.

Recent methodological work has highlighted the promise of nonlinear growth models for addressing substantive questions in the behavioral sciences. In this article, we outline a second-order nonlinear growth model in order to measure a critical notion in development and education: potential. Here, potential is conceptualized as having three components-ability, capacity, and availability-where ability is the amount of skill a student is estimated to have at a given timepoint, capacity is the maximum amount of ability a student is predicted to be able to develop asymptotically, and availability is the difference between capacity and ability at any particular timepoint. We argue that single timepoint measures are typically insufficient for discerning information about potential, and we therefore describe a general framework that incorporates a growth model into the measurement model to capture these three components. Then, we provide an illustrative example using the public-use Early Childhood Longitudinal Study-Kindergarten data set using a Michaelis-Menten growth function (reparameterized from its common application in biochemistry) to demonstrate our proposed model as applied to measuring potential within an educational context. The advantage of this approach compared to currently utilized methods is discussed as are future directions and limitations.

The Creative Stereotype Effect.

Because of its fundamental relevance to scientific innovation, artistic expression, and human ingenuity, creativity has long been the subject of systematic psychological investigation. Concomitantly, the far-reaching effects of stereotypes on various cognitive and social processes have been widely researched. Bridging these two literatures, we show in a series of two studies that stereotypes related to creativity can both enhance and diminish individuals' performance on a divergent thinking task. Specifically, Study 1 demonstrated that participants asked to take on a stereotypically uninhibited perspective performed significantly better on a divergent thinking task than those participants who took on a stereotypically inhibited perspective, and a control group. Relatedly, Study 2 showed that the same effect is found within-subjects, with divergent thinking significantly improving when participants invoke an uninhibited stereotype. Moreover, we demonstrate the efficacy of Latent Semantic Analysis as an objective measure of the originality of ideas, and discuss implications of our findings for the nature of creativity. Namely, that creativity may not be best described as a stable individual trait, but as a malleable product of context and perspective.

Calibration of the Test of Relational Reasoning.

Relational reasoning, or the ability to discern meaningful patterns within a stream of information, is a critical cognitive ability associated with academic and professional success. Importantly, relational reasoning has been described as taking multiple forms, depending on the type of higher order relations being drawn between and among concepts. However, the reliable and valid measurement of such a multidimensional construct of relational reasoning has been elusive. The Test of Relational Reasoning (TORR) was designed to tap 4 forms of relational reasoning (i.e., analogy, anomaly, antinomy, and antithesis). In this investigation, the TORR was calibrated and scored using multidimensional item response theory in a large, representative undergraduate sample. The bifactor model was identified as the best-fitting model, and used to estimate item parameters and construct reliability. To improve the usefulness of the TORR to educators, scaled scores were also calculated and presented. (PsycINFO Database Record