Capitalizing on strengths and minimizing weaknesses of veterans in civilian employment interviews: Perceptions of interviewers and veteran interviewees.

Like all job applicants, veterans have to face the ubiquitous employment interview and pass this potential hurdle to civilian sector employment. So, because of the uniqueness of transitioning from the military to civilian employment, the present paper sought to identify perceived interviewing strengths and weaknesses of veteran interviewees from (a) the perspective of civilian sector human resource professionals (i.e. hiring personnel) with experience interviewing veterans (Study 1, five focus groups, N = 14), and (b) veterans (Study 2, N = 93). Qualitative analysis of the focus group transcripts resulted in the emergence of two theme categories: (1) veteran interviewee strengths and (2) veteran interviewee weaknesses. This information guided the development of a 10-item survey that was completed by 93 veterans (Study 2). In its totality, the results (from both Study 1 and Study 2) indicated that communication of soft skills, confidence, and professionalism were perceived to be strengths that veterans displayed during civilian employment interviews, and conversely, the ineffective translation and communication of relevant technical skills acquired in the military, use of military jargon, and nervousness were considered to be weaknesses. Recommendations to capitalize on the strengths and mitigate the weaknesses are presented.

After-Action Reviews and Long-Term Performance: An Experimental Examination in the Context of an Emergency Simulation.

OBJECTIVE: The present study examined the effectiveness of after-action reviews (AARs; also known as debriefing) in mitigating skill decay. BACKGROUND: Research on the long-term effectiveness of AARs is meager. To address this gap in the literature, we conducted an experimental study that also overcomes some research design issues that characterize the limited extant research. METHOD: Eighty-four participants were randomly assigned to an AAR or non-AAR condition and trained to operate a PC-based fire emergency simulator. During the initial acquisition phase, individuals in the AAR condition were allowed to review their performance after each practice session, whereas individuals in the non-AAR condition completed a filler task. About 12 weeks later, participants returned to the lab to complete four additional practice sessions using a similar scenario (i.e., the retention and reacquisition phase). RESULTS: The performance of participants in the AAR condition degraded more after nonuse but also recovered faster than the performance of participants in the non-AAR condition, although these effects were fairly small and not statistically significant. CONCLUSION: Consistent with the limited research on the long-term effectiveness of AARs, our findings failed to support their effectiveness as a decay-prevention intervention. Because the present study was conducted in a laboratory setting using a relatively small sample of undergraduate students, additional research is warranted. APPLICATION: Based on the results of the present study, we suggest some additional strategies that trainers might consider to support long-term skill retention when using AARs.

Small Animal Resident Selection Processes at a University Teaching Hospital: An Analysis and Recommendations for Improvement.

Concerns regarding resident performance within a small animal department prompted a review of selection practices, with the intent of improving validity and efficiency. Information was gathered from semi-structured interviews and descriptions of current processes; emphasis was placed on determining how the Veterinary Internship and Residency Matching Program application was used. Processes were found to lack standardization and rely heavily on arbitrary judgments. In addition, faculty members expressed concerns regarding their reliability and the time spent generating candidate rankings. Suggestions for improvement were based on current practices in personnel psychology and human resource management. The need for standardization within and across specialty groups was emphasized, along with a multiple-hurdle approach in which a substantial deficit or red flag in any component results in candidate disqualification. Comprehensive recommendations were made for the selection process as follows: Each application undergoes initial administrative screening for employment eligibility and academic cut-offs; eligible applications are scored by 2-3 faculty members using defined ratings on four equally weighted pre-interview criteria (i.e., veterinary education, post-graduation experiences, personal statement, and standardized letters of reference); phone calls to colleagues with knowledge of the applicant follow specific guidelines and a rating scale; veterinary-situational structured interview questions with appropriate rating scales are used to assess candidates' standing on specified competencies identified as important for success; and the interview score is weighted equally and added to the four pre-interview components to determine the final rank. It is hoped this new approach will take less time and facilitate the selection of successful residents.

Assessment and Revision of the Veterinary Internship and Residency Matching Program Standardized Letter of Reference.

The Veterinary Internship and Residency Matching Program (VIRMP) recently revised its electronic standardized letter of reference (SLOR) to improve the quality and usefulness of the data obtained from it and to enhance the relevance of non-cognitive and cognitive candidate attributes assessed. We used a stepwise process including a broad survey of SLOR readers and writers, analysis of past SLORs, and a multi-wave iterative revision that included key stakeholders, such as residency and internship program directors from academia and private practice. Data from the SLOR survey and analysis of past SLOR responses identified opportunities to improve applicant differentiation, mitigate positive bias, and encourage response consistency. The survey and other analytics identified and confirmed performance domains of high relevance. The revised SLOR assesses four performance domains: knowledge base and clinical skills, stress and time management, interpersonal skills, and personal characteristics. Ratings within the revised SLOR are predominantly criterion-referenced to enhance discernment of candidate attributes contained within each domain. Questions assessing areas of strength and targeted mentoring were replaced with free-text boxes, which allow writers to comment on positive and neutral/negative ratings of attributes within domains. Minor revisions were made to certain questions to enhance readability, streamline responses, or address targeted concerns identified in the SLOR survey or stakeholder review. The revised SLOR was deployed in the 2020 VIRMP; data from a survey of writers (n = 647) and readers (n = 378) indicate that the redesign objectives were achieved.

The Importance of Team Sex Composition in Team-Training Research Employing Complex Psychomotor Tasks.

OBJECTIVE: The relationship between team sex composition and team performance on a complex psychomotor task was examined because these types of tasks are commonly used in the lab-based teams literature. BACKGROUND: Despite well-documented sex-based differences on complex psychomotor tasks, the preponderance of studies-mainly lab based-that use these tasks makes no mention of the sex composition of teams across or within experimental conditions. METHOD: A sample of 123 four-person teams with varying team sex composition learned and performed a complex psychomotor task, Steal Beasts Pro PE. Each team completed a 5-hr protocol whereby they conducted several performance missions. RESULTS: The results indicated significant large mean differences such that teams with larger proportions of males had higher performance scores. CONCLUSION: These findings demonstrate the potential effect of team sex composition on the validity of studies that use complex psychomotor tasks to explore and investigate team performance-related phenomena when (a) team sex composition is not a focal variable of interest and (b) it is not accounted for or controlled. APPLICATION: Given the proclivity of complex psychomotor action-based tasks used in lab-based team studies, it is important to understand and control for the impact of team sex composition on team performance. When team sex composition is not controlled for, either methodologically or statistically, it may affect the validity of the results in teams studies using these types of tasks.

A meta-analysis of the effectiveness of the after-action review (or debrief) and factors that influence its effectiveness.

This study examined the effectiveness of the after-action review (AAR)-also commonly termed debrief-and 4 training characteristics within the context of Villado and Arthur's (2013) conceptual framework. Based on a bare-bones meta-analysis of the results from 61 studies (107 ds [915 teams and 3,499 individuals]), the AAR leads to an overall d of 0.79 improvement in multiple training evaluation criteria. This effect is larger than some of the largest training method effects reported in Arthur, Bennett, Edens, and Bell (2003), and it is also larger than Tannenbaum and Cerasoli's (2013) estimate of the effect of the AAR on task performance (d = 0.67). Two training characteristics consistently contributed to the effectiveness of the AAR: (a) alignment to the individual or the team, and (b) objective performance review media. The effects of the other training characteristics were often interactive. Most notably, the facilitation approach contributes to the effectiveness of the AAR in combination with the individual versus the team and the type of review media, with the most effective combinations being the self-led facilitation approach coupled with a team-aligned AAR, and the self-led approach coupled with objective media. Additionally, the AAR that is highly structured is more effective than a less structured AAR in the military, but high and low structured AARs display comparable effectiveness in healthcare. Overall, this study suggests that the effectiveness of the AAR should be understood as a function of the combined influence among multiple interacting characteristics. Future theoretical development and research should be directed at better understanding these interactions. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

The importance of distinguishing between constructs and methods when comparing predictors in personnel selection research and practice.

The authors highlight the importance and discuss the criticality of distinguishing between constructs and methods when comparing predictors. They note that comparisons of constructs and methods in comparative evaluations of predictors result in outcomes that are theoretically to conceptually uninterpretable and thus potentially misleading. The theoretical and practical implications of the distinction between predictor constructs and predictor methods are discussed, with three important streams of personnel psychology research being used to frame this discussion. Researchers, editors, reviewers, educators, and consumers of research are urged to carefully consider the extent to which the construct-method distinction is made and maintained in their own research and that of others, especially when predictors are being compared. It is hoped that this discussion will reorient researchers and practitioners toward a more construct-oriented approach that is aligned with a scientific emphasis in personnel selection research and practice.

An examination of factors contributing to a reduction in subgroup differences on a constructed-response paper-and-pencil test of scholastic achievement.

The authors investigated subgroup differences on a multiple-choice and constructed-response test of scholastic achievement in a sample of 197 African American and 258 White test takers. Although both groups had lower mean scores on the constructed-response test, the results showed a 39% reduction in subgroup differences compared with the multiple-choice test. The results demonstrate that the lower subgroup differences were explained by more favorable test perceptions for African Americans on the constructed-response test. In addition, the two test formats displayed comparable levels of criterion-related validity. The results suggest that the constructed-response test format may be a viable alternative to the traditional multiple-choice test format in efforts to simultaneously use valid predictors of performance and minimize subgroup differences in high-stakes testing.

The use of person-organization fit in employment decision making: an assessment of its criterion-related validity.

Because measures of person-organization (P-O) fit are accountable to the same psychometric and legal standards used for other employment tests when they are used for personnel decision making, the authors assessed the criterion-related validity of P-O fit as a predictor of job performance and turnover. Meta-analyses resulted in estimated true criterion-related validities of .15 (k = 36, N = 5,377) for P-O fit as a predictor of job performance and .24 (k = 8, N = 2,476) as a predictor of turnover, compared with a stronger effect of .31 (k = 109, N = 108,328) for the more commonly studied relation between P-O fit and work attitudes. In contrast to the relations between P-O fit and work attitudes, the lower 95% credibility values for the job performance and turnover relations included zero. In addition, P-O fit's relations with job performance and turnover were partially mediated by work attitudes. Potential concerns pertaining to the use of P-O fit in employment decision making are discussed in light of these results.

Relationships among team ability composition, team mental models, and team performance.

This study examined the relationship between the similarity and accuracy of team mental models and compared the extent to which each predicted team performance. The relationship between team ability composition and team mental models was also investigated. Eighty-three dyadic teams worked on a complex skill task in a 2-week training protocol. Results indicated that although similarity and accuracy of team mental models were significantly related, accuracy was a stronger predictor of team performance. In addition, team ability was more strongly related to the accuracy than to the similarity of team mental models and accuracy partially mediated the relationship between team ability and team performance, but similarity did not.

Red vs. green: Does the exam booklet color matter in higher education summative evaluations? Not likely.

We examined the so-called "red effect" in the context of higher education summative exams under the premise that unlike the conditions or situations where this effect typically has been obtained, the totality of factors, such as higher motivation, familiarity with exam material, and more reliance on domain knowledge that characterize high-stakes testing such as those in operational educational settings, are likely to mitigate any color effects. Using three naturally occurring archival data sets in which students took exams on either red or green exam booklets, the results indicated that booklet color (red vs. green) did not affect exam performance. From a scientific perspective, the results suggest that color effects may be attenuated by factors that characterize high-stakes assessments, and from an applied perspective, they suggest that the choice of red vs. green exam booklets in higher education summative evaluations is likely not a concern.

Team members' interaction anxiety and team-training effectiveness: a catastrophic relationship?

OBJECTIVE: In the present study, we examined the relationship between team members' interaction anxiety and team-training effectiveness. BACKGROUND: Training in the context of teams can be focused on either individual or team outcomes. Relatedly, the individual differences of team members can hinder or facilitate the effectiveness of training for both individuals and the team as a collective. Interaction anxiety is an understudied individual difference that has been shown to be negatively related to training effectiveness in individual contexts. However, its effects in team-training contexts are yet to be investigated. METHOD: A sample of 492 students in 123 four-person teams was trained to operate a complex, computer-based team task RESULTS: Interaction anxiety negatively related to team cohesion and team-training effectiveness. A cusp catastrophe model fit the data and showed that teams with more than one high-interaction-anxiety member were negatively affected. CONCLUSION: Interaction anxiety inhibits the social activities of team training, in turn reducing team-training effectiveness when there is more than one high-interaction-anxiety individual on the training team. APPLICATION: These results highlight the importance of examining interaction anxiety as a training team compositional variable that may inhibit behaviors and team processesnecessary to capitalize on the positive social activities on which team training depends since the interaction anxiety composition of training teams may serve as an important boundary condition on the effectiveness team-training interventions.

Effectiveness of training in organizations: a meta-analysis of design and evaluation features.

The authors used meta-analytic procedures to examine the relationship between specified training design and evaluation features and the effectiveness of training in organizations. Results of the meta-analysis revealed training effectiveness sample-weighted mean ds of 0.60 (k = 15, N = 936) for reaction criteria, 0.63 (k = 234, N = 15,014) for learning criteria, 0.62 (k = 122, N = 15,627) for behavioral criteria, and 0.62 (k = 26, N = 1,748) for results criteria. These results suggest a medium to large effect size for organizational training. In addition, the training method used, the skill or task characteristic trained, and the choice of evaluation criteria were related to the effectiveness of training programs. Limitations of the study along with suggestions for future research are discussed.

Comparative evaluation of three situational judgment test response formats in terms of construct-related validity, subgroup differences, and susceptibility to response distortion.

As a testing method, the efficacy of situational judgment tests (SJTs) is a function of a number of design features. One such design feature is the response format. However, despite the considerable interest in SJT design features, there is little guidance in the extant literature as to which response format is superior or the conditions under which one might be preferable to others. Using an integrity-based SJT measure administered to 31,194 job applicants, we present a comparative evaluation of 3 response formats (rate, rank, and most/least) in terms of construct-related validity, subgroup differences, and score reliability. The results indicate that the rate-SJT displayed stronger correlations with the hypothesized personality traits; weaker correlations with general mental ability and, consequently, lower levels of subgroup differences; and higher levels of internal consistency reliability. A follow-up study with 492 college students (Study 2; details of which are presented in the online supplemental materials) also indicates that the rate response format displayed higher levels of internal consistency and retest reliability as well as favorable reactions from test takers. However, it displayed the strongest relationships with a measure of response distortion, suggesting that it is more susceptible to this threat. Although there were a few exceptions, the rank and most/least response formats were generally quite similar in terms of several of the study outcomes. The results suggest that in the context of SJTs designed to measure noncognitive constructs, the rate response format appears to be the superior, preferred response format, with its main drawback being that it is susceptible to response distortion, although not any more so than the rank response format.

The comparative effect of subjective and objective after-action reviews on team performance on a complex task.

The after-action review (AAR; also known as the after-event review or debriefing) is an approach to training based on a review of trainees' performance on recently completed tasks or performance events. Used by the military for decades, nonmilitary organizations' use of AARs has increased dramatically in recent years. Despite the prevalence of AARs, empirical research investigating their effectiveness has been limited. This study sought to investigate the comparative effectiveness of objective AARs (reviews based on an objective recording and playback of trainees' recent performance) and subjective AARs (reviews based on a subjective, memory-based recall of trainees' recent performance). One hundred eighty-eight individuals, participating in 47 4-person teams, were assigned to 1 of 3 AAR conditions and practiced and tested on a cognitively complex performance task. Although there were no significant differences between objective and subjective AAR teams across the 5 training outcomes, AAR teams had higher levels of team performance, team efficacy, openness of communication, and cohesion than did non-AAR teams but no differences in their levels of team declarative knowledge. Our results suggest that AARs are effective at enhancing training outcomes. Furthermore, AARs may not be dependent on objective reviews and therefore may be a viable training intervention when objective reviews are not feasible or possible.

Team task analysis: differentiating between tasks using team relatedness and team workflow as metrics of team task interdependence.

OBJECTIVE: As a constructive replication and extension of Arthur, Edwards, Bell, Villado, and Bennett (2005), the objective of the current study was to further investigate the efficacy of team relatedness and team workflow ratings (along with their composite) as metrics of interdependence. BACKGROUND: Although an analysis of task and job interdependence has important implications and uses in domains such as job design, selection, and training, the job analysis literature has been slow to develop an effective method to identify team-based tasks and jobs. METHOD: To achieve the study's objectives, 140 F-16 fighter pilots (35 four-person teams) rated 34 task and activity statements in terms of their team relatedness and team workflow. RESULTS: The results indicated that team relatedness and team workflow effectively differentiated between tasks with varying levels of interdependency (as identified by instructor pilots who served as subject matter experts) within the same job. In addition, teams that accurately perceived the level of interdependency performed better on a four-ship F-16 flight-training program than those that did not. CONCLUSION: Team relatedness and team workflow ratings can effectively differentiate between tasks with varying levels of interdependency. APPLICATION: Like traditional individual task or job analysis, this information can serve as the basis for specified human resource functions and interventions, and as diagnostic indicators as well.

Safety climate and injuries: an examination of theoretical and empirical relationships.

Our purpose in this study was to meta-analytically address several theoretical and empirical issues regarding the relationships between safety climate and injuries. First, we distinguished between extant safety climate-->injury and injury-->safety climate relationships for both organizational and psychological safety climates. Second, we examined several potential moderators of these relationships. Meta-analyses revealed that injuries were more predictive of organizational safety climate than safety climate was predictive of injuries. Additionally, the injury-->safety climate relationship was stronger for organizational climate than for psychological climate. Moderator analyses revealed that the degree of content contamination in safety climate measures inflated effects, whereas measurement deficiency attenuated effects. Additionally, moderator analyses showed that as the time period over which injuries were assessed lengthened, the safety climate-->injury relationship was attenuated. Supplemental meta-analyses of specific safety climate dimensions also revealed that perceived management commitment to safety is the most robust predictor of occupational injuries. Contrary to expectations, the operationalization of injuries did not meaningfully moderate safety climate-injury relationships. Implications and recommendations for future research and practice are discussed.

Team task analysis: identifying tasks and jobs that are team based.

This paper presents initial information on the development and validation of three team task analysis scales. These scales were designed to quantitatively assess the extent to which a group of tasks or a job is team based. During a 2-week period, 52 male students working in 4-person teams were trained to perform a complex highly interdependent computer-simulated combat mission consisting of both individual- and team-based tasks. Our results indicated that the scales demonstrated high levels of interrater agreement. In addition, the scales differentiated between tasks that were predetermined to be individual versus team based. Finally, the results indicated that job-level ratings of team workflow were more strongly related to team performance than were aggregated task-level ratings of team-relatedness or team workflow. These results suggest that the scales presented here are an effective means of quantifying the extent to which tasks or jobs are team based. A research and practical implication of our findings is that the team task analysis scales could serve as criterion measures in the evaluation of team training interventions or predictors of team performance.

Convergence of self-report and archival crash involvement data: a two-year longitudinal follow-up.

This study constructively extends Arthur et al. (2001) by assessing the convergence of self-report and archival motor vehicle crash involvement and moving violations data in a 2-year longitudinal follow-up. The relationships among these criteria, conscientiousness, and driving speed were also assessed using both predictive and postdictive criterion-related validation designs. Data were collected from a 2-year follow-up sample of 334 participants. Results suggested a lack of convergence between self-report and archival data at both Time 1 and Time 2. In addition, the predictor/criterion relationships varied across research design and data source. An actual application of our findings is that the interpretation of relationships between specified predictors and crash involvement and moving violations must be made within the context of the criterion-related validation design and criterion data source. Specifically, predictive designs may produce results different from those of postdictive designs (which are more commonly used). Furthermore, self-report data appear to include a broader range of incidents (more crashes and tickets), and thus researchers should consider using self-report data when they are interested in including lower threshold crashes and tickets that may not be reported on state records (e.g., because of the completion of a defensive driving course).