Uncontrolled control variables: The extent that a researcher's degrees of freedom with control variables increases various types of statistical errors.

Researchers have increasingly warned about "p-hacking" and the improper use of control variables. This paper considers the risk that a "researcher's degrees of freedom" with respect to the use of control variables has on the probability of Type I and Type II errors. We also examine the extent that control variables can make marginal effect sizes (i.e., nonzero effects but less than should be statistically significant) appear significant (which we refer to as Type III errors) and how much control variable use can increase effect sizes. We report the results of two computer simulations that include up to 10 control variables. We find that the inappropriate use of control variables is not really a risk for Type I errors, given that the chance of there being truly a null effect in a typical multivariate analysis is very low. We also show that the use of control variables does not have a large effect on Type II errors, and that the practice of running analyses both with and without control variables will most often yield the same conclusion in both analyses. That said, we did find that p-hacking substantially increases the probability of inappropriately being able to detect statistical significance and can notably increase effect sizes. The practice of running analyses both with and without control variables can indeed reveal the potential for p-hacking, and discrepant results between bivariate and multivariate analyses suggest that authors need to carefully and clearly explain why the noted differences are theoretically and logically appropriate. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Only time will tell: the changing relationships between LMX, job performance, and justice.

Although it has been argued that leader-member exchange (LMX) is a phenomenon that develops over time, the existing LMX literature is largely cross-sectional in nature. Yet, there is a great need for unraveling how LMX develops over time. To address this issue in the LMX literature, we examine the relationships of LMX with 2 variables known for changing over time: job performance and justice perceptions. On the basis of current empirical findings, a simulation deductively shows that LMX develops over time, but differently in early stages versus more mature stages. Our findings also indicate that performance and justice trends affect LMX. Implications for LMX theory and for longitudinal research on LMX, performance, and justice are discussed. (PsycINFO Database Record

The effect of culture on the curvilinear relationship between performance and turnover.

Although researchers have theorized that there exists a curvilinear relationship between job performance and voluntary turnover, their research has been tested in the United States or culturally similar Switzerland. Through a study of the performance-turnover relationship from a multinational service-oriented organization in 24 countries, we demonstrate that the general relationship between performance and turnover is similar across countries but the details of that relationship change across countries. Using 4 cultural dimensions--in-group collectivism, power distance, uncertainty avoidance, and performance orientation--we find that cultural factors alter the overall probability of voluntary turnover and influence the degree of curvilinearity in the performance-turnover relationship. Our findings have implications for research on the performance-turnover relationship, turnover research, and practice.

Justice as a dynamic construct: effects of individual trajectories on distal work outcomes.

Despite an amassing organizational justice literature, few studies have directly addressed the temporal patterning of justice judgments and the effects that changes in these perceptions have on important work outcomes. Drawing from Gestalt characteristics theory (Ariely & Carmon, 2000, 2003), we examine the concept of justice trajectories (i.e., levels and trends of individual fairness perceptions over time) and offer empirical evidence to highlight the value of considering fairness within a dynamic context. Participants included 523 working adults who completed surveys about their work experiences on 4 occasions over the course of 1 year. Results indicate that justice trends explained additional variance in distal work outcomes (job satisfaction, organizational commitment, and turnover intentions) after controlling for end-state levels of justice, demonstrating the cumulative effects of justice over time. Findings also reveal that change in procedural justice perceptions affected distal work outcomes more strongly than any other justice dimension. Implications for theory and future investigations of justice as a dynamic construct are discussed.

The impact of job complexity and performance measurement on the temporal consistency, stability, and test-retest reliability of employee job performance ratings.

Although research has shown that individual job performance changes over time, the extent of such changes is unknown. In this article, the authors define and distinguish between the concepts of temporal consistency, stability, and test-retest reliability when considering individual job performance ratings over time. Furthermore, the authors examine measurement type (i.e., subjective and objective measures) and job complexity in relation to temporal consistency, stability, and test-retest reliability. On the basis of meta-analytic results, the authors found that the test-retest reliability of these ratings ranged from .83 for subjective measures in low-complexity jobs to .50 for objective measures in high-complexity jobs. The stability of these ratings over a 1-year time lag ranged from .85 to .67. The analyses also reveal that correlations between performance measures decreased as the time interval between performance measurements increased, but the estimates approached values greater than zero.