A review of applications of the Bayes factor in psychological research.

The last 25 years have shown a steady increase in attention for the Bayes factor as a tool for hypothesis evaluation and model selection. The present review highlights the potential of the Bayes factor in psychological research. We discuss six types of applications: Bayesian evaluation of point null, interval, and informative hypotheses, Bayesian evidence synthesis, Bayesian variable selection and model averaging, and Bayesian evaluation of cognitive models. We elaborate what each application entails, give illustrative examples, and provide an overview of key references and software with links to other applications. The article is concluded with a discussion of the opportunities and pitfalls of Bayes factor applications and a sketch of corresponding future research lines. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

How to Evaluate Theory-Based Hypotheses in Meta-Analysis Using an AIC-Type Criterion.

Meta-analysis techniques allow researchers to aggregate effect sizes-like standardized mean difference(s), correlation(s), or odds ratio(s)-of different studies. This leads to overall effect-size estimates and their confidence intervals. Additionally, researchers can aim for theory development or theory evaluation. That is, researchers may not only be interested in these overall estimates but also in a specific ordering or size of them, which then reflects a theory. Researchers may have expectations regarding the ordering of standardized mean differences or about the (ranges of) sizes of an odds ratio or Hedges' g. Such theory-based hypotheses most probably contain inequality constraints and can be evaluated with the Akaike's information criterion type (i.e., AIC-type) confirmatory model selection criterion called generalized order-restricted information criterion (GORICA). This paper introduces and illustrates how the GORICA can be applied to meta-analyzed estimates. Additionally, it compares the use of the GORICA to that of classical null hypothesis testing and the AIC, that is, the use of theory-based hypotheses versus null hypotheses. By using the GORICA, researchers from all types of fields (e.g., psychology, sociology, political science, biomedical science, and medicine) can quantify the support for theory-based hypotheses specified a priori. This leads to increased statistical power, because of (i) the use of theory-based hypotheses (cf. one-sided vs. two-sided testing) and (ii) the use of meta-analyzed results (that are based on multiple studies which increase the combined sample size). The quantification of support and the power increase aid in, for instance, evaluating and developing theories and, therewith, developing evidence-based treatments and policy.

Relationship quality in higher education and the interplay with student engagement and loyalty.

BACKGROUND: To date, studies that have investigated the bonds between students and their institution have emphasized the importance of student-staff relationships. Measuring the quality of those relationships (i.e., relationship quality) appears to help with investigating the relational ties students have with their higher education institutions. Growing interest has arisen in further investigating relationship quality in higher education, as it might predict students' involvement with the institution (e.g., student engagement and student loyalty). So far, most studies have used a cross-sectional design, so that causality could not be determined. AIMS: The aim of this longitudinal study was twofold. First, we investigated the temporal ordering of the relation between the relationship quality dimensions of trust (in benevolence and honesty) and affect (satisfaction, affective commitment, and affective conflict). Second, we examined the ordering of the paths between relationship quality, student engagement, and student loyalty. Our objectives were to gain a deeper understanding of the relationship quality construct in higher education and its later outcomes. SAMPLE: Participants (N = 1649) were students from three Dutch higher education institutions who were studying in a technology economics or social sciences program. METHODS: Longitudinal data from two time points were used to evaluate two types of cross-lagged panel models. In the first analysis, we could not assume measurement invariance for affective conflict over time. Therefore, we tested an alternative model without affective conflict, using the latent variables of trust and affect, the student engagement dimensions and student loyalty. In the second type of model, we investigated the manifest variables of relationship quality, student engagement, and student loyalty. The hypotheses were tested by evaluating simultaneous comparisons between estimates. RESULTS: Results indicated that the relation between relationship quality at Time 1 with student engagement and loyalty at Time 2 was stronger than the reverse ordering in the first model. In the second model, results indicated that cross-lagged relations between trust in benevolence and trust in honesty at Time 1 and affective commitment, affective conflict, and satisfaction at Time 2 were more likely than the reverse ordering. Furthermore, cross-lagged relations from relationship quality at Time 1 to student engagement and student loyalty at Time 2 also supported our hypothesis. CONCLUSIONS: This study contributes to the existing higher education literature, indicating that students' trust in the quality of their relationship with faculty/staff is essential for developing students' affective commitment and satisfaction and for avoiding conflict over time. Second, relationship quality factors positively influence students' engagement in their studies and their loyalty towards the institution. A relational approach to establishing (long-lasting) bonds with students appears to be fruitful as an approach for educational psychologists and for practitioners' guidance and strategies. Recommendations are made for future research to further examine relationship quality in higher education in Europe and beyond.

Evaluation of inequality constrained hypotheses using a generalization of the AIC.

In the social and behavioral sciences, it is often not interesting to evaluate the null hypothesis by means of a p-value. Researchers are often more interested in quantifying the evidence in the data (as opposed to using p-values) with respect to their own expectations represented by equality and/or inequality constrained hypotheses (as opposed to the null hypothesis). This article proposes an Akaike-type information criterion (AIC; Akaike, 1973, 1974) called the generalized order-restricted information criterion approximation (GORICA) that evaluates (in)equality constrained hypotheses under a very broad range of statistical models. The results of five simulation studies provide empirical evidence showing that the performance of the GORICA on selecting the best hypothesis out of a set of (in)equality constrained hypotheses is convincing. To illustrate the use of the GORICA, the expectations of researchers are investigated in a logistic regression, multilevel regression, and structural equation model. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Business Results and Well-Being: An Engaging Leadership Intervention Study.

The present quasi-experimental study tested the business impact of a leadership development program focusing on psychological well-being through the satisfaction of basic psychological needs. Based on the concept of engaging leadership and self-determination theory, the 8-month program targeted midlevel team leaders of the customer fulfilment center of a health systems multinational organization. The program was designed in co-creation between senior leadership and the team leaders that participated in the program. Outcomes showed positive business results through significant increases in a preselected key performance indicator and decreased employee absenteeism. Through changes in autonomy satisfaction and intrinsic motivation, the team leaders (N = 14) benefitted in a moderate to very large extent relative to a similar control group (N = 52). In contrast, team members (N = 148) displayed no such benefits. Specifically, higher levels of autonomy satisfaction are said to lead to higher levels of psychological well-being and motivation. Still, the link with business performance is absent in most organizational studies within self-determination theory, making the present study one of the first to fill this gap. The study discloses the program design, compares the effects to a relevant control group, evaluates the lessons learned, and provides practical suggestions.

Evaluating a theory-based hypothesis against its complement using an AIC-type information criterion with an application to facial burn injury.

An information criterion (IC) like the Akaike IC (AIC), can be used to select the best hypothesis from a set of competing theory-based hypotheses. An IC developed to evaluate theory-based order-restricted hypotheses is the Generalized Order-Restricted Information Criterion (GORIC). Like for any IC, the values themselves are not interpretable but only comparable. To improve the interpretation regarding the strength, GORIC weights and related evidence ratios can be computed. However, if the unconstrained hypothesis (the default) is used as competing hypothesis, the evidence ratio is not affected by sample-size nor effect-size in case the hypothesis of interest is (also) in agreement with the data. In practice, this means that in such a case strong support for the order-restricted hypothesis is not reflected by a high evidence ratio. Therefore, we introduce the evaluation of an order-restricted hypothesis against its complement using the GORIC (weights). We show how to compute the GORIC value for the complement, which cannot be achieved by current methods. In a small simulation study, we show that the evidence ratio for the order-restricted hypothesis versus the complement increases for larger samples and/or effect-sizes, while the evidence ratio for the order-restricted hypothesis versus the unconstrained hypothesis remains bounded. An empirical example about facial burn injury illustrates our method and shows that using the complement as competing hypothesis results in much more support for the hypothesis of interest than using the unconstrained hypothesis as competing hypothesis. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

A critique of the cross-lagged panel model.

The cross-lagged panel model (CLPM) is believed by many to overcome the problems associated with the use of cross-lagged correlations as a way to study causal influences in longitudinal panel data. The current article, however, shows that if stability of constructs is to some extent of a trait-like, time-invariant nature, the autoregressive relationships of the CLPM fail to adequately account for this. As a result, the lagged parameters that are obtained with the CLPM do not represent the actual within-person relationships over time, and this may lead to erroneous conclusions regarding the presence, predominance, and sign of causal influences. In this article we present an alternative model that separates the within-person process from stable between-person differences through the inclusion of random intercepts, and we discuss how this model is related to existing structural equation models that include cross-lagged relationships. We derive the analytical relationship between the cross-lagged parameters from the CLPM and the alternative model, and use simulations to demonstrate the spurious results that may arise when using the CLPM to analyze data that include stable, trait-like individual differences. We also present a modeling strategy to avoid this pitfall and illustrate this using an empirical data set. The implications for both existing and future cross-lagged panel research are discussed.

Properties of hypothesis testing techniques and (Bayesian) model selection for exploration-based and theory-based (order-restricted) hypotheses.

In this paper, the performance of six types of techniques for comparisons of means is examined. These six emerge from the distinction between the method employed (hypothesis testing, model selection using information criteria, or Bayesian model selection) and the set of hypotheses that is investigated (a classical, exploration-based set of hypotheses containing equality constraints on the means, or a theory-based limited set of hypotheses with equality and/or order restrictions). A simulation study is conducted to examine the performance of these techniques. We demonstrate that, if one has specific, a priori specified hypotheses, confirmation (i.e., investigating theory-based hypotheses) has advantages over exploration (i.e., examining all possible equality-constrained hypotheses). Furthermore, examining reasonable order-restricted hypotheses has more power to detect the true effect/non-null hypothesis than evaluating only equality restrictions. Additionally, when investigating more than one theory-based hypothesis, model selection is preferred over hypothesis testing. Because of the first two results, we further examine the techniques that are able to evaluate order restrictions in a confirmatory fashion by examining their performance when the homogeneity of variance assumption is violated. Results show that the techniques are robust to heterogeneity when the sample sizes are equal. When the sample sizes are unequal, the performance is affected by heterogeneity. The size and direction of the deviations from the baseline, where there is no heterogeneity, depend on the effect size (of the means) and on the trend in the group variances with respect to the ordering of the group sizes. Importantly, the deviations are less pronounced when the group variances and sizes exhibit the same trend (e.g., are both increasing with group number).

Comparisons of means using exploratory and confirmatory approaches.

This article discusses comparisons of means using exploratory and confirmatory approaches. Three methods are discussed: hypothesis testing, model selection based on information criteria, and Bayesian model selection. Throughout the article, an example is used to illustrate and evaluate the two approaches and the three methods. We demonstrate that confirmatory hypothesis testing techniques have more power-that is, have a higher probability of rejecting a false null hypothesis-and confirmatory model selection techniques have a higher probability of choosing the correct or the best hypothesis than their exploratory counterparts. Furthermore, we show that if more than one hypothesis has to be evaluated, model selection has advantages over hypothesis testing. Another, more elaborate example is used to further illustrate confirmatory model selection. The article concludes with recommendations: When a researcher is able to specify reasonable expectations and hypotheses, confirmatory model selection should be used; otherwise, exploratory model selection should be used.