A Confidence Interval for the Difference Between Standardized Regression Coefficients.

Researchers are often interested in comparing predictors, a practice commonly done via informal comparisons of standardized regression slopes. However, formal interval-based approaches offer advantages over informal comparison. Specifically, this article examines a delta-method-based confidence interval for the difference between two standardized regression coefficients, building upon previous work on confidence intervals for single coefficients. Using Monte Carlo simulation studies, the proposed approach is evaluated at finite sample sizes with respect to coverage rate, interval width, Type I error rate, and statistical power under a variety of conditions, and is shown to outperform an alternative approach that uses the standard covariance matrix found in regression textbooks. Additional simulations evaluate current software implementations, small sample performance, and multiple comparison procedures for simultaneously testing multiple differences of interest. Guidance on sample size planning for narrow confidence intervals, an R function to conduct the proposed method, and two empirical demonstrations are provided. The goal is to offer researchers a different tool in their toolbox for when comparisons among standardized coefficients are desired, as a supplement to, rather than a replacement for, other potentially useful analyses.

Evaluating avoidable heterogeneity in exploratory factor analysis results.

Meaningful interpretations of scores derived from psychological scales depend on the replicability of psychometric properties. Despite this, and unexpected inconsistencies in psychometric results across studies, psychometrics has often been overlooked in the replication literature. In this article, we begin to address replication issues in exploratory factor analysis (EFA). We use a Monte Carlo simulation to investigate methodological choices made throughout the EFA process that have the potential to add heterogeneity to results. Our findings show that critical decision points for EFA include the method for determining the number of factors as well as rotation. The results also demonstrate the relevancy of data characteristics, as some contexts are more susceptible to the effects of methodological choice on the heterogeneity of results. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Questionable research practices and cumulative science: The consequences of selective reporting on effect size bias and heterogeneity.

Despite increased attention to open science and transparency, questionable research practices (QRPs) remain common, and studies published using QRPs will remain a part of the published record for some time. A particularly common type of QRP involves multiple testing, and in some forms of this, researchers report only a selection of the tests conducted. Methodological investigations of multiple testing and QRPs have often focused on implications for a single study, as well as how these practices can increase the likelihood of false positive results. However, it is illuminating to consider the role of these QRPs from a broader, literature-wide perspective, focusing on consequences that affect the interpretability of results across the literature. In this article, we use a Monte Carlo simulation study to explore the consequences of two QRPs involving multiple testing, cherry picking and question trolling, on effect size bias and heterogeneity among effect sizes. Importantly, we explicitly consider the role of real-world conditions, including sample size, effect size, and publication bias, that amend the influence of these QRPs. Results demonstrated that QRPs can substantially affect both bias and heterogeneity, although there were many nuances, particularly relating to the influence of publication bias, among other factors. The present study adds a new perspective to how QRPs may influence researchers' ability to evaluate a literature accurately and cumulatively, and points toward yet another reason to continue to advocate for initiatives that reduce QRPs. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Validity and reliability of the Body-Esteem Scale among a diverse sample of preadolescent youth.

Researchers have called for more psychometric research on body image assessment measures in children. The Body-Esteem Scale (BES) is a commonly used 24-item measure of self-evaluation of body image in youth, yet only one study has evaluated the psychometric properties of this measure in preadolescent children. The goal of the present study was to replicate and extend the original psychometric work on the BES by examining the factor structure, reliability, and convergent validity in a diverse sample of children ages 9-11 years old. Teachers who taught 4th- and 5th-grade students in the United States were invited to participate. A total of 556 students completed an online survey in their classrooms. Findings indicate a one-factor model best fits the data after eliminating three BES items. The BES had strong scale score consistency. Supporting the convergent validity of the BES, body-esteem scores were positively correlated with general self-esteem and body satisfaction scores, while negatively correlated with body dissatisfaction, engagement in appearance conversations, and social comparison scores. There were no significant group differences on mean BES scores between gender and grade level. Overall, the BES is suitable to assess the body esteem of preadolescent children. Future research needs to replicate these findings and examine the BES's ability to detect a change in body-esteem scores across time in youth. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

The effects of the 4/20 cannabis holiday and adult-use cannabis legalization on medical cannabis sales and medical cannabis patient registration in Arizona.

BACKGROUND: This study examined whether the 4/20 cannabis holiday was associated with increases in medical cannabis sales from licensed dispensaries in Arizona from 2018-2021, and whether adult-use cannabis legalization (the vote in November 2020 and retail sales in January 2021) was associated with declines in medical cannabis sales and in the number of registered medical patients. METHODS: Data came from the Arizona Medical Marijuana Program monthly reports from January 2018-December 2021. The reports show daily sales from licensed medical cannabis dispensaries (i.e., the number of medical cannabis dispensary transactions and the amount of cannabis sold in pounds), which we averaged by week, and show the number of registered medical cannabis patients each month. Autoregressive integrated moving average models were used to test changes in these outcomes associated with the 4/20 cannabis holiday and with legalization of adult-use cannabis. RESULTS: During the week of the 4/20 cannabis holiday, medical cannabis dispensary transactions abruptly increased by an average of 2,319.4 transactions each day (95% CI: 1636.1, 3002.7), and the amount of medical cannabis sold increased by an average of 120.3 pounds each day (95% CI: 99.3-141.3). During the first week of adult-use cannabis sales in late January 2021, medical cannabis dispensary transactions abruptly decreased by an average of 5,073 transactions each day (95% CI: -5,929.5, -4216.7), and the amount of medical cannabis sold decreased by an average of 119.1 pounds each day (95% CI: -144.2, -94.0). Moreover, medical cannabis sales continued to gradually decline each week after the start of adult-use retail sales, with declines in sales preceding declines in registered patients. By December 2021, slightly over a year after the vote to legalize adult-use cannabis, the actual number of registered medical cannabis patients fell short of the forecasted number, had adult-use not been legalized, by 36.5%. Moreover, the number of medical dispensary transactions and the amount of medical cannabis sold fell short of expectations, had adult-use cannabis not been legalized, by 58% and 53%, respectively. CONCLUSIONS: Findings document the blurred boundary between medical and non-medical cannabis use and are consistent with the possibility that medical cannabis legalization contributes to increases in adult cannabis use and dependence.

Appropriately estimating the standardized average treatment effect with missing data: A simulation and primer.

Reporting standardized effects in randomized treatment studies aids interpretation and facilitates future meta-analyses and policy considerations. However, when outcome data are missing, achieving an unbiased, accurate estimate of the standardized average treatment effect, sATE, can pose challenges even for those with general knowledge of missing data handling, given that the sATE is a ratio of a mean difference to a (within-group) standard deviation. Under both homogeneity and heterogeneity of variance, a Monte Carlo simulation study was conducted to compare missing data handling strategies in terms of bias and accuracy in the sATE, under specific missingness patterns plausible for randomized pretest posttest studies. Within two broad missing data handling approaches, maximum likelihood and multiple imputation, modeling choices were thoroughly investigated including the analysis model, variance estimator, imputation algorithm, and method of pooling results across imputed datasets. Results demonstrated that although the sATE can be estimated with little bias using either maximum likelihood or multiple imputation, particular attention should be paid to the model and variance estimator, especially at smaller sample sizes (i.e., N = 50). Differences in accuracy were driven by differences in bias. To improve estimation of the sATE in practice, recommendations and a software demonstration are provided. Moreover, a pedagogical explanation of the causes of bias, described separately for the numerator and denominator of the sATE is provided, demonstrating visually how and why bias occurs with certain methods.

Sample size planning for replication studies: The devil is in the design.

Replication is central to scientific progress. Because of widely reported replication failures, replication has received increased attention in psychology, sociology, education, management, and related fields in recent years. Replication studies have generally been assessed dichotomously, designated either a "success" or "failure" based entirely on the outcome of a null hypothesis significance test (i.e., p < .05 or p > .05, respectively). However, alternative definitions of success depend on researchers' goals for the replication. Previous work on alternative definitions for success has focused on the analysis phase of replication. However, the design of the replication is also important, as emphasized with the adage, "an ounce of prevention is better than a pound of cure." One critical component of design often ignored or oversimplified in replication studies is sample size planning, indeed, the details here are crucial. Sample size planning for replication studies should correspond to the method by which success will be evaluated. Researchers have received little guidance, some of which is misguided, on sample size planning for replication goals other than the aforementioned dichotomous null hypothesis significance testing approach. In this article, we describe four different replication goals. Then, we formalize sample size planning methods for each of the four goals. This article aims to provide clarity on the procedures for sample size planning for each goal, with examples and syntax provided to show how each procedure can be used in practice. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Multiplicity in multiple regression: Defining the issue, evaluating solutions, and integrating perspectives.

When multiple hypothesis tests are conducted, the familywise Type I error probability correspondingly increases. Various multiple test procedures (MTPs) have been developed, which generally aim to control the familywise Type I error rate at the desired level. However, although multiplicity is frequently discussed in the ANOVA literature and MTPs are correspondingly employed, the issue has received considerably little attention in the regression literature and it is rare to see MTPs employed empirically. The present aims are three-fold. First, within the eclectic uses of multiple regression, specific situations are delineated wherein adjusting for multiplicity may be most relevant. Second, the performance of ten MTPs amenable to regression is investigated via familywise Type I error control, statistical power, and, where appropriate, false discovery rate, simultaneous confidence interval coverage and width. Although methodologists may anticipate general patterns, the focus is on the magnitude of error inflation and the size of the differences among methods under plausible scenarios. Third, perspectives from across the scientific literature are discussed, which shed light on contextual factors to consider when evaluating whether multiplicity adjustment is advantageous. Results indicated that multiple testing can be problematic, even in nonextreme situations where multiplicity consequences may not be immediately expected. Results pointed toward several effective, balanced, MTPs, particularly those that accommodate correlated parameters. Importantly, the goal is not to universally recommend MTPs for all regression models, but. rather to identify a set of circumstances wherein multiplicity is most relevant, evaluate MTPs, and integrate diverse perspectives that suggest multiplicity adjustment or alternate solutions. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

A revised and expanded taxonomy for understanding heterogeneity in research and reporting practices.

Concerns about replication failures can be partially recast as concerns about excessive heterogeneity in research results. Although this heterogeneity is an inherent part of science (e.g., sampling variability; studying different conditions), not all heterogeneity results from unavoidable sources. In particular, the flexibility researchers have when designing studies and analyzing data adds additional heterogeneity. This flexibility has been the topic of considerable discussion in the last decade. Ideas, and corresponding phrases, have been introduced to help unpack researcher behaviors, including researcher degrees of freedom and questionable research practices. Using these concepts and phrases, methodological and substantive researchers have considered how researchers' choices impact statistical conclusions and reduce clarity in the research literature. While progress has been made, inconsistent, vague, and overlapping use of the terminology surrounding these choices has made it difficult to have clear conversations about the most pressing issues. Further refinement of the language conveying the underlying concepts can catalyze further progress. We propose a revised, expanded taxonomy for assessing research and reporting practices. In addition, we redefine several crucial terms in a way that reduces overlap and enhances conceptual clarity, with particular focus on distinguishing practices along two lines: research versus reporting practices and choices involving multiple empirically supported options versus choices known to be subpar. We illustrate the effectiveness of these changes using conceptual and simulated demonstrations, and we discuss how this taxonomy can be valuable to substantive researchers by helping to navigate this flexibility and to methodological researchers by motivating research toward areas of greatest need. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

The prospective interactive effects of alcohol expectancies and subjective response on future drinking behavior.

Although alcohol expectancies and subjective response are independent predictors of drinking, social-cognitive theory suggests that expectancies may distort one's subjective response, creating discrepancies between expected and actual alcohol effects. A recent cross-sectional study found that unmet expectancies (using difference scores) were associated with heavier drinking. However, cross-sectional data cannot establish temporal precedence, and using difference scores ignores important conditional main effects. As such, the current study sought to evaluate how expectancy-subjective response discrepancies predict future drinking using prospective data and an interaction approach. Participants (N = 258) were randomly assigned to consume alcohol (target BAC = .08%) within a placebo-controlled alcohol administration session. Alcohol expectancies and subjective response were assessed across the full valence by arousal affective space using parallel measures. Results indicated a significant high arousal positive (HIGH+) interaction, such that, as HIGH+ expectancies increased, individuals at low and mean levels of HIGH+ subjective response drank more heavily 12 months later. There was also a significant high arousal negative (HIGH-) interaction with a similar pattern of moderated effects. No interactions were found for low arousal effects. These results indicate that individuals with unmet HIGH+ and HIGH- expectancies drink more heavily 12 months later, controlling for prior drinking. This suggests that clinicians may consider recommending specific interventions (e.g., expectancy challenges vs. pharmacotherapy) based upon an individual's levels of expectancies and subjective response to optimize intervention efficacy. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Discrimination and depressive symptoms in sexual minority Latinx youth: Moderation by religious importance and attendance.

Relative to heterosexual peers, sexual minority youth experience significant mental health disparities. This is partly due to prejudicial encounters (e.g., discrimination, victimization) because of their sexual minority status, and potential intersecting and compounding prejudicial experiences related to their ethnic minority status. However, even though religiosity has been identified as a protective factor in the general literature and may be especially relevant for youth in Latinx families, few studies have examined whether religiosity serves as a buffer of the relations between discrimination-related stress and mental health in sexual minority Latinx youth. Thus, the aims were to examine (a) whether ethnic and sexuality discrimination have additive or multiplicative effects on depressive symptoms, and (b) whether own or family religiosity (defined by religious importance and attendance) moderates the relations between discrimination and depressive symptoms. Sample included 377 youth (Mage = 20.29, SD = 2.61, age range 14-24). Results did not support an interaction between ethnic and sexuality discrimination, nor between own religiosity and ethnic discrimination, in predicting depressive symptoms. There were significant interactions between family religiosity and discrimination (ethnic and sexuality), in which family religiosity was negatively associated with depressive symptoms only at average and low levels of discrimination. There was a significant interaction between own religiosity and sexuality discrimination, in which own religiosity was negatively associated with depressive symptoms only at the low level of sexuality discrimination. Findings highlight the importance of examining the intersection of religion, sexuality, and Latinx minority status in relation to mental health outcome. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Power(ful) myths: misconceptions regarding sample size in quality of life research.

PURPOSE: Carefully selecting the sample size for a research study is one of the most fundamental ways to utilize resources in an ethical manner, maximize impact and replicability, and minimize research waste when investigating questions relevant to health-related quality of life (HRQOL). Despite an increased focus on sample size in the methodological literature, the topic has received limited attention in the HRQOL field, and there are still misconceptions that can weaken even well-intentioned sample size planning. This article aims to highlight common misconceptions, provide accessible and non-technical corrections to these misconceptions, and show how HRQOL researchers can benefit from a more nuanced understanding of sample size planning. METHOD: Misconceptions were identified broadly through examples within the health, psychology, and HRQOL literatures. In examining these misconceptions, study-level (e.g., missing data, multilevel designs, multiple reported outcomes) and field-level (e.g., publication bias, replicability) issues relevant to HRQOL research were considered. RESULTS: Misconceptions include: (a) researchers should use rules of thumb or the largest sample size possible, (b) sample size planning should always focus on power, (c) planned power = actual power, (d) there is only one level of power per study, and (e) power is only relevant for the individual researcher. Throughout the article, major themes linked to these misconceptions are mapped onto recent HRQOL studies to make the connections more tangible. CONCLUSION: By clarifying several challenges and misconceptions regarding sample size planning and statistical power, HRQOL researchers will have the tools needed to augment the research literature in effective and meaningful ways.

The click is not the trick: the efficacy of clickers and other reinforcement methods in training naïve dogs to perform new tasks.

BACKGROUND: A handheld metal noisemaker known as a "clicker" is widely used to train new behaviors in dogs; however, evidence for their superior efficacy compared to providing solely primary reinforcement or other secondary reinforcers in the acquisition of novel behavior in dogs is largely anecdotal. METHODS: Three experiments were conducted to determine under what circumstances a clicker secondary reinforcer may result in acquisition of a novel behavior more rapidly or to a higher level compared to other readily available reinforcement methods. In Experiment 1, three groups of 30 dogs each were shaped to emit a novel sit and stay behavior of increasing duration with either the delivery of food alone, a verbal stimulus paired with food, or a clicker with food. The group that received only a primary reinforcer reached a significantly higher criterion of training success than the group trained with a verbal secondary reinforcer. Performance of the group experiencing a clicker as a secondary reinforcer was intermediate between the other two groups, but not significantly different from either. In Experiment 2, three groups of 25 dogs each were shaped to emit a nose targeting behavior and then perform that behavior at increasing distances from the experimenter using the same three methods of positive reinforcement as in Experiment 1. No statistically significant differences between the groups were found. In Experiment 3, three groups of 30 dogs each were shaped to emit a nose-targeting behavior upon an array of wooden blocks with task difficulty increasing throughout testing using the same three methods of positive reinforcement as previously tested. No statistically significant differences between the groups were found. RESULTS: Overall, the findings suggest that both primary reinforcement alone as well as a verbal or clicker secondary reinforcer can be used successfully in training a dog to perform a novel behavior, but that no positive reinforcement method demonstrated significantly greater efficacy than any other.

Negative affect reactivity to stress and internalizing symptoms over the transition to college for Latinx adolescents: Buffering role of family support.

Affect reactivity to stress may play a role in the development of internalizing symptoms during the college transition, a critical developmental juncture for Latinx adolescents, the largest ethnic minority group on college campuses. This study examined whether affect reactivity during high school is associated with internalizing symptoms in college and explored two potential protective factors, perceived family and peer support. Participants were 209 Latinx adolescents (Mage = 18.10; 64.4% female) who completed standard surveys and four diary assessments per day over 7 days (N > 4,500 momentary observations). First, to measure affect reactivity, we assessed whether perceived stress was associated with negative affect at the momentary level during high school (senior year). Second, we tested whether affect reactivity predicted internalizing symptoms during the first year of college. Third, we tested whether perceived family or peer support buffered the negative consequences of affect reactivity. Results indicated statistically significant within- and between-person associations between stress and negative affect. Moreover, affect reactivity significantly predicted depressive, but not anxiety, symptoms. Buffering was found for family, but not peer, support. Findings extend previous research by detecting associations between momentary affect reactivity and internalizing symptoms during a sociocultural shift in Latinx adolescents' lives and have implications for culturally appropriate programs to prevent depressive symptoms.

Model specification for nonlinearity and heterogeneity of regression in randomized pretest posttest studies: Practical solutions for missing data.

The randomized pretest posttest design is common in psychology, as is the corresponding missing data concern. Although missing data handling has seen advances over the past several decades, effective and practical solutions for handling missing data in randomized pretest posttest designs are lacking, particularly when assumptions of commonly used statistical models are violated. Although analysis of covariance can capture the average treatment effect with complete data, even when assumptions are tenuous, this becomes more difficult with missing data. This investigation fills this gap in the literature by comparing a variety of analysis models for estimating the average treatment effect under violations of linearity and homogeneity of regression slopes, when data are missing by several plausible, but understudied, missing at random patterns for randomized pretest posttest studies. Two missing data handling techniques, listwise deletion and multiple imputation, were considered. Listwise deletion provided maximum likelihood estimates (unbiased and appropriately precise) of the average treatment effect as long as the analysis model was appropriately specified to handle the violated assumption and the pretest mean was estimated using all cases. Although multiple imputation was effective as long as the imputation model was correct, the results highlight to the importance of model specification in the context of missing data. Importantly, the specific pattern of missing at random data had implications for results, emphasizing the need to consider the particular pattern of missingness beyond the general appropriateness of the missing at random assumption. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Using prior information to plan appropriately powered regression studies: A tutorial using BUCSS.

Despite increased attention to the role of statistical power in psychological studies, navigating the process of sample size planning for linear regression designs can be challenging. In particular, it can be difficult to decide upon an appropriate value for the effect size, owing to a variety of factors, including the influence of the correlations among the predictors and between the other predictors and the outcome, in addition to the correlation between the particular predictor(s) in question and the outcome, on statistical power. One approach that addresses these concerns is to use available prior sample information but adjust the sample effect size appropriately for publication bias and/or uncertainty. This article motivates a procedure that accomplishes this, Bias Uncertainty Corrected Sample Size (BUCSS), as a valid approach for linear regression, carefully illustrating how BUCSS may be used in practice. To demonstrate the relevant factors influencing BUCSS performance and ensure it performs well in plausible regression contexts, a Monte Carlo simulation is reported. Importantly, the present difficulties in sample size planning for regression are explained, followed by clear illustrations using BUCSS software for a variety of common practical scenarios in regression studies. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Misinterpreting p: The discrepancy between p values and the probability the null hypothesis is true, the influence of multiple testing, and implications for the replication crisis.

The p value is still misinterpreted as the probability that the null hypothesis is true. Even psychologists who correctly understand that p values do not provide this probability may not realize the degree to which p values differ from the probability that the null hypothesis is true. Importantly, previous research on this topic has not addressed the influence of multiple testing, often a reality in psychological studies, and has not extensively considered the influence of different prior probabilities favoring the null and alternative hypotheses. Simulation studies are presented that emphasize the magnitude by which p values are distinct from the posterior probability that the null hypothesis is true, under an extensive set of conditions including multiple testing. Particular emphasis is placed on p values just under .05, given the prevalence of these p values in the published literature, though p values in other intervals are also assessed. In diverse conditions, results indicate that posterior probabilities favoring the null hypothesis are often far removed from .05, and this pattern quickly gets much worse when multiple testing is conducted. Rather than simply telling researchers that p values do not reflect the probability favoring the null hypothesis, as has been done previously, the results presented here allow psychologists to see the evidence provided by various p values. These results have particularly topical implications for the replication crisis, for how much weight should be placed on a single study, and for how the term statistical significance should be interpreted, particularly in conditions typical in psychological research. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Life stress and major depression: The mysteries of recurrences.

Approximately half of the people who suffer a major depressive episode for the first time experience recurrences, while the other half do not. Among the initially depressed, however, who will have recurrences remains a mystery, and cannot be forecasted with any statistical or clinical confidence. It is well documented, though, that highly stressful life events commonly precede first episodes of major depression, and that these experiences become progressively less common prior to recurrences. Determining the basis for this consistent empirical observation holds promise for discovering among the initially depressed who will become recurrent, helping to solve the current-day conundrum of recurrences. The present article has 2 overarching objectives. First, we evaluate stress sensitization, the prevailing theory for explaining the decreasing association of major life events with successive recurrences. Conceptual gaps, discrepancies, and misunderstandings are found for understanding the decreasing association, as well as for understanding recurrences. Research practices and logical errors also are exposed that compromise the integrity of the existing empirical record. Second, alternative theoretical accounts are proposed to explain the decreasing association of major life stress with recurrences. Two "dual pathway models" provide viable alternative explanations, fill in existing theoretical gaps, and supply additional advantages for understanding life stress, depression, and recurrences. Recommendations are made for evaluating the 3 respective models. In closing, limitations and remaining questions are discussed for discovering who, early in the lifetime course of major depression, is likely to have a lifetime of repeated recurrences. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Best (but oft forgotten) practices: sample size planning for powerful studies.

Given recent concerns regarding replicability and trustworthiness in several areas of science, it is vital to encourage researchers to conduct statistically rigorous studies. Achieving a high level of statistical power is one particularly important domain in which researchers can improve the quality and reproducibility of their studies. Although several factors influence statistical power, appropriate sample size planning is often under the control of the researcher and can result in powerful studies. However, the process of conducting sample size planning to achieve a specified level of desired statistical power is often complex and the literature can be difficult to navigate. This article aims to provide an approachable overview of statistical power and sample size planning, with emphasis on why statistical power is important for high-quality science. Thorough examples relevant to nutrition researchers are included to illustrate the process of sample size planning. Special consideration is also given to issues that may arise when conducting sample size planning in practice. The overarching goal is to provide nutrition researchers with the tools and expertise needed to conduct effective sample size planning for future studies.