Evaluating the performance of existing and novel equivalence tests for fit indices in structural equation modelling.

It has been suggested that equivalence testing (otherwise known as negligible effect testing) should be used to evaluate model fit within structural equation modelling (SEM). In this study, we propose novel variations of equivalence tests based on the popular root mean squared error of approximation and comparative fit index fit indices. Using Monte Carlo simulations, we compare the performance of these novel tests to other existing equivalence testing-based fit indices in SEM, as well as to other methods commonly used to evaluate model fit. Results indicate that equivalence tests in SEM have good Type I error control and display considerable power for detecting well-fitting models in medium to large sample sizes. At small sample sizes, relative to traditional fit indices, equivalence tests limit the chance of supporting a poorly fitting model. We also present an illustrative example to demonstrate how equivalence tests may be incorporated in model fit reporting. Equivalence tests in SEM also have unique interpretational advantages compared to other methods of model fit evaluation. We recommend that equivalence tests be utilized in conjunction with descriptive fit indices to provide more evidence when evaluating model fit.

Negligible interaction test for continuous predictors.

Behavioral science researchers are often interested in whether there is negligible interaction among continuous predictors of an outcome variable. For example, a researcher might be interested in demonstrating that the effect of perfectionism on depression is very consistent across age. In this case, the researcher is interested in assessing whether the interaction between the predictors is too small to be meaningful. Unfortunately, most researchers address the above research question using a traditional association-based null hypothesis test (e.g. regression) where their goal is to fail to reject the null hypothesis of no interaction. Common problems with traditional tests are their sensitivity to sample size and their opposite (and hence inappropriate) hypothesis setup for finding a negligible interaction effect. In this study, we investigated a method for testing for negligible interaction between continuous predictors using unstandardized and standardized regression-based models and equivalence testing. A Monte Carlo study provides evidence for the effectiveness of the equivalence-based test relative to traditional approaches.

Denouncing the use of field-specific effect size distributions to inform magnitude.

An effect size (ES) provides valuable information regarding the magnitude of effects, with the interpretation of magnitude being the most important. Interpreting ES magnitude requires combining information from the numerical ES value and the context of the research. However, many researchers adopt popular benchmarks such as those proposed by Cohen. More recently, researchers have proposed interpreting ES magnitude relative to the distribution of observed ESs in a specific field, creating unique benchmarks for declaring effects small, medium or large. However, there is no valid rationale whatsoever for this approach. This study was carried out in two parts: (1) We identified articles that proposed the use of field-specific ES distributions to interpret magnitude (primary articles); and (2) We identified articles that cited the primary articles and classified them by year and publication type. The first type consisted of methodological papers. The second type included articles that interpreted ES magnitude using the approach proposed in the primary articles. There has been a steady increase in the number of methodological and substantial articles discussing or adopting the approach of interpreting ES magnitude by considering the distribution of observed ES in that field, even though the approach is devoid of a theoretical framework. It is hoped that this research will restrict the practice of interpreting ES magnitude relative to the distribution of ES values in a field and instead encourage researchers to interpret such by considering the specific context of the study.

The role of cognitive control and emotion regulation in predicting mental health problems in children with neurodevelopmental disorders.

BACKGROUND: Many youth with neurodevelopmental disorders (NDDs) experience mental health problems such as anxiety, depression or anger, and these are often associated with impairments of cognition and emotion regulation. The mechanisms that may be linking cognitive difficulties, emotion regulation and mental health are not known. AIMS: The current study examined whether adaptive and maladaptive (dysregulated) emotion regulation mediated the link between different cognitive control processes (working memory, inhibition and shifting) and internalizing/externalizing symptoms in children with NDDs. METHODS: Participants included 48 children (8-13 years of age) with one or more diagnoses of autism, attention deficit hyperactivity disorder, cerebral palsy and learning disability, who were enrolled in a larger study of cognitive behaviour therapy targeting emotion regulation. Multiple mediation analyses were implemented using the PROCESS macro. The mediation effects of adaptive and maladaptive emotion regulation were examined on the relationships between (1) working memory and internalizing/externalizing symptoms, (2) inhibition and internalizing/externalizing symptoms and (3) shifting and internalizing/externalizing symptoms. All data were collected prior to intervention, at baseline. RESULTS: Shifting, inhibitory control and working memory predicted increased emotion dysregulation, which functioned as a full mediator to both internalizing and externalizing problems in children with NDDs. CONCLUSIONS: In the presence of emotionally triggering situations, children with greater cognitive challenges experience greater maladaptive emotion regulation, which results in both internalizing and externalizing problems. For youth with NDDs, therapeutic plans that include strengthening of working memory, inhibition and shifting abilities in addition to emotion regulation skills training may be helpful in alleviating externalizing and internalizing behaviour.

Fear of movement in children and adolescents undergoing major surgery: A psychometric evaluation of the Tampa Scale for Kinesiophobia.

BACKGROUND: The objective of this study was to evaluate the psychometric properties of the 17-item Tampa Scale for Kinesiophobia (TSK) in youth. METHODS: Participants were 264 children and adolescents (58.7% female, Mage  = 14.1 years, SDage  = 2.51) scheduled for major surgery who were assessed before surgery, while in hospital postoperatively, and at 6 and 12 months after surgery. Exploratory factor analyses (EFA) were conducted to determine the factor structure of pre-operative TSK scores. Reliability, and convergent, discriminant, and predictive validity were examined. RESULTS: EFA on the 17-item TSK revealed a two-factor model distinguishing the 13 positively scored items from the 4 reverse scored items, but the fit was poor. A second EFA was conducted on the 13 positively scored items (TSK-13) revealing a three-factor model: Fear of injury, bodily vulnerability, and activity avoidance. The TSK-13 showed adequate internal consistency (Ω = 0.82) and weak convergent validity. The TSK-13 was not correlated with postoperative, in-hospital physical activity (actigraphy; r (179) = -0.10, p = 0.18) and showed adequate discriminant validity, that is correlations less than 0.70, with measures of depression (r (225) = 0.41, p < 0.001) and general anxiety (r (224)=0.35, p < 0.001). Predictive validity for pain-related disability at 12 months (r (70) = 0.34, p < 0.001) was adequate. CONCLUSIONS: The original TSK-17 does not appear to be a meaningful measure of kinesiophobia in youth after surgery possibly because of the syntactic structure of the reverse scored items. In contrast, a modified TSK-13, comprised of only the positively scored items, revealed a 3-factor structure that is reliable and demonstrates adequate convergent, discriminant, and predictive validity. SIGNIFICANCE: Kinesiophobia is an important construct to evaluate in the transition from acute to chronic pain among children and adolescents. The 17 item Tampa Scale for Kinesiophobia (TSK) does not show adequate validity or reliability in youth undergoing major surgery, however, the psychometric properties of a 13-item modified scale (TSK-13) are promising.

Clinical and parental predictors of emotion regulation following cognitive behaviour therapy in children with autism.

LAY ABSTRACT: Children with autism commonly experience difficulty controlling their emotions. Although existing treatments are successful in teaching critical emotion regulation skills, not all children improve. It is important to identify the factors that influence treatment response to be able to reach more children. This study aimed to identify child and parent characteristics that predict treatment response in a 10-week cognitive behaviour therapy treatment for children with autism, 8-12 years of age, and their parents. We found that youth who started the treatment with higher verbal abilities, who were more anxious in social situations, and had parents who were more anxious, were more likely to improve in learning new emotion regulation skills. We also found that children who had more physical discomforts or complaints before starting the treatment were less likely to improve in their negative expressions of emotion. Our study suggests that it is important for clinicians to promote active involvement and learning by avoiding complex language and to use more visual materials to supplement the learning process, and make sure that sessions are sensitive to the individual needs of participants.

The role of alexithymia in trauma therapy outcomes: Examining improvements in PTSD, dissociation, and interpersonal problems.

OBJECTIVE: Alexithymia is a personality trait that reflects deficits in the cognitive processing and regulation of emotions (Taylor & Bagby, 2013). It has been closely linked to childhood trauma and reported by individuals presenting with other trauma-related conditions, such as posttraumatic stress disorder (PTSD), dissociation, and interpersonal problems (Powers, Etkin, Gyurak, Bradley, & Jovanovic, 2015). Addressing the emotional deficits associated with alexithymia is fundamental to resolving issues of childhood trauma and, therefore, is at the core of many trauma therapy models (e.g., Cloitre, Koenen, Cohen, & Han, 2002). The current study aims to build upon this foundation by examining the role of alexithymia in the improvements of trauma-specific difficulties prior to and following trauma therapy among treatment-seeking women with histories of childhood abuse. METHOD: Data were collected from 167 participants attending Women Recovering from Abuse Program (WRAP), an 8-week, Stage I, day treatment program using primarily group therapy for women with histories of severe childhood trauma. Participants' level of alexithymia, PTSD, and dissociative symptoms, and interpersonal difficulties were assessed at three time points. RESULTS: Significant positive relationships were found between improvements in alexithymia and improvements on all trauma-specific outcomes over the course of treatment (e.g., baseline to posttreatment) and between distinct stages of WRAP. CONCLUSIONS: These findings underscore the role of alexithymia in trauma therapy, and the need to properly attend to the deficits and issues related to alexithymia at initial stages of therapy with survivors of childhood abuse in order to facilitate improvements in trauma-specific symptoms. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Evaluating Equivalence Testing Methods for Measurement Invariance.

Measurement Invariance (MI) is often concluded from a nonsignificant chi-square difference test. Researchers have also proposed using change in goodness-of-fit indices ([Formula: see text]GOFs) instead. Both of these commonly used methods for testing MI have important limitations. To combat these issues, To combat these issues, it was proposed using an equivalence test (EQ) to replace the chi-square difference test commonly used to test MI. Due to concerns with the EQ's power, and adjusted version (EQ-A) was created, but provides little evaluation of either procedure. The current study evaluated the Type I error and power of both the EQ and EQ-A, and compared their performance to that of the traditional chi-square difference test and [Formula: see text]GOFs. The EQ was the only procedure that maintained empirical error rates below the nominal alpha level. Results also highlight that the EQ requires larger sample sizes than traditional difference-based approaches or using equivalence bounds based on larger than conventional RMSEA values (e.g., > .05) to ensure adequate power rates. We do not recommend the proposed adjustment (EQ-A) over the EQ.

Health Care Administrators' Cardiac Rehabilitation Attitudes (HACRA) in North and South America and the Development of a Scale to Assess Them.

BACKGROUND: Cardiac rehabilitation (CR) is insufficiently available; even less so in low-resource settings. Health care administrators (HAs) are responsible for ensuring CR programs are offered and resourced. This study compared HA CR attitudes in North and South America, the contextual factors associated with these attitudes, and developed a scale/survey to assess them with global applicability. METHODS: Data were retrospectively analysed from three cross-sectional studies: in (1) 195 HAs from Canada (i.e., North America), (2) 44 HAs from seven South American countries, and (3) 43 HAs from Brazil (all South America). Contextual and perceptual/attitudinal items (five-point Likert scale, with higher scores indicating more positive attitudes) were compared by region. Psychometric properties of items were tested. A literature review was performed, and items generated for content validation by experts. RESULTS: The most negative CR attitudes related to government funding models (mean=2.46±1.14/5), and that patients have responsibility for their disease management (3.03±1.25). North American HAs rated their institutions' perceptions of the importance of CR (p<0.001) and quality of the local program (p<0.001) higher than their South American counterparts, but rated the utility of CR in lowering length of stay (p<0.001), promoting behaviour change (p<0.05) and the need for more government funding (p<0.05) lower, among others. Total attitude scores were positively associated with CR knowledge/awareness in both cohorts (all p<0.01). North American HAs more often perceived CR should be funded by hospitals (p<0.001) than South Americans. A 39-item HA CR Attitudes (HACRA-R) scale was developed. CONCLUSIONS: Health care administrators' perceptions vary by context, which could impact CR resourcing.

Exploring perceptions of meaningfulness in visual representations of bivariate relationships.

Researchers often need to consider the practical significance of a relationship. For example, interpreting the magnitude of an effect size or establishing bounds in equivalence testing requires knowledge of the meaningfulness of a relationship. However, there has been little research exploring the degree of relationship among variables (e.g., correlation, mean difference) necessary for an association to be interpreted as meaningful or practically significant. In this study, we presented statistically trained and untrained participants with a collection of figures that displayed varying degrees of mean difference between groups or correlations among variables and participants indicated whether or not each relationship was meaningful. The results suggest that statistically trained and untrained participants differ in their qualification of a meaningful relationship, and that there is significant variability in how large a relationship must be before it is labeled meaningful. The results also shed some light on what degree of relationship is considered meaningful by individuals in a context-free setting.

Psychosocial well-being over the two years following cardiac rehabilitation initiation & association with heart-health behaviors.

OBJECTIVE: To track psychosocial well-being over 2 years following cardiac rehabilitation (CR) initiation, and its' association with heart-health behaviors. METHODS: Patients from 3 CR programs were approached at their first visit, and consenters completed a survey. Participants were emailed surveys again 6 months, 1 and 2 years later. Depressive symptoms (PHQ-8) and quality of life were assessed at each point, as were exercise, nutrition, smoking and medication adherence, among other well-being indicators. RESULTS: Of 411 participants, 46.7% were retained at 2 years. Post-CR, there was 70% concordance between participants' desired and actual work status. Depressive symptoms were consistently minimal over time (mean = 3.17 ±â€¯0.37); Quality of life was high, and increased over time (p = .01). At 2 years, 56.9% participants met exercise recommendations, and 5.4% smoked. With adjustment, greater self-regulation was associated with significantly greater exercise at intake; greater exercise self-efficacy was significantly associated with greater exercise at 1 year; greater disease management self-efficacy was significantly associated with greater exercise at 2 years; greater environmental mastery (actual) was significantly associated with greater exercise at 2 years. Lower depressive symptoms were significantly associated with better nutrition at 2 years. CONCLUSION: CR initiators are thriving, and this relates to better exercise and diet.

ANOVA and the variance homogeneity assumption: Exploring a better gatekeeper.

Valid use of the traditional independent samples ANOVA procedure requires that the population variances are equal. Previous research has investigated whether variance homogeneity tests, such as Levene's test, are satisfactory as gatekeepers for identifying when to use or not to use the ANOVA procedure. This research focuses on a novel homogeneity of variance test that incorporates an equivalence testing approach. Instead of testing the null hypothesis that the variances are equal against an alternative hypothesis that the variances are not equal, the equivalence-based test evaluates the null hypothesis that the difference in the variances falls outside or on the border of a predetermined interval against an alternative hypothesis that the difference in the variances falls within the predetermined interval. Thus, with the equivalence-based procedure, the alternative hypothesis is aligned with the research hypothesis (variance equality). A simulation study demonstrated that the equivalence-based test of population variance homogeneity is a better gatekeeper for the ANOVA than traditional homogeneity of variance tests.

Preschool children's coping responses and outcomes in the vaccination context: child and caregiver transactional and longitudinal relationships.

This article, based on 2 companion studies, presents an in-depth analysis of preschoolers coping with vaccination pain. Study 1 used an autoregressive cross-lagged path model to investigate the dynamic and reciprocal relationships between young children's coping responses (how they cope with pain and distress) and coping outcomes (pain behaviors) at the preschool vaccination. Expanding on this analysis, study 2 then modeled preschool coping responses and outcomes using both caregiver and child variables from the child's 12-month vaccination (n = 548), preschool vaccination (n = 302), and a preschool psychological assessment (n = 172). Summarizing over the 5 path models and post hoc analyses over the 2 studies, novel transactional and longitudinal pathways predicting preschooler coping responses and outcomes were elucidated. Our research has provided empirical support for the need to differentiate between coping responses and coping outcomes: 2 different, yet interrelated, components of "coping." Among our key findings, the results suggest that a preschooler's ability to cope is a powerful tool to reduce pain-related distress but must be maintained throughout the appointment; caregiver behavior and poorer pain regulation from the 12-month vaccination appointment predicted forward to preschool coping responses and/or outcomes; robust concurrent relationships exist between caregiver behaviors and both child coping responses and outcomes, and finally, caregiver behaviors during vaccinations are not only critical to both child pain coping responses and outcomes in the short- and long-term but also show relationships to broader child cognitive abilities as well.

Testing for negligible interaction: A coherent and robust approach.

Researchers often want to demonstrate a lack of interaction between two categorical predictors on an outcome. To justify a lack of interaction, researchers typically accept the null hypothesis of no interaction from a conventional analysis of variance (ANOVA). This method is inappropriate as failure to reject the null hypothesis does not provide statistical evidence to support a lack of interaction. This study proposes a bootstrap-based intersection-union test for negligible interaction that provides coherent decisions between the omnibus test and post hoc interaction contrast tests and is robust to violations of the normality and variance homogeneity assumptions. Further, a multiple comparison strategy for testing interaction contrasts following a non-significant omnibus test is proposed. Our simulation study compared the Type I error control, omnibus power and per-contrast power of the proposed approach to the non-centrality-based negligible interaction test of Cheng and Shao (2007, Statistica Sinica, 17, 1441). For 2 × 2 designs, the empirical Type I error rates of the Cheng and Shao test were very close to the nominal α level when the normality and variance homogeneity assumptions were satisfied; however, only our proposed bootstrapping approach was satisfactory under non-normality and/or variance heterogeneity. In general a × b designs, although the omnibus Cheng and Shao test, as expected, is the most powerful, it is not robust to assumption violation and results in incoherent omnibus and interaction contrast decisions that are not possible with the intersection-union approach.

The mental health status of ethnocultural minorities in Ontario and their mental health care.

BACKGROUND: Mental disorders are a leading cause of disability and early mortality. The objective of this study was to describe and compare psychosocial indicators and mental health service use among ethnoculturally-diverse Ontarians. METHODS: This is a cross-sectional analysis of the Ontario Health Study pilot investigation. Residents were mailed an invitation to one of 3 assessment centres (urban, rural and northern sites) from March 2009 to July 2010. Participants had an interview with a nurse and completed a questionnaire on a touchscreen kiosk. The questionnaire included sociodemographic items, and scales assessing symptoms of depressive symptoms (CES-D) and anxiety (GAD-7), social support (Lubben Social Network Scale), stressful life events, and mental health service use. RESULTS: Eight thousand two hundred thirty-five residents participated, among whom 6652 (82.4 %) self-reported their ethnocultural background as White, 225 (2.8 %) as South Asian, 222 (2.8 %) East Asian, 214 (2.7 %) Southeast Asian, 197 (2.4 %) Black, and 28 (0.3 %) as Aboriginal. Based on their sociodemographic characteristics, participants from these ethnocultural minority groups were matched to White participants. Black participants reported significantly greater stressful life events than White participants (p = .04), particularly death (p < .05), divorce (p = .002) and financial difficulties (p < .001). East Asian participants reported significantly less social support than their White counterparts (p < .001), and this was not confounded by measurement variance. Mental health service use was significantly lower in all ethnocultural minorities except Aboriginals, when compared to White participants (p = .001). CONCLUSIONS: There is a high burden of psychosocial distress in several preponderant ethnocultural minorities in Ontario; many of whom are not accessing available mental health services.

Increasing Literacy in Quantitative Methods: The Key to the Future of Canadian Psychology.

Quantitative methods (QM) dominate empirical research in psychology. Unfortunately most researchers in psychology receive inadequate training in QM. This creates a challenge for researchers who require advanced statistical methods to appropriately analyze their data. Many of the recent concerns about research quality, replicability, and reporting practices are directly tied to the problematic use of QM. As such, improving quantitative literacy in psychology is an important step towards eliminating these concerns. The current paper will include two main sections that discuss quantitative challenges and opportunities. The first section discusses training and resources for students and presents descriptive results on the number of quantitative courses required and available to graduate students in Canadian psychology departments. In the second section, we discuss ways of improving quantitative literacy for faculty, researchers, and clinicians. This includes a strong focus on the importance of collaboration. The paper concludes with practical recommendations for improving quantitative skills and literacy for students and researchers in Canada.

The relationships between therapeutic alliance and internalizing and externalizing symptoms in Trauma-Focused Cognitive Behavioral Therapy.

Therapeutic alliance has been considered an important factor in child psychotherapy and is consistently associated with positive outcomes. Nevertheless, research on alliance in the context of child trauma therapy is very scarce. This study examined the relationships between child therapeutic alliance and psychopathology in an empirically supported child trauma therapy model designed to address issues related to trauma with children and their caregivers. Specifically, we examined the extent to which the child's psychopathology would predict the establishment of a positive alliance early in treatment, as well as the association between alliance and outcome. Participants were 95 children between the ages of 7 and 12 and their caregivers, who went through a community-based Trauma-Focused Cognitive Behavioral Therapy program in Canada. Caregivers filled out the CBCL prior to assessment and following treatment. Children and therapists completed an alliance measure (TASC) at three time points throughout treatment. Symptomatology and child gender emerged as important factors predicting alliance at the beginning of treatment. Girls and internalizing children developed stronger alliances early in treatment. In addition, a strong early alliance emerged as a significant predictor of improvement in internalizing symptoms at the end of treatment. Our findings indicate that symptomatology and gender influence the development of a strong alliance in trauma therapy. We suggest that clinicians should adjust therapeutic style to better engage boys and highly externalizing children in the early stages of therapy.

Equivalence tests for comparing correlation and regression coefficients.

Equivalence tests are an alternative to traditional difference-based tests for demonstrating a lack of association between two variables. While there are several recent studies investigating equivalence tests for comparing means, little research has been conducted on equivalence methods for evaluating the equivalence or similarity of two correlation coefficients or two regression coefficients. The current project proposes novel tests for evaluating the equivalence of two regression or correlation coefficients derived from the two one-sided tests (TOST) method (Schuirmann, 1987, J. Pharmacokinet. Biopharm, 15, 657) and an equivalence test by Anderson and Hauck (1983, Stat. Commun., 12, 2663). A simulation study was used to evaluate the performance of these tests and compare them with the common, yet inappropriate, method of assessing equivalence using non-rejection of the null hypothesis in difference-based tests. Results demonstrate that equivalence tests have more accurate probabilities of declaring equivalence than difference-based tests. However, equivalence tests require large sample sizes to ensure adequate power. We recommend the Anderson-Hauck equivalence test over the TOST method for comparing correlation or regression coefficients.

Assessing clinical significance using robust normative comparisons.

OBJECTIVE: Clinical significance determines whether an intervention makes a real difference in the everyday life of a client. One of the most recommended approaches for conducting group-level analyses of clinical significance is to evaluate whether the treated clinical group is equivalent to a normal comparison group (normative comparisons). The purpose of this study was to demonstrate the analytical and practical power of assessing clinical significance using normative comparisons that are robust to violations of normality and homogeneity of variance assumptions. METHOD: Six datasets were gleaned from published intervention studies for depression. RESULTS: We found that normative comparisons using a robust Schuirmann-Yuen test determined equivalency for 11% fewer clinical samples compared to original normative comparisons that use a Schuirmann test of equivalence. CONCLUSIONS: We recommend that researchers conducting normative comparisons utilize the Schuirmann-Yuen procedure as it provides the most reliable method available for determining if a treated clinical group is equivalent to a normative comparison group.

Evaluating clinical significance: incorporating robust statistics with normative comparison tests.

The purpose of this study was to evaluate a modified test of equivalence for conducting normative comparisons when distribution shapes are non-normal and variances are unequal. A Monte Carlo study was used to compare the empirical Type I error rates and power of the proposed Schuirmann-Yuen test of equivalence, which utilizes trimmed means, with that of the previously recommended Schuirmann and Schuirmann-Welch tests of equivalence when the assumptions of normality and variance homogeneity are satisfied, as well as when they are not satisfied. The empirical Type I error rates of the Schuirmann-Yuen were much closer to the nominal α level than those of the Schuirmann or Schuirmann-Welch tests, and the power of the Schuirmann-Yuen was substantially greater than that of the Schuirmann or Schuirmann-Welch tests when distributions were skewed or outliers were present. The Schuirmann-Yuen test is recommended for assessing clinical significance with normative comparisons.