Measurement invariance in the social sciences: Historical development, methodological challenges, state of the art, and future perspectives.

This review summarizes the current state of the art of statistical and (survey) methodological research on measurement (non)invariance, which is considered a core challenge for the comparative social sciences. After outlining the historical roots, conceptual details, and standard procedures for measurement invariance testing, the paper focuses in particular on the statistical developments that have been achieved in the last 10 years. These include Bayesian approximate measurement invariance, the alignment method, measurement invariance testing within the multilevel modeling framework, mixture multigroup factor analysis, the measurement invariance explorer, and the response shift-true change decomposition approach. Furthermore, the contribution of survey methodological research to the construction of invariant measurement instruments is explicitly addressed and highlighted, including the issues of design decisions, pretesting, scale adoption, and translation. The paper ends with an outlook on future research perspectives.

Individual recovery of health-related quality of life during 18 months post-burn using a retrospective pre-burn measurement: an exploratory study.

PURPOSE: This study explored the individual trajectories of health-related quality of life (HRQL) compared to recalled pre-burn level of HRQL and investigated whether burn severity and post-traumatic stress disorder (PTSD) symptoms increase the risk of not returning to pre-burn level of HRQL. METHODS: Data were obtained from 309 adult patients with burns in a multicenter study. Patients completed the EQ-5D-3L questionnaire with a Cognition bolt-on shortly after hospital admission, which included a recalled pre-injury measure, and, again, at 3, 6, 12 and 18 months post-burn. Burn severity was indicated by the number of surgeries, and PTSD symptoms were assessed with the IES-R at three months post-burn. Pre- and post-injury HRQL were compared to norm populations. RESULTS: Recalled pre-injury HRQL was higher than population norms and HRQL at 18 months post-burn was comparable to population norms. Compared to the pre-injury level of functioning, four HRQL patterns of change over time were established: Stable, Recovery, Deterioration, and Growth. In each HRQL domain, a subset of patients did not return to their recalled pre-injury levels, especially with regard to Pain, Anxiety/Depression, and Cognition. Patients with more severe burns or PTSD symptoms were less likely to return to pre-injury level of functioning within 18 months post-burn. CONCLUSION: This study identified four patterns of individual change. Patients with more severe injuries and PTSD symptoms were more at risk of not returning to their recalled pre-injury HRQL. This study supports the face validity of using a recalled pre-burn HRQL score as a reference point to monitor HRQL after burns.

On Imbalance of Impulse Control and Sensation Seeking and Adolescent Risk: An Intra-individual Developmental Test of the Dual Systems and Maturational Imbalance Models.

Heterogeneity in development of imbalance between impulse control and sensation seeking has not been studied until now. The present study scrutinized this heterogeneity and the link between imbalance and adolescent risk. Seven-wave data of 7,558 youth (50.71% males; age range from 12/13 until 24/25) were used. Three developmental trajectories were identified. The first trajectory, "sensation seeking to balanced sensation seeking", included participants with a higher level of sensation seeking than impulse control across all ages. The second trajectory, "moderate dominant control", included participants showing moderate and increasing impulse control relative to sensation seeking across all ages. The third trajectory, "strong late dominant control", included participants showing the highest level of impulse control which was about as strong as sensation seeking from early to middle adolescence and became substantially stronger from late adolescence to early adulthood. Although the systematic increase of impulse control in all subgroups is in line with both models, neither of these combined trajectories of control and sensation seeking was predicted by the Dual Systems Model or the Maturational Imbalance Model. Consistent with both models the "sensation seeking to balanced sensation seeking" trajectory showed the highest level of substance use. It can be concluded that, even though both theories adequately predict the link between imbalance and risk, neither the Dual Systems Model nor the Maturational Imbalance Model correctly predict the heterogeneity in development of imbalance between impulse control and sensation seeking.

Quantifying the informational value of classification images.

Reverse correlation is an influential psychophysical paradigm that uses a participant's responses to randomly varying images to build a classification image (CI), which is commonly interpreted as a visualization of the participant's mental representation. It is unclear, however, how to statistically quantify the amount of signal present in CIs, which limits the interpretability of these images. In this article, we propose a novel metric, infoVal, which assesses informational value relative to a resampled random distribution and can be interpreted like a z score. In the first part, we define the infoVal metric and show, through simulations, that it adheres to typical Type I error rates under various task conditions (internal validity). In the second part, we show that the metric correlates with markers of data quality in empirical reverse-correlation data, such as the subjective recognizability, objective discriminability, and test-retest reliability of the CIs (convergent validity). In the final part, we demonstrate how the infoVal metric can be used to compare the informational value of reverse-correlation datasets, by comparing data acquired online with data acquired in a controlled lab environment. We recommend a new standard of good practice in which researchers assess the infoVal scores of reverse-correlation data in order to ensure that they do not read signal in CIs where no signal is present. The infoVal metric is implemented in the open-source rcicr R package, to facilitate its adoption.

How the Choice of Distance Measure Influences the Detection of Prior-Data Conflict.

The present paper contrasts two related criteria for the evaluation of prior-data conflict: the Data Agreement Criterion (DAC; Bousquet, 2008) and the criterion of Nott et al. (2016). One aspect that these criteria have in common is that they depend on a distance measure, of which dozens are available, but so far, only the Kullback-Leibler has been used. We describe and compare both criteria to determine whether a different choice of distance measure might impact the results. By means of a simulation study, we investigate how the choice of a specific distance measure influences the detection of prior-data conflict. The DAC seems more susceptible to the choice of distance measure, while the criterion of Nott et al. seems to lead to reasonably comparable conclusions of prior-data conflict, regardless of the distance measure choice. We conclude with some practical suggestions for the user of the DAC and the criterion of Nott et al.

Correction: Veen, D.; Stoel, D.; Schalken, N.; Mulder, K.; Van de Schoot, R. Using the Data Agreement Criterion to Rank Experts' Beliefs. Entropy 2018, 20, 592.

Due to a coding error the marginal likelihoods have not been correctly calculated for the empirical example and thus the Bayes Factors following from these marginal likelihoods are incorrect. The corrections required occur in Section 3.2 and in two paragraphs of the discussion in which the results are referred to. The corrections have limited consequences for the paper and the main conclusions hold. Additionally typos in Equations, and, an error in the numbering of the Equations are remedied.

Bayesian PTSD-Trajectory Analysis with Informed Priors Based on a Systematic Literature Search and Expert Elicitation.

There is a recent increase in interest of Bayesian analysis. However, little effort has been made thus far to directly incorporate background knowledge via the prior distribution into the analyses. This process might be especially useful in the context of latent growth mixture modeling when one or more of the latent groups are expected to be relatively small due to what we refer to as limited data. We argue that the use of Bayesian statistics has great advantages in limited data situations, but only if background knowledge can be incorporated into the analysis via prior distributions. We highlight these advantages through a data set including patients with burn injuries and analyze trajectories of posttraumatic stress symptoms using the Bayesian framework following the steps of the WAMBS-checklist. In the included example, we illustrate how to obtain background information using previous literature based on a systematic literature search and by using expert knowledge. Finally, we show how to translate this knowledge into prior distributions and we illustrate the importance of conducting a prior sensitivity analysis. Although our example is from the trauma field, the techniques we illustrate can be applied to any field.

Using the Data Agreement Criterion to Rank Experts' Beliefs.

Experts' beliefs embody a present state of knowledge. It is desirable to take this knowledge into account when making decisions. However, ranking experts based on the merit of their beliefs is a difficult task. In this paper, we show how experts can be ranked based on their knowledge and their level of (un)certainty. By letting experts specify their knowledge in the form of a probability distribution, we can assess how accurately they can predict new data, and how appropriate their level of (un)certainty is. The expert's specified probability distribution can be seen as a prior in a Bayesian statistical setting. We evaluate these priors by extending an existing prior-data (dis)agreement measure, the Data Agreement Criterion, and compare this approach to using Bayes factors to assess prior specification. We compare experts with each other and the data to evaluate their appropriateness. Using this method, new research questions can be asked and answered, for instance: Which expert predicts the new data best? Is there agreement between my experts and the data? Which experts' representation is more valid or useful? Can we reach convergence between expert judgement and data? We provided an empirical example ranking (regional) directors of a large financial institution based on their predictions of turnover.

Effectiveness of Two Web-Based Interventions for Chronic Cancer-Related Fatigue Compared to an Active Control Condition: Results of the "Fitter na kanker" Randomized Controlled Trial.

BACKGROUND: Approximately one third of all patients who have been successfully treated for cancer suffer from chronic cancer-related fatigue (CCRF). Effective and easily accessible interventions are needed for these patients. OBJECTIVE: The current paper reports on the results of a 3-armed randomized controlled trial investigating the clinical effectiveness of two different guided Web-based interventions for reducing CCRF compared to an active control condition. METHODS: Severely fatigued cancer survivors were recruited via online and offline channels, and self-registered on an open-access website. After eligibility checks, 167 participants were randomized via an embedded automated randomization function into: (1) physiotherapist-guided Ambulant Activity Feedback (AAF) therapy encompassing the use of an accelerometer (n=62); (2) psychologist-guided Web-based mindfulness-based cognitive therapy (eMBCT; n=55); or (3) an unguided active control condition receiving psycho-educational emails (n=50). All interventions lasted nine weeks. Fatigue severity was self-assessed using the Checklist Individual Strength - Fatigue Severity subscale (primary outcome) six times from baseline (T0b) to six months (T2). Mental health was self-assessed three times using the Hospital Anxiety and Depression Scale and Positive and Negative Affect Schedule (secondary outcome). Treatment dropout was investigated. RESULTS: Multiple group latent growth curve analysis, corrected for individual time between assessments, showed that fatigue severity decreased significantly more in the AAF and eMBCT groups compared to the psycho-educational group. The analyses were checked by a researcher who was blind to allocation. Clinically relevant changes in fatigue severity were observed in 66% (41/62) of patients in AAF, 49% (27/55) of patients in eMBCT, and 12% (6/50) of patients in psycho-education. Dropout was 18% (11/62) in AAF, mainly due to technical problems and poor usability of the accelerometer, and 38% (21/55) in eMBCT, mainly due to the perceived high intensity of the program. CONCLUSIONS: Both the AAF and eMBCT interventions are effective for managing fatigue severity compared to receiving psycho-educational emails. TRIAL REGISTRATION: Trialregister.nl NTR3483; http://www.trialregister.nl/trialreg/admin/rctview.asp?TC=3483 (Archived by WebCite at http://www.webcitation.org/6NWZqon3o).

Eye movement desensitisation and reprocessing therapy v. stabilisation as usual for refugees: randomised controlled trial.

BACKGROUND: Eye movement desensitisation and reprocessing (EMDR) therapy is a first-line treatment for adults with post-traumatic stress disorder (PTSD). Some clinicians argue that with refugees, directly targeting traumatic memories through EMDR may be harmful or ineffective. AIMS: To determine the safety and efficacy of EMDR in adult refugees with PTSD (trial registration: ISRCTN20310201). METHOD: In total, 72 refugees referred for specialised treatment were randomly assigned to 12 h of EMDR (3×60 min planning/preparation followed by 6×90 min desensitisation/reprocessing) or 12 h (12×60 min) of stabilisation. The Clinician-Administered PTSD Scale (CAPS) and Harvard Trauma Questionnaire (HTQ) were primary outcome measures. RESULTS: Intention-to-treat analyses found no differences in safety (one severe adverse event in the stabilisation condition only) or efficacy (effect sizes: CAPS -0.04 and HTQ 0.20) between the two conditions. CONCLUSIONS: Directly targeting traumatic memories through 12 h of EMDR in refugee patients needing specialised treatment is safe, but is only of limited efficacy.

PTSD Symptom Trajectories in Disaster Volunteers: The Role of Self-Efficacy, Social Acknowledgement, and Tasks Carried Out.

Millions of volunteers respond after disasters, with a 24% to 46% risk of developing posttraumatic stress disorder (PTSD). It is unclear which symptom trajectories develop and how they differ between core (volunteering before the disaster) and noncore volunteers (joining after the disaster) and which factors predict trajectories. Symptoms of PTSD were assessed at 6-, 12-, and 18-months postearthquake in 449 volunteers in Indonesia. Demographics, previous mental health service use, self-efficacy, social acknowledgment, and type of tasks were assessed at 6 months. In both core and noncore volunteers, 2 PTSD symptom trajectories emerged: a resilient trajectory (moderate levels of symptoms with a slow decrease over time; 90.9%) and a chronic trajectory (higher levels of symptoms with an increase over time; 9.1%). In both trajectories, core volunteers had fewer symptoms than noncore volunteers. Core volunteers in the chronic trajectory were characterized by having sought prior mental help, reported lower levels of self-efficacy and social acknowledgment, and were more likely to have provided psychosocial support to beneficiaries (Cramér's V = .17 to .27, partial η(2) = .02 to .06). Aid organizations should identify and follow up chronic PTSD trajectories in volunteers, including the noncore, who may be out of sight to the organization after the acute response phase.

Child and adolescent internalizing and externalizing problems 12 months postburn: the potential role of preburn functioning, parental posttraumatic stress, and informant bias.

Adjustment after pediatric burn injury may be a challenge for children as well as their parents. This prospective study examined associations of internalizing and externalizing problems in children and adolescents 12 months postburn with preburn functioning, and parental acute and chronic posttraumatic stress symptoms (PTSS) from different perspectives. Child, mother, and father reports of 90 children (9-18 years), collected within the first month and 12 months postburn, were analyzed. Results indicated that overall, child and parental appraisals of pre- and postburn behavioral problems were not significantly different from reference data. Rates of (sub)clinical postburn behavioral problems ranged from 6 to 17 %, depending on the informant. Pre- and postburn behavioral problems were significantly related, but only from the parents' perspective. Path models showed an association between parental PTSS 12 months postburn and parental reports of child internalizing problems, as well as a significant indirect relationship from parental acute stress symptoms via PTSS 12 months postburn. Notably, no associations between parental PTSS and child reports of postburn behavioral problems were found. In conclusion, parental observations of child externalizing problems appear to be influenced by their perspectives on the child's preburn functioning, while parental observations of internalizing problems are also related to long-term parental PTSS. However, these factors seem of no great value in predicting behavioral problems from the child's perspective, suggesting substantial informant deviations. To optimize adjustment, clinical burn practice is recommended to adopt a family perspective including parent perception of preburn functioning and parental PTSS in assessment and intervention.

Direct Aggression and Generalized Anxiety in Adolescence: Heterogeneity in Development and Intra-Individual Change.

Co-occurrence of aggression and anxiety might change during adolescence, or stay stable. We studied change and stability of four types of co-occurrence regarding direct aggression and anxiety in adolescence: an anxious and non-aggressive type, an aggressive and non-anxious type, a comorbid aggressive-anxious type and a no problems type. We applied a person-centered approach to assess increases and decreases of these types, and tested various models of intra-individual change of the types: the stability, acting out and failure models. We used data from a five-wave study of 923 early-to-middle and 390 middle-to-late adolescents (48.5 % male), thereby covering the ages of 12-20. We observed accelerated development in the older cohort: adolescents tended to grow faster out of the aggressive types in middle-to-late adolescence than in early-to-middle adolescence. We observed one other group-dependent pattern of heterogeneity in development, namely "gender differentiation": gender differences in aggression and generalized anxiety became stronger over time. We found support for two perspectives on intra-individual change of the four types, namely the stability and the acting out perspective. The no problems--and to a lesser extent the anxious--type proved to be stable across time. Acting out was found in early-to-middle adolescents, males, and adolescents with poorer-quality friendships. In all three groups, there were substantial transitions from the anxious type to the aggressive type during 4 years (between 20 and 41 %). Remarkably, acting out was most prevalent in subgroups that, generally speaking, are more vulnerable for aggressive behavior, namely early-to-middle adolescents and males. We interpret acting out as the attempt of adolescents to switch from anxiety to instrumental aggression, in order to become more visible and obtain an autonomous position in the adolescent world. Acting out contributed to the explanation of accelerated development and gender differentiation. We also observed an increase of adolescents with no problems. These findings highlight that the co-occurrence of aggression and anxiety changes considerably during adolescence, but also that the anxious and no problems types are quite stable in this period.

Measurement invariance of the Illness Invalidation Inventory (3*I) across language, rheumatic disease and gender.

OBJECTIVES: The Illness Invalidation Inventory (3*I) assesses patients' perception of responses of others that are perceived as denying, lecturing, not supporting and not acknowledging the condition of the patient. It includes two factors: 'discounting' and 'lack of understanding'. In order to use the 3*I to compare and pool scores across groups and countries, the questionnaire must have measurement invariance; that is, it should measure identical concepts with the same factor structure across groups. The aim of this study was to examine measurement invariance of the 3*I across rheumatic diseases, gender and languages. METHODS: Participants with rheumatic disease from various countries completed an online study using the 3*I, which was presented in Dutch, English, French, German, Portuguese and Spanish; 6057 people with rheumatic diseases participated. Single and multiple group confirmatory factor analyses were used to test the factorial structure and measurement invariance of the 3*I with Mplus. RESULTS: The model with strong measurement invariance, that is, equal factor loadings and thresholds (distribution cut-points) across gender and rheumatic disease (fibromyalgia vs other rheumatic diseases) had the best fit estimates for the Dutch version, and good fit estimates across the six language versions. CONCLUSIONS: The 3*I showed measurement invariance across gender, rheumatic disease and language. Therefore, it is appropriate to compare and pool scores of the 3*I across groups. Future research may use the questionnaire to examine antecedents and consequences of invalidation as well as the effect of treatments targeting invalidation.

A gentle introduction to bayesian analysis: applications to developmental research.

Bayesian statistical methods are becoming ever more popular in applied and fundamental research. In this study a gentle introduction to Bayesian analysis is provided. It is shown under what circumstances it is attractive to use Bayesian estimation, and how to interpret properly the results. First, the ingredients underlying Bayesian methods are introduced using a simplified example. Thereafter, the advantages and pitfalls of the specification of prior knowledge are discussed. To illustrate Bayesian methods explained in this study, in a second example a series of studies that examine the theoretical framework of dynamic interactionism are considered. In the Discussion the advantages and disadvantages of using Bayesian statistics are reviewed, and guidelines on how to report on Bayesian statistics are provided.

Anger: cause or consequence of posttraumatic stress? A prospective study of Dutch soldiers.

Many studies have shown that individuals with posttraumatic stress disorder (PTSD) experience more anger over time and across situations (i.e., trait anger) than trauma-exposed individuals without PTSD. There is a lack of prospective research, however, that considers anger levels before trauma exposure. The aim of this study was to prospectively assess the relationship between trait anger and PTSD symptoms, with several known risk factors, including baseline symptoms, neuroticism, and stressor severity in the model. Participants were 249 Dutch soldiers tested approximately 2 months before and approximately 2 months and 9 months after their deployment to Afghanistan. Trait anger and PTSD symptom severity were measured at all assessments. Structural equation modeling including cross-lagged effects showed that higher trait anger before deployment predicted higher PTSD symptoms 2 months after deployment (ß = .36), with stressor severity and baseline symptoms in the model, but not with neuroticism in the model. Trait anger at 2 months postdeployment did not predict PTSD symptom severity at 9 months, and PTSD symptom severity 2 months postdeployment did not predict subsequent trait anger scores. Findings suggest that trait anger may be a pretrauma vulnerability factor for PTSD symptoms, but does not add variance beyond the effect of neuroticism.

The relationship between behavioural problems in preschool children and parental distress after a paediatric burn event.

This study examines mother- and father-rated emotional and behaviour problems in and worries about 0- to 5-year-old children at 3 and 12 months after a burn event and the relation with parental distress. Mothers (n = 150) and fathers (n = 125) representing 155 children participated in this study. Child emotional and behaviour problems and parental worries about the child were assessed with the Child Behavior Checklist at both time points. Parents' level of acute subjective distress was assessed within the first month after the burn event with the Impact of Event Scale. Mothers and fathers held comparable views of their child's emotional and behaviour problems, which were generally within the normal limits. Parents' own acute stress reactions were significantly related to parent-rated child behaviour problems at 3 and 12 months postburn. A substantial part of mothers' and fathers' worries about the child concerned physical and emotional aspects of the burn trauma, and potential future social problems. Parents with high acute stress scores more often reported burn-related concerns about their child at 3 and 12 months postburn. Health-care professionals should be informed that parents' distress in the subacute phase of their child's burn event may be related to subsequent worries about their child and to (parent-observed) child emotional and behaviour problems. The authors recommend a family perspective, with particular attention for the interplay between parents' distress and parent-reported child behaviour problems and worries, in each phase of paediatric burn care.

The SAFE procedure: a practical stopping heuristic for active learning-based screening in systematic reviews and meta-analyses.

Active learning has become an increasingly popular method for screening large amounts of data in systematic reviews and meta-analyses. The active learning process continually improves its predictions on the remaining unlabeled records, with the goal of identifying all relevant records as early as possible. However, determining the optimal point at which to stop the active learning process is a challenge. The cost of additional labeling of records by the reviewer must be balanced against the cost of erroneous exclusions. This paper introduces the SAFE procedure, a practical and conservative set of stopping heuristics that offers a clear guideline for determining when to end the active learning process in screening software like ASReview. The eclectic mix of stopping heuristics helps to minimize the risk of missing relevant papers in the screening process. The proposed stopping heuristic balances the costs of continued screening with the risk of missing relevant records, providing a practical solution for reviewers to make informed decisions on when to stop screening. Although active learning can significantly enhance the quality and efficiency of screening, this method may be more applicable to certain types of datasets and problems. Ultimately, the decision to stop the active learning process depends on careful consideration of the trade-off between the costs of additional record labeling against the potential errors of the current model for the specific dataset and context.

Addressing the challenges of reconstructing systematic reviews datasets: a case study and a noisy label filter procedure.

Systematic reviews and meta-analyses typically require significant time and effort. Machine learning models have the potential to enhance screening efficiency in these processes. To effectively evaluate such models, fully labeled datasets-detailing all records screened by humans and their labeling decisions-are imperative. This paper presents the creation of a comprehensive dataset for a systematic review of treatments for Borderline Personality Disorder, as reported by Oud et al. (2018) for running a simulation study. The authors adhered to the PRISMA guidelines and published both the search query and the list of included records, but the complete dataset with all labels was not disclosed. We replicated their search and, facing the absence of initial screening data, introduced a Noisy Label Filter (NLF) procedure using active learning to validate noisy labels. Following the NLF application, no further relevant records were found. A simulation study employing the reconstructed dataset demonstrated that active learning could reduce screening time by 82.30% compared to random reading. The paper discusses potential causes for discrepancies, provides recommendations, and introduces a decision tree to assist in reconstructing datasets for the purpose of running simulation studies.

Impact of Active learning model and prior knowledge on discovery time of elusive relevant papers: a simulation study.

Software that employs screening prioritization through active learning (AL) has accelerated the screening process significantly by ranking an unordered set of records by their predicted relevance. However, failing to find a relevant paper might alter the findings of a systematic review, highlighting the importance of identifying elusive papers. The time to discovery (TD) measures how many records are needed to be screened to find a relevant paper, making it a helpful tool for detecting such papers. The main aim of this project was to investigate how the choice of the model and prior knowledge influence the TD values of the hard-to-find relevant papers and their rank orders. A simulation study was conducted, mimicking the screening process on a dataset containing titles, abstracts, and labels used for an already published systematic review. The results demonstrated that AL model choice, and mostly the choice of the feature extractor but not the choice of prior knowledge, significantly influenced the TD values and the rank order of the elusive relevant papers. Future research should examine the characteristics of elusive relevant papers to discover why they might take a long time to be found.