Examining engagement with and acceptability and usability of REAL Parenting: A brief online parent-based intervention to reduce alcohol use and consequences among high school students.

This study is a formative assessment of REAL Parenting (RP): a brief, digital intervention for parents of high school students that encourages parent-teen communication about alcohol and, in turn, aims to prevent teen alcohol use. The aims of this study were to describe engagement with, and acceptability and usability of RP; and to explore the relationship of these measures with each other and with short-term outcomes. Participants were 160 parents randomly assigned to the treatment group who received RP as part of a randomized pilot trial (Mage = 45.43[SD = 7.26], 59.3% female, 56% White, 19% Hispanic). App-based program analytics captured real-time engagement with RP. Parents completed self-report measures of acceptability, usability, perceived communication effectiveness, perceived self-efficacy to communicate, and frequency of communication post-intervention. Descriptive statistics were calculated to describe engagement, acceptability and usability, and zero-order correlations were calculated to examine associations between these and self-report variables. About 75% (n = 118) of parents accessed the intervention and two-thirds (n = 110) accessed at least one module. Self-report ratings of acceptability and usability were neutral to positive, and mothers liked RP more than fathers. Self-report, but not program analytic indicators were associated with short-term outcomes. Findings suggest that, with little incentive, most parents will access an app focused on parent-teen communication about alcohol. While parent feedback was positive, it also highlighted areas for improvement with app content and design. Correlations suggest that analytic metrics of engagement are useful to discern who is and is not using interventions, and self-report measures are important for understanding pathways by which interventions are associated with short-term outcomes.

Cross-Tailoring Integrative Alcohol and Risky Sexual Behavior Feedback for College Students: Protocol for a Hybrid Type 1 Effectiveness-Implementation Trial.

BACKGROUND: Underage drinking and related risky sexual behavior (RSB) are major public health concerns on United States college campuses. Although technology-delivered personalized feedback interventions (PFIs) are considered a best practice for individual-level campus alcohol prevention, there is room for improving the effectiveness of this approach with regard to alcohol-related RSB. OBJECTIVE: The aims of this study are to (1) evaluate the impact of a brief PFI that integrates content on alcohol use and RSB and is adapted to include a novel cross-tailored dynamic feedback (CDF) component for at-risk first-year college students and (2) identify implementation factors critical to the CDF's success to facilitate future scale-up in campus settings. METHODS: This study uses a hybrid type 1 effectiveness-implementation design and will be conducted in 3 phases. Phase 1 is a stakeholder-engaged PFI+CDF adaptation guided by focus groups and usability testing. In phase 2, 600 first-year college students who drink and are sexually active will be recruited from 2 sites (n=300 per site) to participate in a 4-group randomized controlled trial to examine the effectiveness of PFI+CDF in reducing alcohol-related RSB. Eligible participants will complete a baseline survey during the first week of the semester and follow-up surveys at 1, 2, 3, 6, and 13 months post baseline. Phase 3 is a qualitative evaluation with stakeholders to better understand relevant implementation factors. RESULTS: Recruitment and enrollment for phase 1 began in January 2022. Recruitment for phases 2 and 3 is planned for the summer of 2023 and 2024, respectively. Upon collection of data, the effectiveness of PFI+CDF will be examined, and factors critical to implementation will be evaluated. CONCLUSIONS: This hybrid type 1 trial is designed to impact the field by testing an innovative adaptation that extends evidence-based alcohol programs to reduce alcohol-related RSB and provides insights related to implementation to bridge the gap between research and practice at the university level. TRIAL REGISTRATION: ClinicalTrials.gov NCT05011903; https://clinicaltrials.gov/ct2/show/NCT05011903. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/43986.

A randomized controlled trial of a web-based personalized feedback intervention targeting frequent indoor tanning bed users: Engagement, acceptability, and preliminary behavioral outcomes.

Frequent indoor tanning bed use is an established public health concern, yet research on tanning cessation interventions for frequent tanners is lacking. We describe the protocol for a brief, web-based tanning behavior change intervention and present evidence that it is acceptable and engaging to frequent indoor tanners. Lower tanning rates were not observed among participants receiving the intervention in a randomized controlled trial but participants' interest in changing tanning increased. This intervention could be a useful approach to increasing frequent tanners' interest in behavior change and openness to engaging within a more intensive, multi-component tanning cessation program.Trial Registration: NCT03448224 Clinical Trials.gov (https://clinicaltrials.gov/ct2/show/NCT03448224?cond=NCT03448224&draw=2&rank=1).

Protective behavioral strategies are more helpful for avoiding alcohol-related problems for college drinkers who drink less.

Objective: To examine race, gender, and alcohol use level as moderators of the association between protective behavioral strategies (PBS) and alcohol-related problems. Participants: A sample of 12,011 participants who reported recent drinking (87.7% White, 61% Women) from Project INTEGRATE, a study that combined individual participant data (IPD) from 24 brief motivational intervention trials for college students. Methods: Hierarchical regressions were conducted to determine whether there was a moderated effect of PBS on alcohol problems across alcohol use levels, and whether the moderated protective effect of PBS by alcohol use differed by gender and race. Results: The protective association between PBS and alcohol-related problems was greater for those who drank less. This moderated effect did not differ across men and women or across racial groups. Conclusions: College drinking prevention programs should ensure that students are aware of the limits of PBS as a mitigator of alcohol problems.

The Role of Engagement in Effective, Digital Prevention Interventions: the Function of Engagement in the REAL Media Substance Use Prevention Curriculum.

Prevention curricula rely on audience engagement to effectively communicate their messages. However, to date, measurement of engagement has primarily focused on self-report that is often an indicator of liking or satisfaction. Emerging technologies for intervention delivery hold promise not only for additional engagement indicators but also for dissemination outside of traditional vehicles such as classroom delivery. The present study, grounded in the theory of active involvement (Greene 2013), explores the role of engagement (as measured by self-report, program analytics, and observation) with short-term substance use prevention outcomes such as self-efficacy to counter-argue and descriptive and injunctive norms. The study tracks 4-H youth (N = 310) engaged with a media literacy focused e-learning substance prevention curriculum, REAL media. Results indicate that self-reports of engagement predicted self-efficacy to counter-argue, but a program-analytic indicator of dosage predicted lower injunctive and descriptive norms, all at 3 months. The observational indicator was correlated with self-efficacy to counter-argue but not significant in the predictive models. The implications and directions for future research regarding how engagement is measured in prevention and included in studying program effects are discussed. Clinical trial: NCT03157700, May 2017.

Short term effects of the REAL media e-learning media literacy substance prevention curriculum: An RCT of adolescents disseminated through a community organization.

BACKGROUND: The primary aim of this study was to evaluate the short-term effects of testing an e-learning program to reduce adolescent substance use and abuse. Early initiation of substance use is linked to a variety of negative outcomes, thus effective intervention programs are needed. One approach is to use media literacy to capitalize on adolescents' immersion with media in a variety of forms. We developed, implemented, and tested an engaging substance use prevention program by collaborating with a youth-oriented community partner (4-H). METHODS: 639 middle adolescents from nine U.S. states participated in an RCT of REAL media. Participants completed a series of online surveys and were randomized to use an online substance prevention program (REAL media) or serve as control (delayed program use). Self-report surveys were administered at three points in time. This short-term evaluation uses data from the pretest (Time 1) and short-term posttest three-month surveys, which measured demographics, self-efficacy to counterargue, and injunctive and descriptive substance use norms. RESULTS: Participants who completed the REAL media program reported increased self-efficacy to counterargue and decreased positive injunctive norms compared to control participants who did not complete the program. No significant differences were observed for descriptive norms. CONCLUSIONS: We found support for the REAL media program in changing key predictors of youth substance use demonstrating (1) the efficacy of media literacy interventions targeting adolescents and (2) that e-learning substance use prevention efforts can be adapted for and implemented through community organizations.

Concussion Symptom Underreporting Among Incoming National Collegiate Athletic Association Division I College Athletes.

OBJECTIVE: Accurate diagnosis of sport-related concussions relies heavily on truthful self-reporting of symptom severity. Previous studies have emphasized lack of knowledge as a factor in symptom nondisclosure. This study sought to examine concussion knowledge and the relationship of knowledge to reasons for symptom nondisclosure. DESIGN: Cross-sectional study. SETTING: Data were collected during preparticipation athletic evaluations via electronic survey. PARTICIPANTS: One hundred fifty-six incoming National Collegiate Athletic Association Division I student-athletes. MAIN OUTCOME MEASURES: Survey items included previous concussion diagnosis, concussion fact and symptom knowledge, reasons and situational contexts for nondisclosure, and stakeholder attitudes. RESULTS: Participants, on average, had substantial concussion symptom and fact knowledge. Unexpectedly, participants with higher concussion fact knowledge endorsed more reasons that athletes may hide symptoms. Concussion symptom knowledge was unrelated to reasons for nondisclosure. Athletes believed that symptom reporting was less likely in high-stakes versus low-stakes situations and consistently identified their teammates as holding attitudes that support underreporting and athletic trainers as engaging in behaviors that support player safety. CONCLUSIONS: Greater concussion knowledge did not reduce the number of reasons that participants viewed as drivers for concussion nondisclosure. In other words, participants understood why athletes choose to hide symptoms even when they also understood the symptoms, risks, sequelae, and consequences of concussion (and potential harm of nondisclosure). Situational contexts and important stakeholder attitudes also appeared to importantly influence symptom disclosure decisions. A multifaceted approach that goes beyond current educational strategies to addresses situational, social, and athletic pressures may be needed to initiate a widespread cultural shift away from concussion nondisclosure.

Exploring Indicators of Engagement in Online Learning as Applied to Adolescent Health Prevention: A Pilot Study of REAL media.

Engagement is central to the effectiveness of online health messages and the related educational programs that aim to deliver these messages to the intended audience (Li, Won, Yang et al. 2019: Lin, Hung, Kinshuk et al. 2019). Drawing from health communication and social learning theories, the Theory of Active Involvement (TAI) (Greene, 2013) posits that an online prevention program's impact depends on how engaged participants are. In practice, measuring engagement in this context has relied primarily on self-report measures (e.g., Hamutoglu, Gemikonakli, Duman et al. 2019). However, the emergence and growth of online learning platforms to deliver health-specific information offers other options for assessing engagement. This includes program analytics that capture interaction with content and facilitate examination of patterns via multiple indicators such as responses to interactive questions and time spent in the program (Herodotou, Rienties, Boroowa, et al. 2019; Li, Wong, Yang et al. 2019; van Leeuwen, 2019). However, little is known about the relationships between these different indicators of engagement as it applies to health curricula. This study uses self-report, observational, and program analytic data collected on a small (N = 38) sample using REAL media, an online substance use prevention program, to examine relationships among various indicators of engagement. Findings suggest a cluster of indicators across the three modalities that provide a useful way of measuring engagement. A cluster centered around complexity suggests a separate factor to be considered when designing engaging interventions.

"Choose Today, Live Tomorrow": A Content Analysis of Anti-Substance Use Messages Produced by Adolescents.

Adolescent-produced anti-substance use messaging is an increasingly popular and effective prevention strategy. However, little is known about the content of these messages and the production elements adolescents use to bring that content to life. In this article, we present a content analysis of 95 anti-substance use messages developed by 4-H club members across nine U.S. states as part of their participation in the media literacy program REAL media. Posters and videos were content-analyzed for target substance, prevention goal, message form, message content, persuasion strategies, and production elements. Results of the content analysis revealed that combustible tobacco (smoking) was the most popular target substance in the sample among the choices of alcohol, marijuana, e-cigarettes, and chewing tobacco. More youth developed messages with the goal of preventing substance use, rather than stopping current use. Slogans were used in the majority of messages, and nearly all messages took an informational form, rather than narrative or statistical form. Persuasion strategies covered in the curriculum, including fun with the group, unexpected, style, and endorsement were scantily used. Finally, results showed that production value was high in this sample, reflected by the extensive use of color and variety of fonts and font sizes. Implications for future media literacy interventions and research are discussed.

An E-Learning Adaptation of an Evidence-Based Media Literacy Curriculum to Prevent Youth Substance Use in Community Groups: Development and Feasibility of REAL Media.

BACKGROUND: There is a need for evidence-based substance use prevention efforts that target high school-aged youth that are easy to implement and suitable for dissemination in school and community groups. The Youth Message Development (YMD) program is a brief, four-lesson, in-person curriculum that aims to prevent youth substance use through the development of youth media literacy. Specifically, YMD aims to increase understanding of advertising reach and costs, along with the techniques used to sell products; develop counterarguing and critical thinking skills in response to advertisements; and facilitate application of these skills to the development of youth-generated antisubstance messages. Although YMD has demonstrated evidence of success, it is limited by its delivery method and focus on alcohol and smoking. OBJECTIVE: Study objectives were two-fold: (1) to adapt the YMD curriculum to a self-paced, interactive, electronic-learning (e-learning) format and expand its content to cover alcohol, combustible cigarettes, e-cigarettes, smokeless tobacco, marijuana, and prescription drugs, and (2) to test the feasibility of the adapted curriculum in partnership with a national youth organization. METHODS: An iterative process was employed in partnership with the 4-H youth development organization and a technology developer and consisted of six phases: (1) focus groups to guide adaptation, (2) adaptation to an e-learning format renamed REAL media, (3) pilot-testing of the REAL media prototype to determine feasibility and acceptability, (4) program revisions, (5) usability testing of the revised prototype, and (6) final revisions. Focus groups and pilot and usability testing were conducted with 4-H youth club members and adult club leaders. RESULTS: Focus group feedback guided the build of an e-learning prototype of REAL media, which consisted of five online levels and interactive content guided by a mix of narration and on-screen text. Results of a pilot test of the prototype were neutral to positive, and the program was refined based on end-user feedback. An independent usability test indicated that youth 4-H members felt favorably about navigating REAL media, and they reported high self-efficacy in applying skills learned in the program. Additional refinements to the program were made based on their feedback. CONCLUSIONS: The iterative build process involving the end user from the outset yielded an overall successful technology-driven adaptation of an evidence-based curriculum. This should increase the likelihood of effectively impacting behavioral outcomes as well as uptake within community organizations.

Smokers' perceptions of risks and harm from snus relative to cigarettes: A latent profile analysis study.

INTRODUCTION: Certain types of smokeless tobacco (SLT) products, particularly snus, carry fewer health risks than cigarette smoking and might be able to serve as harm-reduction products for smokers. However, studies frequently find that smokers misperceive SLT and snus to be as or more harmful than smoking. This perception is often measured with a single general harm question, and research on underlying risk perceptions is limited. METHODS: Using a sample of 256 current smokers, we utilized Latent Profile Analysis to examine response profiles to items that assessed perceived risk of specific health outcomes (lung cancer, heart disease, oral cancer) from snus relative to cigarettes, along with the typical single item measure of overall harm from snus compared to cigarettes. RESULTS: Three smoker response profiles emerged. Almost half (44.9%) of smokers perceived snus to be as or more risky than cigarettes for all three specific health outcomes (group 1), while over one third (38.3%) had an elevated perceived risk for oral cancer only (group 2). About 17% of smokers perceived snus to have lower risks for lung cancer only (group 3). Across each profile, perceived risk was highest for oral cancer. CONCLUSIONS: If smokers are to consider snus for harm-reduction, efforts may be needed to better inform smokers about their lower relative risks, including for particular health outcomes of interest. This study also suggests that smokers may vary in their level of need for information to correct their relative risk misperceptions.

Taking an HPV vaccine research-tested intervention to scale in a clinical setting.

Research tested interventions are seldom ready for wide spread use. Successful intervention adaptation to clinical settings demands an iterative process with target audience feedback. We describe the adaptation process of implementing an NCI research tested HPV vaccine intervention, Women's Stories, to a community clinic context (Planned Parenthood). Five phases are described for the adaptation of content and the development of a health kiosk intervention delivery system: (a) informant interviews with the target audience of young adult, predominantly African-American women, (b) translating HPV vaccine decision narratives into prevention messages, (c) health kiosk interface design, (d) conducting a usability study of the health kiosk intervention product, and (e) conducting a waiting room observational study. Lessons learned and challenges in adapting prevention interventions to clinical settings are discussed.

Do drinking motives distinguish extreme drinking college students from their peers?

OBJECTIVE: The literature highlights the need to move beyond the traditional heavy episodic ("binge") drinking criteria when trying to identify at-risk college drinkers. Thus, recent attention has focused on more extreme levels of drinking. This study examines whether drinking motives can distinguish college student extreme drinkers from lighter drinkers. METHOD: We used data from 3518 college student current drinkers (63.4% women) who participated in eight different studies at five different college campuses across the United States; a subsample of these students was followed up at 6months post-baseline. At baseline and follow-up, drinkers were divided into three groups: nonbinge drinkers (<4 drinks for women and 5 for men on their maximum drinking occasion), binge drinkers (4-7 drinks for women; 5-9 for men), and extreme drinkers (8+ for women and 10+ for men). RESULTS: At baseline, extreme drinkers, compared to nonbinge and binge drinkers, reported greater social, enhancement, and coping motives, as well as greater quantity and frequency of drinking per week and more alcohol-related problems. Those who were not extreme drinkers at baseline and later became extreme drinkers at follow-up reported significantly greater increases in social and enhancement motives, compared to those who remained nonextreme drinkers. Those who were extreme drinkers at baseline and reduced their drinking 6months later, compared to those who remained extreme drinkers, reported greater reductions in enhancement and coping motives. CONCLUSIONS: Focusing on drinking motives might be an efficacious target for preventive intervention programs to reduce extreme drinking among college students.

The association between protective behavioral strategies and alcohol-related problems: An examination of race and gender differences among college drinkers.

This study examined race and gender differences in use of specific types of protective behavioral strategies (PBS) and the moderating effects of race and gender on the relationship between PBS use and alcohol problems, controlling for alcohol use, among a large sample of Asian, Black, and White college drinkers. There were significant racial and gender differences in the types of PBS used. Moderation analyses indicated that PBS were more protective for women than men against experiencing alcohol-related problems. There were no significant race effects or race-by-gender interaction effects on alcohol problems. Implementing PBS may be beneficial for all college students.

Drinking norms, readiness to change, and gender as moderators of a combined alcohol intervention for first-year college students.

INTRODUCTION: Alcohol interventions targeting college students and their parents have been shown to be efficacious. Little research has examined moderators of intervention efficacy to help tailor interventions for subgroups of students. METHOD: This study is a secondary data analysis of readiness to change, drinking norms, and gender as moderators of an efficacious peer- and parent-based intervention (Turrisi et al., 2009). Students (n=680) were randomized to the combined peer and parent intervention (n=342) or assessment-only control (n=338). RESULTS: The combined intervention reduced peak blood alcohol content (BAC) compared to control. Gender and norms did not moderate the relationship between the intervention and drinking. Significant interactions were found between gender, precontemplation, and intervention. Students in the combined condition with higher precontemplation had lower weekly drinking compared to those with lower precontemplation. This pattern was also found among men for peak BAC and alcohol-related consequences but not among women, indicating a three-way interaction. CONCLUSION: Interventions may need to consider readiness to change and gender to optimize effectiveness.

Randomized controlled trial of a web-based indoor tanning intervention: Acceptability and preliminary outcomes.

OBJECTIVE: This article describes the acceptability and preliminary behavioral outcomes of a pilot randomized control trial of a web-based indoor tanning intervention for young adult women. The intervention targets indoor tanning users' perceptions of the benefits and value of tanning and addresses the role of body image-related constructs in indoor tanning. METHOD: Participants were 186 young adult women who reported indoor tanning at least once in the past 12 months. The study design was a 2-arm randomized controlled trial with pre- and postintervention assessments and random assignment to an intervention or control condition. Intervention acceptability was assessed by obtaining participants' evaluation of the intervention. Regression analyses were used to test for intervention condition differences in preliminary behavioral outcomes measured at 6 weeks postintervention. RESULTS: Participants provided favorable evaluations of the intervention on several dimensions and a highly positive overall rating. Intervention participants were more likely to report abstaining from indoor tanning and indicated a lower likelihood of using indoor tanning in the future compared with control participants on the postintervention assessment. No differences were found for sunburns. CONCLUSIONS: The results of this pilot randomized controlled trial provide evidence that the indoor tanning intervention is acceptable to participants and may encourage cessation of indoor tanning behavior. The findings provide preliminary support for an indoor tanning intervention that engages tanners to challenge their beliefs about the benefits of indoor tanning. The use of a web-based indoor tanning intervention is unique and provides strong potential for dissemination.

Are there secondary effects on marijuana use from brief alcohol interventions for college students?

OBJECTIVE: This study examined whether brief motivational interventions (BMIs) designed for reducing heavy drinking among college students have secondary effects on reducing marijuana use. METHOD: The data came from Project INTEGRATE, which combined data from 24 independent trials of BMIs and other individual-focused interventions designed to reduce heavy drinking and related problems among college students. We analyzed data from 10 samples across nine studies that used random assignment of participants into either a BMI or a control group and assessed marijuana use outcomes (N = 6,768; 41.5% men; 73.2% White; 57.7% first-year students; 19.2% current marijuana users at baseline). We derived three marijuana use groups within studies by cross-tabulating baseline and follow-up data: Nonusers, Reducers, and Stayers/Increasers. RESULTS: Peto's one-step odds ratio analyses for meta-analysis revealed no significant intervention effects on marijuana use at either short-term (1-3 month) or long-term (6-12 month) follow-up. Subsequent exploratory analyses showed that those who reduced drinking were more likely to be a marijuana Reducer or Nonuser, compared with a Stayer/Increaser, at both follow-ups. CONCLUSIONS: The BMIs to reduce heavy drinking evaluated in this study did not reduce marijuana use. However, our exploratory results suggest that if we can develop interventions for college students that effectively reduce drinking, we may also reduce their marijuana use. Furthermore, as recreational use of marijuana becomes legal or decriminalized and marijuana becomes more readily available, it may be necessary to develop interventions specifically targeting marijuana use among college students.

Brief motivational interventions for college student drinking may not be as powerful as we think: an individual participant-level data meta-analysis.

BACKGROUND: For over 2 decades, brief motivational interventions (BMIs) have been implemented on college campuses to reduce heavy drinking and related negative consequences. Such interventions include in-person motivational interviews (MIs), often incorporating personalized feedback (PF), and stand-alone PF interventions delivered via mail, computer, or the Web. Both narrative and meta-analytic reviews using aggregate data from published studies suggest at least short-term efficacy of BMIs, although overall effect sizes have been small. METHODS: This study was an individual participant-level data (IPD) meta-analysis of 17 randomized clinical trials evaluating BMIs. Unlike typical meta-analysis based on summary data, IPD meta-analysis allows for an analysis that correctly accommodates the sampling, sample characteristics, and distributions of the pooled data. In particular, highly skewed distributions with many zeroes are typical for drinking outcomes, but have not been adequately accounted for in existing studies. Data are from Project INTEGRATE, one of the largest IPD meta-analysis projects to date in alcohol intervention research, representing 6,713 individuals each with 2 to 5 repeated measures up to 12 months postbaseline. RESULTS: We used Bayesian multilevel over dispersed Poisson hurdle models to estimate intervention effects on drinks per week and peak drinking, and Gaussian models for alcohol problems. Estimates of overall intervention effects were very small and not statistically significant for any of the outcomes. We further conducted post hoc comparisons of 3 intervention types (individual MI with PF, PF only, and group MI) versus control. There was a small, statistically significant reduction in alcohol problems among participants who received an individual MI with PF. Short-term and long-term results were similar. CONCLUSIONS: This study questions the efficacy and magnitude of effects of BMIs for college drinking prevention and intervention and suggests a need for the development of more effective intervention strategies.

Project INTEGRATE: An integrative study of brief alcohol interventions for college students.

This article provides an overview of a study that synthesizes multiple, independently collected alcohol intervention studies for college students into a single, multisite longitudinal data set. This research embraced innovative analytic strategies (i.e., integrative data analysis or meta-analysis using individual participant-level data), with the overall goal of answering research questions that are difficult to address in individual studies such as moderation analysis, while providing a built-in replication for the reported efficacy of brief motivational interventions for college students. Data were pooled across 24 intervention studies, of which 21 included a comparison or control condition and all included one or more treatment conditions. This yielded a sample of 12,630 participants (42% men; 58% first-year or incoming students). The majority of the sample identified as White (74%), with 12% Asian, 7% Hispanic, 2% Black, and 5% other/mixed ethnic groups. Participants were assessed 2 or more times from baseline up to 12 months, with varying assessment schedules across studies. This article describes how we combined individual participant-level data from multiple studies, and discusses the steps taken to develop commensurate measures across studies via harmonization and newly developed Markov chain Monte Carlo (MCMC) algorithms for 2-parameter logistic item response theory models and a generalized partial credit model. This innovative approach has intriguing promises, but significant barriers exist. To lower the barriers, there is a need to increase overlap in measures and timing of follow-up assessments across studies, better define treatment and control groups, and improve transparency and documentation in future single intervention studies.

A Hierarchical Multi-Unidimensional IRT Approach for Analyzing Sparse, Multi-Group Data for Integrative Data Analysis.

The present paper proposes a hierarchical, multi-unidimensional two-parameter logistic item response theory (2PL-MUIRT) model extended for a large number of groups. The proposed model was motivated by a large-scale integrative data analysis (IDA) study which combined data (N = 24,336) from 24 independent alcohol intervention studies. IDA projects face unique challenges that are different from those encountered in individual studies, such as the need to establish a common scoring metric across studies and to handle missingness in the pooled data. To address these challenges, we developed a Markov chain Monte Carlo (MCMC) algorithm for a hierarchical 2PL-MUIRT model for multiple groups in which not only were the item parameters and latent traits estimated, but the means and covariance structures for multiple dimensions were also estimated across different groups. Compared to a few existing MCMC algorithms for multidimensional IRT models that constrain the item parameters to facilitate estimation of the covariance matrix, we adapted an MCMC algorithm so that we could directly estimate the correlation matrix for the anchor group without any constraints on the item parameters. The feasibility of the MCMC algorithm and the validity of the basic calibration procedure were examined using a simulation study. Results showed that model parameters could be adequately recovered, and estimated latent trait scores closely approximated true latent trait scores. The algorithm was then applied to analyze real data (69 items across 20 studies for 22,608 participants). The posterior predictive model check showed that the model fit all items well, and the correlations between the MCMC scores and original scores were overall quite high. An additional simulation study demonstrated robustness of the MCMC procedures in the context of the high proportion of missingness in data. The Bayesian hierarchical IRT model using the MCMC algorithms developed in the current study has the potential to be widely implemented for IDA studies or multi-site studies, and can be further refined to meet more complicated needs in applied research.