Pedagogy, Partnership, and Collaboration: A Longitudinal, Empirical Study of Serious Educational Gameplay in Secondary Biology Classrooms.

The use of serious educational games has the potential to increase student learning outcomes in science education by providing students with opportunities to explore phenomena in ways that vary from traditional instruction; yet, empirical research to support this assertion is limited. This study aimed to explore deeply what learning gains were associated with the use of three serious educational games (SEGs) created for use in secondary biology classrooms that partner teachers implemented during a 2-week curriculum unit. This longitudinal, mixed method study includes a control year, in which we examined how six highly qualified teachers taught students (n = 407) a 2-week curriculum unit addressing cellular biology without the SEGs, followed by 2 years in which the teachers integrated the SEGs into the curriculum unit with students (n =871). Data were collected from multiple sources, including a validated content pre- and post-test measure, embedded gameplay data, participant observation, teacher interviews, and focus groups. Quantitative findings showed significant learning gains associated with students who experienced the game condition during year 2, when compared with the control condition. During the replication year (year 3), learning gains increased again, compared with year two. Although the SEGs did not change between years 2 and 3, teachers were provided real-time access to students' performance during gameplay. Thematic analysis of observation notes, teacher interviews, and student performance in-game identified four affordances teachers identified related to the use of serious educational games in their classrooms and the extended partnership model employed. Implications for researchers and game designers are discussed.

Priors in Bayesian Estimation under the Rasch Model.

A review of various priors used in Bayesian estimation under the Rasch model is presented together with clear mathematical definitions of the hierarchical prior distributions. A Bayesian estimation method, Gibbs sampling, was compared with conditional, marginal, and joint maximum likelihood estimation methods using the Knox Cube Test data under the Rasch model. The shrinkage effect of the priors on item and ability parameter estimates was also investigated using the Knox Cube Test data. In addition, item response data for a mathematics test with 14 items by 765 examinees were analyzed with the joint maximum likelihood estimation method and Gibbs sampling under the Rasch model. Both methods yielded nearly identical item parameter estimates. The shrinkage effect was observed in the ability estimates from Gibbs sampling. The computer program OpenBUGS that implemented the rejection sampling method of Gibbs sampling was the main program employed in the study.

Patient perspectives on buprenorphine/naloxone: a qualitative study of retention during the starting treatment with agonist replacement therapies (START) study.

This study examines the barriers and facilitators of retention among patients receiving buprenorphine/naloxone at eight community-based opioid treatment programs across the United States. Participants (n = 105) were recruited up to three and a half years after having participated in a randomized clinical trial comparing the effect of buprenorphine/naloxone and methadone on liver function. Semi-structured interviews were conducted with 67 patients provided with buprenorphine/naloxone who had terminated early and 38 patients who had completed at least 24 weeks of the trial. Qualitative data were analyzed using the constant comparison method. Barriers to buprenorphine/naloxone retention that emerged included factors associated with: (1) the design of the clinical trial; (2) negative medication or treatment experience; and (3) personal circumstances. The facilitators comprised: (1) positive experience with the medication; (2) personal determination and commitment to complete; and (3) staff encouragement and support. The themes drawn from interviews highlight the importance of considering patients' prior experience with buprenorphine/naloxone and methadone, medication preference, personal circumstances, and motivation to abstain from illicit use or misuse of opioids, as these may influence retention. Ongoing education of patients and staff regarding buprenorphine/naloxone, especially in comparison to methadone, and support from staff and peers are essential.

An Evaluation of Fit Indices Used in Model Selection of Dichotomous Mixture IRT Models.

A Monte Carlo simulation study was conducted to compare fit indices used for detecting the correct latent class in three dichotomous mixture item response theory (IRT) models. Ten indices were considered: Akaike's information criterion (AIC), the corrected AIC (AICc), Bayesian information criterion (BIC), consistent AIC (CAIC), Draper's information criterion (DIC), sample size adjusted BIC (SABIC), relative entropy, the integrated classification likelihood criterion (ICL-BIC), the adjusted Lo-Mendell-Rubin (LMR), and Vuong-Lo-Mendell-Rubin (VLMR). The accuracy of the fit indices was assessed for correct detection of the number of latent classes for different simulation conditions including sample size (2,500 and 5,000), test length (15, 30, and 45), mixture proportions (equal and unequal), number of latent classes (2, 3, and 4), and latent class separation (no-separation and small separation). Simulation study results indicated that as the number of examinees or number of items increased, correct identification rates also increased for most of the indices. Correct identification rates by the different fit indices, however, decreased as the number of estimated latent classes or parameters (i.e., model complexity) increased. Results were good for BIC, CAIC, DIC, SABIC, ICL-BIC, LMR, and VLMR, and the relative entropy index tended to select correct models most of the time. Consistent with previous studies, AIC and AICc showed poor performance. Most of these indices had limited utility for three-class and four-class mixture 3PL model conditions.

Exploring examinees' responses to constructed response items with a supervised topic model.

Textual data are increasingly common in test data as many assessments include constructed response (CR) items as indicators of participants' understanding. The development of techniques based on natural language processing has made it possible for researchers to rapidly analyse large sets of textual data. One family of statistical techniques for this purpose are probabilistic topic models. Topic modelling is a technique for detecting the latent topic structure in a collection of documents and has been widely used to analyse texts in a variety of areas. The detected topics can reveal primary themes in the documents, and the relative use of topics can be useful in investigating the variability of the documents. Supervised latent Dirichlet allocation (SLDA) is a popular topic model in that family that jointly models textual data and paired responses such as could occur with participants' textual answers to CR items and their rubric-based scores. SLDA has an assumption of a homogeneous relationship between textual data and paired responses across all documents. This approach, while useful for some purposes, may not be satisfied for situations in which a population has subgroups that have different relationships. In this study, we introduce a new supervised topic model that incorporates finite-mixture modelling into the SLDA. This new model can detect latent groups of participants that have different relationships between their textual responses and associated scores. The model is illustrated with an example from an analysis of a set of textual responses and paired scores from a middle grades assessment of science inquiry knowledge. A simulation study is presented to investigate the performance of the proposed model under practical testing conditions.

The Impact of Sample Size and Various Other Factors on Estimation of Dichotomous Mixture IRT Models.

The purpose of this study was to examine the effects of different data conditions on item parameter recovery and classification accuracy of three dichotomous mixture item response theory (IRT) models: the Mix1PL, Mix2PL, and Mix3PL. Manipulated factors in the simulation included the sample size (11 different sample sizes from 100 to 5000), test length (10, 30, and 50), number of classes (2 and 3), the degree of latent class separation (normal/no separation, small, medium, and large), and class sizes (equal vs. nonequal). Effects were assessed using root mean square error (RMSE) and classification accuracy percentage computed between true parameters and estimated parameters. The results of this simulation study showed that more precise estimates of item parameters were obtained with larger sample sizes and longer test lengths. Recovery of item parameters decreased as the number of classes increased with the decrease in sample size. Recovery of classification accuracy for the conditions with two-class solutions was also better than that of three-class solutions. Results of both item parameter estimates and classification accuracy differed by model type. More complex models and models with larger class separations produced less accurate results. The effect of the mixture proportions also differentially affected RMSE and classification accuracy results. Groups of equal size produced more precise item parameter estimates, but the reverse was the case for classification accuracy results. Results suggested that dichotomous mixture IRT models required more than 2,000 examinees to be able to obtain stable results as even shorter tests required such large sample sizes for more precise estimates. This number increased as the number of latent classes, the degree of separation, and model complexity increased.

Sequential Bayesian Ability Estimation Applied to Mixed-Format Item Tests.

Large-scale tests often contain mixed-format items, such as when multiple-choice (MC) items and constructed-response (CR) items are both contained in the same test. Although previous research has analyzed both types of items simultaneously, this may not always provide the best estimate of ability. In this paper, a two-step sequential Bayesian (SB) analytic method under the concept of empirical Bayes is explored for mixed item response models. This method integrates ability estimates from different item formats. Unlike the empirical Bayes method, the SB method estimates examinees' posterior ability parameters with individual-level sample-dependent prior distributions estimated from the MC items. Simulations were used to evaluate the accuracy of recovery of ability and item parameters over four factors: the type of the ability distribution, sample size, test length (number of items for each item type), and person/item parameter estimation method. The SB method was compared with a traditional concurrent Bayesian (CB) calibration method, EAPsum, that uses scaled scores for summed scores to estimate parameters from the MC and CR items simultaneously in one estimation step. From the simulation results, the SB method showed more accurate and reliable ability estimation than the CB method, especially when the sample size was small (150 and 500). Both methods presented similar recovery results for MC item parameters, but the CB method yielded a bit better recovery of the CR item parameters. The empirical example suggested that posterior ability estimated by the proposed SB method had higher reliability than the CB method.

Corrigendum: Integrating a Statistical Topic Model and a Diagnostic Classification Model for Analyzing Items in a Mixed Format Assessment.

[This corrects the article DOI: 10.3389/fpsyg.2020.579199.].

Predictors of hepatitis knowledge improvement among methadone maintained clients enrolled in a hepatitis intervention program.

This randomized, controlled study (n = 256) was conducted to compare three interventions designed to promote hepatitis A virus (HAV) and hepatitis B virus (HBV) vaccination completion, among clients undergoing methadone maintenance treatment (MMT) in Los Angeles and Santa Monica. The participants were randomized into three groups: Motivational Interviewing-Single Session (MI-Single), Motivational Interviewing-Group (MI-Group), or Nurse-Led Hepatitis Health Promotion (HHP). All three treatment groups received the 3-series HAV/HBV vaccine. The MI sessions were provided by trained therapists, the Nurse-Led HHP sessions were delivered by a research nurse. The main outcome variable of interest was improvement in HBV and HCV knowledge, measured by a 6-item HBV and a 7-item HCV knowledge and attitude tool that was administered at baseline and at 6-month follow-up. The study results showed that there was a significant increase in HBV- and HCV-related knowledge across all three groups (p < 0.0001). There were no significant differences found with respect to knowledge acquisition among the groups. Irrespective of treatment group, gender (P = 0.008), study site (P < 0.0001) and whether a participant was abused as a child (P = 0.017) were all found to be predictors of HCV knowledge improvement; only recruitment site (P < 0.0001) was found to be a predictor of HBV knowledge. The authors concluded that, although MI-Single, MI-Group and Nurse-Led HHP are all effective in promoting HBV and HCV knowledge acquisition among MMT clients, Nurse-Led HHP may be the method of choice for this population as it may be easier to integrate and with additional investigation may prove to be more cost efficient.

Predictors of HAV/HBV vaccination completion among methadone maintenance clients.

This randomized, controlled study (N = 256) was conducted to compare three interventions designed to promote hepatitis A virus (HAV) and hepatitis B virus (HBV) vaccination completion among clients undergoing methadone maintenance (MM) treatment. Participants were recruited from five MM treatment sites in Southern California and randomized into three groups: Motivational Interviewing-Single (MI-Single), Motivational Interviewing-Group (MI-Group); and Nurse-Led Hepatitis Health Promotion (HHP). All were offered the three-series HAV/HBV vaccine. A total of 148 participants completed the vaccine. Groups did not differ in rate of vaccination completion (73.6%, HHP group, vs. 65% and 69% for the MI-Single and MI-Group, respectively). The equivalence of findings across groups suggests the value of including nurses with a comprehensive health focus in promoting vaccination completion.

The Impact of Markov Chain Convergence on Estimation of Mixture IRT Model Parameters.

A nonconverged Markov chain can potentially lead to invalid inferences about model parameters. The purpose of this study was to assess the effect of a nonconverged Markov chain on the estimation of parameters for mixture item response theory models using a Markov chain Monte Carlo algorithm. A simulation study was conducted to investigate the accuracy of model parameters estimated with different degree of convergence. Results indicated the accuracy of the estimated model parameters for the mixture item response theory models decreased as the number of iterations of the Markov chain decreased. In particular, increasing the number of burn-in iterations resulted in more accurate estimation of mixture IRT model parameters. In addition, the different methods for monitoring convergence of a Markov chain resulted in different degrees of convergence despite almost identical accuracy of estimation.

Sample Size Requirements for Applying Diagnostic Classification Models.

Results of a comprehensive simulation study are reported investigating the effects of sample size, test length, number of attributes and base rate of mastery on item parameter recovery and classification accuracy of four DCMs (i.e., C-RUM, DINA, DINO, and LCDMREDUCED). Effects were evaluated using bias and RMSE computed between true (i.e., generating) parameters and estimated parameters. Effects of simulated factors on attribute assignment were also evaluated using the percentage of classification accuracy. More precise estimates of item parameters were obtained with larger sample size and longer test length. Recovery of item parameters decreased as the number of attributes increased from three to five but base rate of mastery had a varying effect on the item recovery. Item parameter and classification accuracy were higher for DINA and DINO models.

The Impact of Test and Sample Characteristics on Model Selection and Classification Accuracy in the Multilevel Mixture IRT Model.

The standard item response theory (IRT) model assumption of a single homogenous population may be violated in real data. Mixture extensions of IRT models have been proposed to account for latent heterogeneous populations, but these models are not designed to handle multilevel data structures. Ignoring the multilevel structure is problematic as it results in lower-level units aggregated with higher-level units and yields less accurate results, because of dependencies in the data. Multilevel data structures cause such dependencies between levels but can be modeled in a straightforward way in multilevel mixture IRT models. An important step in the use of multilevel mixture IRT models is the fit of the model to the data. This fit is often determined based on relative fit indices. Previous research on mixture IRT models has shown that performances of these indices and classification accuracy of these models can be affected by several factors including percentage of class-variant items, number of items, magnitude and size of clusters, and mixing proportions of latent classes. As yet, no studies appear to have been reported examining these issues for multilevel extensions of mixture IRT models. The current study aims to investigate the effects of several features of the data on the accuracy of model selection and parameter recovery. Results are reported on a simulation study designed to examine the following features of the data: percentages of class-variant items (30, 60, and 90%), numbers of latent classes in the data (with from 1 to 3 latent classes at level 1 and 1 and 2 latent classes at level 2), numbers of items (10, 30, and 50), numbers of clusters (50 and 100), cluster size (10 and 50), and mixing proportions [equal (0.5 and 0.5) vs. non-equal (0.25 and 0.75)]. Simulation results indicated that multilevel mixture IRT models resulted in less accurate estimates when the number of clusters and the cluster size were small. In addition, mean Root mean square error (RMSE) values increased as the percentage of class-variant items increased and parameters were recovered more accurately under the 30% class-variant item conditions. Mixing proportion type (i.e., equal vs. unequal latent class sizes) and numbers of items (10, 30, and 50), however, did not show any clear pattern. Sample size dependent fit indices BIC, CAIC, and SABIC performed poorly for the smaller level-1 sample size. For the remaining conditions, the SABIC index performed better than other fit indices.

Reliability for Tests With Items Having Different Numbers of Ordered Categories.

This study describes a structural equation modeling (SEM) approach to reliability for tests with items having different numbers of ordered categories. A simulation study is provided to compare the performance of this reliability coefficient, coefficient alpha and population reliability for tests having items with different numbers of ordered categories, a one-factor and a bifactor structures, and different skewness distributions of test scores. Results indicated that the proposed reliability coefficient was close to the population reliability in most conditions. An empirical example was used to illustrate the performance of the different coefficients for a test of items with two or three ordered categories.

Integrating a Statistical Topic Model and a Diagnostic Classification Model for Analyzing Items in a Mixed Format Assessment.

Selected response items and constructed response (CR) items are often found in the same test. Conventional psychometric models for these two types of items typically focus on using the scores for correctness of the responses. Recent research suggests, however, that more information may be available from the CR items than just scores for correctness. In this study, we describe an approach in which a statistical topic model along with a diagnostic classification model (DCM) was applied to a mixed item format formative test of English and Language Arts. The DCM was used to estimate students' mastery status of reading skills. These mastery statuses were then included in a topic model as covariates to predict students' use of each of the latent topics in their written answers to a CR item. This approach enabled investigation of the effects of mastery status of reading skills on writing patterns. Results indicated that one of the skills, Integration of Knowledge and Ideas, helped detect and explain students' writing patterns with respect to students' use of individual topics.

Correlates of heavy smoking among alcohol-using methadone maintenance clients.

This cross-sectional study examines predictors of heavy smoking among 256 male and female methadone maintenance therapy (MMT) clients from five MMT clinics in the Los Angeles area. The authors find that women report lower rates of heavy smoking than men (47% vs. 54%, respectively), in concordance with current literature pointing to gender differences in smoking behaviors. In particular, men who report heavy drinking, fair or poor health, and recent heroin use are more likely to report heavy smoking compared with men not reporting these factors. Women who report recent heroin use, a lifetime history of sex trade, and who have been ill enough to require a blood transfusion also have greater odds of reporting heavy cigarette smoking. Findings from this study may aid not only in designing gender-based smoking cessation programs for MMT clients but also in addressing the gender-based issues related to smoking in such a population.

Correlates of hospitalization for alcohol-using methadone-maintained persons with physical health problems.

This cross-sectional study (n = 190) examined correlates of hospitalization for physical health problems among methadone maintenance therapy (MMT) clients with a history of alcohol abuse. The study was derived from baseline data collected for a longitudinal trial assessing the effect of motivational interviewing among alcohol-abusing adults undergoing MMT. The sample included clients who were 18-55 years of age, abusing alcohol, and receiving MMT from five large methadone maintenance clinics in the Los Angeles area. A structured questionnaire was used to collect the data. Correlates of hospitalization in logistic regression analysis included lack of social support, recent victimization, age of first alcohol use, chronic severe pain in the previous 6 months, not having children, and ethnicity. Identification of hospitalization risk factors among alcohol-abusing MMT clients is a first step to developing risk-reducing interventions designed to lower hospitalization rates in this population.

Model Selection for Multilevel Mixture Rasch Models.

Mixture item response theory (MixIRT) models can sometimes be used to model the heterogeneity among the individuals from different subpopulations, but these models do not account for the multilevel structure that is common in educational and psychological data. Multilevel extensions of the MixIRT models have been proposed to address this shortcoming. Successful applications of multilevel MixIRT models depend in part on detection of the best fitting model. In this study, performance of information indices, Akaike information criterion (AIC), Bayesian information criterion (BIC), consistent Akaike information criterion (CAIC), and sample-size adjusted Bayesian information criterion (SABIC), were compared for use in model selection with a two-level mixture Rasch model in the context of a real data example and a simulation study. Level 1 consisted of students and Level 2 consisted of schools. The performances of the model selection criteria under different sample sizes were investigated in a simulation study. Total sample size (number of students) and Level 2 sample size (number of schools) were studied for calculation of information criterion indices to examine the performance of these fit indices. Simulation study results indicated that CAIC and BIC performed better than the other indices at detection of the true (i.e., generating) model. Furthermore, information indices based on total sample size yielded more accurate detections than indices at Level 2.

Use of Information Criteria in the Study of Group Differences in Trace Lines.

A brief review of various information criteria is presented for the detection of differential item functioning (DIF) under item response theory (IRT). An illustration of using information criteria for model selection as well as results with simulated data are presented and contrasted with the IRT likelihood ratio (LR) DIF detection method. Use of information criteria for general IRT model selection is discussed.