Developing Computerized Adaptive Testing for a National Health Professionals Exam: An Attempt from Psychometric Simulations.

Introduction: The accurate assessment of health professionals' competence is critical for ensuring public health safety and quality of care. Computerized Adaptive Testing (CAT) based on the Item Response Theory (IRT) has the potential to improve measurement accuracy and reduce respondent burden. In this study, we conducted psychometric simulations to develop a CAT for evaluating the candidates' competence of health professionals. Methods: The initial CAT item bank was sourced from the Standardized Competence Test for Clinical Medicine Undergraduates (SCTCMU), a nationwide summative test in China, consisting of 300 multiple-choice items. We randomly selected response data from 2000 Chinese clinical medicine undergraduates for analysis. Two types of analyses were performed: first, evaluating the psychometric properties of all items to meet the requirements of CAT; and second, conducting multiple CAT simulations using both simulated and real response data. Results: The final CAT item bank consisted of 121 items, for which item parameters were calculated using a two-parameter logistic model (2PLM). The CAT simulations, based on both simulated and real data, revealed sufficient marginal reliability (coefficient of marginal reliability above 0.750) and criterion-related validity (Pearson's correlations between CAT scores and aggregate scores of the SCTCMU exceeding 0.850). Discussion: In national-level medical education assessment, there is an increasing need for concise yet valid evaluations of candidates' competence of health professionals. The CAT developed in this study demonstrated satisfactory reliability and validity, offering a more efficient assessment of candidates' competence of health professionals. The psychometric properties of the CAT could lead to shorter test durations, reduced information loss, and a decreased testing burden for participants.

The NEAT Equating Via Chaining Random Forests in the Context of Small Sample Sizes: A Machine-Learning Method.

The part of responses that is absent in the nonequivalent groups with anchor test (NEAT) design can be managed to a planned missing scenario. In the context of small sample sizes, we present a machine learning (ML)-based imputation technique called chaining random forests (CRF) to perform equating tasks within the NEAT design. Specifically, seven CRF-based imputation equating methods are proposed based on different data augmentation methods. The equating performance of the proposed methods is examined through a simulation study. Five factors are considered: (a) test length (20, 30, 40, 50), (b) sample size per test form (50 versus 100), (c) ratio of common/anchor items (0.2 versus 0.3), and (d) equivalent versus nonequivalent groups taking the two forms (no mean difference versus a mean difference of 0.5), and (e) three different types of anchors (random, easy, and hard), resulting in 96 conditions. In addition, five traditional equating methods, (1) Tucker method; (2) Levine observed score method; (3) equipercentile equating method; (4) circle-arc method; and (5) concurrent calibration based on Rasch model, were also considered, plus seven CRF-based imputation equating methods for a total of 12 methods in this study. The findings suggest that benefiting from the advantages of ML techniques, CRF-based methods that incorporate the equating result of the Tucker method, such as IMP_total_Tucker, IMP_pair_Tucker, and IMP_Tucker_cirlce methods, can yield more robust and trustable estimates for the "missingness" in an equating task and therefore result in more accurate equated scores than other counterparts in short-length tests with small samples.

Cost-Effectiveness Analysis in Performance Assessments: A Case Study of the Objective Structured Clinical Examination.

Medical education assessments are becoming more complex, resulting in the inappropriateness of traditional methods primarily consisting of direct observations, oral examinations, and multiple-choice tests. Advancements in research methods have led to the formation of new modalities, namely performance assessments, which are, on the other hand, always costly in development and implementation. Proposing using the Program Effectiveness and Cost Generalization flow within an assessment context (PRECOG-A), this brief report explores the real financial cost drivers associated with an assessment case in the context of medical education, presents the steps in bridging the effectiveness with its psychometric properties via cost-effectiveness analysis, and evaluates the two-side outcomes for further evaluation decision-making. Referentially providing a framework to investigators and researchers, the illustration of PRECOG-A in this study outlines instructional guidelines for conducting cost-effectiveness analysis in a performance assessment.

The reciprocal longitudinal relationship between body mass index and subjective well-being in China.

The relationship between body mass index (BMI) and subjective well-being (SWB) has been widely discussed in empirical literature. However, most previous studies were cross-sectional and inadequate to assess directionality. The current study examines the prospective relationship between BMI and SWB by analyzing nationally representative longitudinal data from 2009 to 2015. Eligible subjects were those who participated in a three-wave measurement (n = 18,443, age 13-99 years). We used a random-intercept cross-lagged panel model (RI-CLPM) to investigate the dynamic relationship between BMI and SWB, controlling for covariate effects. Results indicated that BMI and SWB were positively associated at the population level, and there was a reciprocal relationship at the individual level. An individual's BMI levels positively influenced his or her subsequent SWB values, with effects ranging from 0.15 (95% CI: 0.09-0.22) to 0.16 (95% CI: 0.09-0.24). At the same time, an individual's SWB levels positively influenced subsequent BMI values, with effects ranging from 0.12 (95% CI: 0.08-0.17) to 0.17 (95% CI: 0.10-0.24). This mutual reinforcement process supports the "jolly fat" hypothesis from a longitudinal perspective, implying that the psychosocial burden of excess weight may not hold in China. The implications are that mental health interventions could help prevent the risk of being underweight, and nutritional care would contribute to the creation of a happier society.

Psychometric evaluation of a national exam for clinical undergraduates.

Background: As a high-stake national-level examination administrated before students' clerkship in China, the Standardized Competence Test for Clinical Medicine Undergraduates (SCTCMU) has received much attention from the relevant educational departments and society at large. Investigating SCTCMU's validity and reliability is critical to the national healthcare profession education. Materials and methods: Raw responses from SCTCMU, answered by 44,332 examines of 4th-year undergraduate medical students on 300 multiple-choice items, were used to evaluate the quality of the exam via psychometric methods based on item response theory (IRT). The core assumptions and model-data fit of IRT models were evaluated, as well as the item properties and information functions. Results: The IRT models were fitted to the observed assessment data, where all the required assumptions were met. The IRT analysis showed that most items had acceptable psychometric properties, and the passing score was located close to the lowest measurement error computed from the model outcomes. Conclusion: The proposed modern psychometric method provides a practical and informative approach to calibrating and analyzing medical education assessments. This work showcases a realistic depiction of the IRT analysis process and therefore facilitates the work of applied researchers wanting to conduct, interpret, and report IRT analyses on medical assessments.

Modeling the change trajectory of sleep duration and its associated factors: based on an 11-year longitudinal survey in China.

BACKGROUND: Sleep duration is a vital public health topic, yet most existing studies have been limited to cross-sectional surveys or inconsistent classifications of sleep duration categories, and few characterized its continuous development process. The current study aimed to depict its change trajectory in the general population and identify associated factors from a dynamic perspective. METHODS: A total of 3788 subjects (45.4% male, mean age 46.72 ± 14.89 years) from the China Health and Nutrition Survey were recruited, and their daily sleep duration for five consecutive measurements from 2004 to 2015 was recorded. We adopted latent growth modelling to establish systematic relations between sleep duration and time. Participants' sociodemographic characteristics, lifestyle, and health factors were taken as covariates. RESULTS: The change in sleep duration could be depicted by a linear decreasing trajectory with the mean yearly decrease at 2.5 min/day. The trajectory did not differ by residence, BMI category, chronic disease situation, smoking status, or drinking status. Moreover, there were sex and age differences in the trajectory, and females and those under 30 were prone to larger decrease rates. CONCLUSION: The quantified yearly change in sleep duration provided insights for the prediction and early warning of insufficient sleep. Public health interventions focusing on slowing down the decrease rates among females and young individuals are warranted.