Detecting Preknowledge Cheating via Innovative Measures: A Mixture Hierarchical Model for Jointly Modeling Item Responses, Response Times, and Visual Fixation Counts.

Preknowledge cheating jeopardizes the validity of inferences based on test results. Many methods have been developed to detect preknowledge cheating by jointly analyzing item responses and response times. Gaze fixations, an essential eye-tracker measure, can be utilized to help detect aberrant testing behavior with improved accuracy beyond using product and process data types in isolation. As such, this study proposes a mixture hierarchical model that integrates item responses, response times, and visual fixation counts collected from an eye-tracker (a) to detect aberrant test takers who have different levels of preknowledge and (b) to account for nuances in behavioral patterns between normally-behaved and aberrant examinees. A Bayesian approach to estimating model parameters is carried out via an MCMC algorithm. Finally, the proposed model is applied to experimental data to illustrate how the model can be used to identify test takers having preknowledge on the test items.

Students' acceptance of online learning in developing nations: scale development and validation.

Most education systems were severely impacted by the COVID-19 pandemic, and as a result, learning shifted from face-to-face to online in higher education institutions. This unprecedented shift in the learning environment caused substantial challenges for students. The situation was more severe in developing nations such as Bangladesh, which lacked available resources and knowledge of online education to support their students. Recent studies suggest that students resisted online learning in various developing nations. To support online learning in developing nations, this study develops the Acceptance of Online Learning (AOL) scale comprised of both institutional and student-related factors. To validate the AOL scale, the study collected data from 441 students across 30 higher education institutions in Bangladesh to determine the factors explaining students' acceptance of online learning using AOL measurements. The results showed that institutional factors, such as technological sufficiency, instructor efficiency, and technical assistance play significant roles in students' acceptance of online learning in developing nations. These findings will help education policymakers and administrators in developing nations to assess the needs of students with respect to online learning, and the AOL scale will assist in the evaluation of students' acceptance of online learning in these nations.

Health and Fitness Comparisons Among Middle-Aged and Older Adults with Long-Term Participation in Four Different Sports.

Physical activity is known to confer numerous health benefits. However, few studies have assessed the prolonged impact of participation in different sports on health and fitness. The purpose of this cross-sectional study was to compare the impact of long-term participation (i.e., ≥9 years) in four different sports, including two traditional Chinese (Tai Chi; diabolo) and two modern sports (aerobics; track and field [TF]) on health and fitness measures among middle-aged and older adults. Participants (n=252, 56.6±8.5y, 66% female) completed the following measures: height, weight, BMI, waist circumference, waist-hip ratio (WHR), body fat%, resting blood pressure (BP) and heart rate, vital capacity, grip strength, reaction time, flexibility, balance. Compared to Tai Chi, aerobics was associated with lower systolic BP and fat%, and greater vital capacity, while TF was associated with lower systolic BP, shorter reaction time, greater vital capacity and better balance (p-values<0.05). Diabolo was associated with lower WHR compared to aerobics and Tai Chi, and greater vital capacity than Tai Chi (p-values<0.05). No significant differences were observed across sports in other measures. Long-term participation in four sports might have different effects on BP, vital capacity, balance, reaction time, body fat, and central adiposity. Individuals who prefer to choose traditional sports (Tai Chi, diabolo) may anticipate similar long-term effects on resting heart rate, BMI, muscle mass, and grip strength compared to those who perform modern sports (aerobics, TF). Information presented in this study may be valuable when designing population, group and individually-tailored PA recommendations in culturally diverse individuals.

Bridging Models of Biometric and Psychometric Assessment: A Three-Way Joint Modeling Approach of Item Responses, Response Times, and Gaze Fixation Counts.

Recently, joint models of item response data and response times have been proposed to better assess and understand test takers' learning processes. This article demonstrates how biometric information such as gaze fixation counts obtained from an eye-tracking machine can be integrated into the measurement model. The proposed joint modeling framework accommodates the relations among a test taker's latent ability, working speed and test engagement level via a person-side variance-covariance structure, while simultaneously permitting the modeling of item difficulty, time-intensity, and the engagement intensity through an item-side variance-covariance structure. A Bayesian estimation scheme is used to fit the proposed model to data. Posterior predictive model checking based on three discrepancy measures corresponding to various model components are introduced to assess model-data fit. Findings from a Monte Carlo simulation and results from analyzing experimental data demonstrate the utility of the model.

Effects of Compounded Nonnormality of Residuals in Hierarchical Linear Modeling.

While hierarchical linear modeling is often used in social science research, the assumption of normally distributed residuals at the individual and cluster levels can be violated in empirical data. Previous studies have focused on the effects of nonnormality at either lower or higher level(s) separately. However, the violation of the normality assumption simultaneously across all levels could bias parameter estimates in unforeseen ways. This article aims to raise awareness of the drawbacks associated with compounded nonnormality residuals across levels when the number of clusters range from small to large. The effects of the breach of the normality assumption at both individual and cluster levels were explored. A simulation study was conducted to evaluate the relative bias and the root mean square of the model parameter estimates by manipulating the normality of the data. The results indicate that nonnormal residuals have a larger impact on the random effects than fixed effects, especially when the number of clusters and cluster size are small. In addition, for a simple random-effects structure, the use of restricted maximum likelihood estimation is recommended to improve parameter estimates when compounded residuals across levels show moderate nonnormality, with a combination of small number of clusters and a large cluster size.

Assessing Preknowledge Cheating via Innovative Measures: A Multiple-Group Analysis of Jointly Modeling Item Responses, Response Times, and Visual Fixation Counts.

Many approaches have been proposed to jointly analyze item responses and response times to understand behavioral differences between normally and aberrantly behaved test-takers. Biometric information, such as data from eye trackers, can be used to better identify these deviant testing behaviors in addition to more conventional data types. Given this context, this study demonstrates the application of a new method for multiple-group analysis that concurrently models item responses, response times, and visual fixation counts collected from an eye-tracker. It is hypothesized that differences in behavioral patterns between normally behaved test-takers and those who have different levels of preknowledge about the test items will manifest in latent characteristics of the different data types. A Bayesian estimation scheme is used to fit the proposed model to experimental data and the results are discussed.

Variable Speed Across Dimensions of Ability in the Joint Model for Responses and Response Times.

Working speed as a latent variable reflects a respondent's efficiency to apply a specific skill, or a piece of knowledge to solve a problem. In this study, the common assumption of many response time models is relaxed in which respondents work with a constant speed across all test items. It is more likely that respondents work with different speed levels across items, in specific when these items measure different dimensions of ability in a multidimensional test. Multiple speed factors are used to model the speed process by allowing speed to vary across different domains of ability. A joint model for multidimensional abilities and multifactor speed is proposed. Real response time data are analyzed with an exploratory factor analysis as an example to uncover the complex structure of working speed. The feasibility of the proposed model is examined using simulation data. An empirical example with responses and response times is presented to illustrate the proposed model's applicability and rationality.

Joint Modeling of Compensatory Multidimensional Item Responses and Response Times.

Computer-based testing (CBT) is becoming increasingly popular in assessing test-takers' latent abilities and making inferences regarding their cognitive processes. In addition to collecting item responses, an important benefit of using CBT is that response times (RTs) can also be recorded and used in subsequent analyses. To better understand the structural relations between multidimensional cognitive attributes and the working speed of test-takers, this research proposes a joint-modeling approach that integrates compensatory multidimensional latent traits and response speediness using item responses and RTs. The joint model is cast as a multilevel model in which the structural relation between working speed and accuracy are connected through their variance-covariance structures. The feasibility of this modeling approach is investigated via a Monte Carlo simulation study using a Bayesian estimation scheme. The results indicate that integrating RTs increased model parameter recovery and precision. In addition, Program of International Student Assessment (PISA) 2015 mathematics standard unit items are analyzed to further evaluate the feasibility of the approach to recover model parameters.

How Do Test Takers Interact With Simulation-Based Tasks? A Response-Time Perspective.

Many traditional educational assessments use multiple-choice items and constructed-response items to measure fundamental skills. Virtual performance assessments, such as game- or simulation-based assessments, are designed recently in the field of educational measurement to measure more integrated skills through the test takers' interactive behaviors within an assessment in a virtual environment. This paper presents a systematic timing study based on data collected from a simulation-based task designed recently at Educational Testing Service. The study is intended to understand the response times in complex simulation-based tasks so as to shed light on possible ways of leveraging response time information in designing, assembling, and scoring of simulation-based tasks. To achieve this objective, a series of five analyses were conducted to first understand the statistical properties of the timing data, and then investigate the relationship between the timing patterns and the test takers' performance on the items/task, demographics, motivation level, personality, and test-taking behaviors through use of different statistical approaches. We found that the five analyses complemented each other and revealed different useful timing aspects of this test-taker sample's behavioral features in the simulation-based task. The findings were also compared with notable existing results in the literature related to timing data.

Negative Binomial Models for Visual Fixation Counts on Test Items.

With the development of technology-enhanced learning platforms, eye-tracking biometric indicators can be recorded simultaneously with students item responses. In the current study, visual fixation, an essential eye-tracking indicator, is modeled to reflect the degree of test engagement when a test taker solves a set of test questions. Three negative binomial regression models are proposed for modeling visual fixation counts of test takers solving a set of items. These models follow a similar structure to the lognormal response time model and the two-parameter logistic item response model. The proposed modeling structures include individualized latent person parameters reflecting the level of engagement of each test taker and two item parameters indicating the visual attention intensity and discriminating power of each test item. A Markov chain Monte Carlo estimation method is implemented for parameter estimation. Real data are fitted to the three proposed models, and the results are discussed.