Mattering in the community: Domain and demographic differences in a US sample.

Mattering is defined as experiences of feeling valued and adding value in different domains of life: self, relationships, work, and community. Mattering is a construct with great relevance across psychological and social issues. Research has suggested there may be value in understanding group differences in mattering. Following the recent validation of a scale which measures mattering across multiple domains of life (MIDLS), the present study analyzed a representative US sample to identify demographic group differences in domain-specific mattering. Despite the presence of few differences in Overall Mattering, significant differences were found among all domains and between groups for each demographic variable. Overall, high incomes, advanced degrees, and employment were most consistently associated with higher mattering across domains. In addition, individuals across demographic groups and domains were more likely to report adding value than feeling valued. Age, gender, ethnicity, and marital status correlations were found in certain domains. These results demonstrate the value of a multidimensional conception of mattering and provide initial insight into demographic differences in mattering in a United States, English-speaking sample.

Assessing multidimensional mattering: Development and exploratory validation of the Mattering in Domains of Life Scale (MIDLS).

Mattering, defined as feeling valued and adding value, is a basic psychological need with significant explanatory power. Although several specific measures have been introduced to assess the construct, no integrated, multidimensional measure exists. This limits the ability of researchers to investigate mattering in ecological contexts. This paper seeks to address this gap by introducing the Mattering in Domains of Life Scale (MIDLS) and evaluating its internal structure and convergent validity. Using data from a single, large US representative sample, the analysis validated the overall hypothesized factor structure, consisting of feeling valued and adding value across the personal, interpersonal, occupational, and community domains, as well as overall mattering. Convergent validity was assessed by comparing the various MIDLS subscales to eight established scales. Overall results provide evidence that the MIDLS is a valid scale measuring an essentially unidimensional construct composed of eight subdomains.

Improving the measurement of food insecurity among people with HIV in South Africa: a psychometric examination.

OBJECTIVE: Food insecurity is a structural barrier to HIV care in peri-urban areas in South Africa (SA), where approximately 80 % of households are moderately or severely food insecure. For people with HIV (PWH), food insecurity is associated with poor antiretroviral therapy adherence and survival rates. Yet, measurement of food insecurity among PWH remains a challenge. DESIGN: The current study examines the factor structure of the nine-item Household Food Insecurity Access Scale (HFIAS, isiXhosa-translated) among PWH in SA using a restrictive bifactor model. SETTING: Primary care clinics in Khayelitsha, a peri-urban settlement in Cape Town, SA. PARTICIPANTS: Participants (n 440) were PWH who received HIV care in Khayelitsha screening for a clinical trial. Most were categorised as severely (n 250, 56·82 %) or moderately (n 107, 24·32 %) food insecure in the past 30 d. RESULTS: Revised parallel analysis suggested a three-factor structure, which was inadmissible. A two-factor structure was examined but did not adequately fit the data. A two-factor restrictive bifactor model was examined, such that all items loaded on a general factor (food insecurity) and all but two items loaded on one of two specific additional factors, which adequately fit the data (comparative fit index = 0·995, standardised root mean square residual = 0·019). The two specific factors identified were: anxiety/insufficient quality and no food intake. Reliability was adequate (ω = 0·82). CONCLUSIONS: Results supported the use of a total score, and identified two specific factors of the HFIAS, which may be utilised in future research and intervention development. These findings help identify aspects of food insecurity that may drive relationships between the construct and important HIV-related variables.

Illustrating the Applicability of IRT to Implementation Science: Examining an Instrument of Therapist Attitudes.

Pragmatic instruments with psychometric support are important to advance dissemination and implementation (D&I) research, but few well-researched D&I instruments exist. Item response theory (IRT), an approach that is underutilized in D&I, can help with the development of actionable and brief instruments. This paper provides an overview of IRT for D&I researchers and examines an instrument of therapist attitudes using IRT measurement models. Eight items of the Attitudes Towards Individualized Assessment-Monitoring and Feedback (AIA-MF) Clinical Utility scale were fit to the Graded Response Model in a national sample of master's level therapists. Various IRT model characteristics including item threshold and discrimination parameters, information, and item and person fit were examined. Discrimination and thresholds parameters showed significant variability across the eight items. Item information curves also showed that each item contributed variably to the total test information, suggesting that items 4 and 5 reliably measure therapist attitudes across the latent continuum and items 3 and 6 warrant further investigation. Results suggest that IRT models can help D&I researchers examine existing instruments with greater specificity than traditional measurement methods, thus increasing measurement precision while lowering response burden, both important considerations for the field.

Validation of Response Similarity Analysis for the Detection of Academic Cheating: An Experimental Study.

The purpose of the present study was to evaluate various analytical means to detect academic cheating in an experimental setting. The omega index was compared and contrasted given a gold criterion of academic cheating which entailed a discrepant score between two administrations using an experimental study with real test takers. Participants were 164 elementary school students who were administered a mathematics exam followed by an equivalent mock exam under conditions of strict and relaxed, invigilation, respectively. Discrepant scores were defined as exceeding 7 responses in any direction (correct or incorrect), based on what was expected due to chance. Results indicated that the omega index was successful in capturing more than 39% of the cases who exceeded the conventional plus or minus 7 discrepancy criteria. It is concluded that the response similarity analysis may be an important tool in detecting academic cheating.

Evaluating the Sampling Performance of Exploratory and Cross-Validated DETECT Procedure with Imperfect Models.

Among the methods proposed for identifying the number of latent traits in multidimensional IRT models, DETECT has attracted the attention of both methodologists and applied researchers as a nonparametric counterpart to other procedures. The current study investigated the overall performance of the DETECT procedure and its outcomes using a real-data sampling design recommended by MacCallum (2003) and compared the results from a purely simulated data set that was generated with a well-specified "perfect" model. The comparison revealed that the sampling behavior of the maximized DETECT value and R-ratio statistics was quite robust to minor factors and other model misspecifications that potentially exist in the real data set, as there were negligible differences between the results of the real and simulated data sets. Item classification accuracy was also nearly identical for the real and simulated data sets. The accuracy of the identified number of dimensions reported by DETECT was the only outcome with an obvious difference between the purely simulated data set and the real data set. While the difference was small for smaller sample sizes, the identified number of dimensions was more accurate for larger sample sizes when the population data set was purely simulated. In many instances, exploratory DETECT analysis outperformed the cross-validated DETECT analysis in terms of overall accuracy.

Rotation to a partially specified target matrix in exploratory factor analysis in practice.

The purpose of the present study was to explore the influence of the number of targets specified on the quality of exploratory factor analysis solutions with a complex underlying structure and incomplete substantive measurement theory. We extended previous research in this area by (a) exploring this phenomenon in situations in which both the common factor model and the targeted pattern matrix contained specification errors and (b) comparing the performance of target rotation to an easier-to-use default rotation criterion (i.e., geomin) under conditions commonly observed in practice. A Monte Carlo study manipulated target error, number of targets, model error, overdetermination, communality, and sample size. Outcomes included bias (i.e., accuracy) and variability (i.e., stability) with regard to the rotated pattern matrix. The effects of target error were negligible for both accuracy and stability, whereas small effects were observed for the number of targets for both outcomes. Further, target rotation outperformed geomin rotation with regard to accuracy but generally performed worse than geomin rotation with regard to stability. These findings underscore the potential importance (or caution, in the case of stability) of using extant, even if incomplete and somewhat inaccurate, substantive measurement theory to inform the rotation criterion in a nonmechanical way.

A Comparative Study of Item Response Theory Models for Mixed Discrete-Continuous Responses.

Language proficiency assessments are pivotal in educational and professional decision-making. With the integration of AI-driven technologies, these assessments can more frequently use item types, such as dictation tasks, producing response features with a mixture of discrete and continuous distributions. This study evaluates novel measurement models tailored to these unique response features. Specifically, we evaluated the performance of the zero-and-one-inflated extensions of the Beta, Simplex, and Samejima's Continuous item response models and incorporated collateral information into the estimation using latent regression. Our findings highlight that while all models provided highly correlated results regarding item and person parameters, the Beta item response model showcased superior out-of-sample predictive accuracy. However, a significant challenge was the absence of established benchmarks for evaluating model and item fit for these novel item response models. There is a need for further research to establish benchmarks for evaluating the fit of these innovative models to ensure their reliability and validity in real-world applications.

A comparison of two estimation algorithms for Samejima's continuous IRT model.

This study compares two algorithms, as implemented in two different computer softwares, that have appeared in the literature for estimating item parameters of Samejima's continuous response model (CRM) in a simulation environment. In addition to the simulation study, a real-data illustration is provided, and CRM is used as a potential psychometric tool for analyzing measurement outcomes in the context of curriculum-based measurement (CBM) in the field of education. The results indicate that a simplified expectation-maximization (EM) algorithm is as effective and efficient as the traditional EM algorithm for estimating the CRM item parameters. The results also show promise for using this psychometric model to analyze CBM outcomes, although more research is needed in order to recommend CRM as a standard practice in the CBM context.

Differential Item Functioning Effect Size From the Multigroup Confirmatory Factor Analysis for a Meta-Analysis: A Simulation Study.

This study presents a new approach to synthesizing differential item functioning (DIF) effect size: First, using correlation matrices from each study, we perform a multigroup confirmatory factor analysis (MGCFA) that examines measurement invariance of a test item between two subgroups (i.e., focal and reference groups). Then we synthesize, across the studies, the differences in the estimated factor loadings between the two subgroups, resulting in a meta-analytic summary of the MGCFA effect sizes (MGCFA-ES). The performance of this new approach was examined using a Monte Carlo simulation, where we created 108 conditions by four factors: (1) three levels of item difficulty, (2) four magnitudes of DIF, (3) three levels of sample size, and (4) three types of correlation matrix (tetrachoric, adjusted Pearson, and Pearson). Results indicate that when MGCFA is fitted to tetrachoric correlation matrices, the meta-analytic summary of the MGCFA-ES performed best in terms of bias and mean square error values, 95% confidence interval coverages, empirical standard errors, Type I error rates, and statistical power; and reasonably well with adjusted Pearson correlation matrices. In addition, when tetrachoric correlation matrices are used, a meta-analytic summary of the MGCFA-ES performed well, particularly, under the condition that a high difficulty item with a large DIF was administered to a large sample size. Our result offers an option for synthesizing the magnitude of DIF on a flagged item across studies in practice.

A Finite Mixture Item Response Theory Model for Continuous Measurement Outcomes.

A mixture extension of Samejima's continuous response model for continuous measurement outcomes and its estimation through a heuristic approach based on limited-information factor analysis is introduced. Using an empirical data set, it is shown that two groups of respondents that differ both qualitatively and quantitatively in their response behavior can be revealed. In addition to the real data application, the effectiveness of the heuristic estimation approach under real data analytic conditions was examined through a Monte Carlo simulation study. The results showed that the heuristic estimation approach provided reliable parameter estimates and the model successfully converged above 80% when the sample size was 250 and above 90% when the sample size was 500 or 1,000 for most conditions.

Infant Movement Response to Auditory Rhythm.

Rhythmic entrainment occurs when an auditory rhythm drives an internal movement oscillator, thus providing a continuous time reference that improves temporal and spatial movement parameters. Entrainment processes and outcomes are well known for adults, but research is lacking for infants who might benefit from diagnosis and treatment of irregular rhythms within biological, sensorimotor, cognitive, and social domains. The present study used a combination of inertial measurement units and custom-made software to determine the amount, tempo, and regularity of movement in 28 infants aged 6-10 months while they were exposed to silence, an irregular rhythmic cue, or a regular rhythmic cue with tempo changes. We also assessed changes in the infants' movement parameters following a one-week rhythm training protocol. While results revealed no significant effect of auditory condition on amount or tempo of movement, infant movement was significantly more regular when infants were exposed to 120 bpm (beats per minute) than to an irregular rhythmic cue or a 10% faster rhythmic cue (132 bpm). Infants showed no notable changes in movement amount, tempo, or regularity following one week of training involving auditory and physical rhythm. Overall, infants seem to engage in spontaneous movements with or without auditory rhythm but may not show tempo sensitivity through their movements. Increased movement regularity suggests that 120 bpm may be a preferred tempo for infants, at which they are more likely to demonstrate well-timed movements that may reflect interval entrainment. Infants' auditory-motor systems appear not to respond to a 1-week rhythm training protocol.

Detecting Examinees With Item Preknowledge in Large-Scale Testing Using Extreme Gradient Boosting (XGBoost).

Researchers frequently use machine-learning methods in many fields. In the area of detecting fraud in testing, there have been relatively few studies that have used these methods to identify potential testing fraud. In this study, a technical review of a recently developed state-of-the-art algorithm, Extreme Gradient Boosting (XGBoost), is provided and the utility of XGBoost in detecting examinees with potential item preknowledge is investigated using a real data set that includes examinees who engaged in fraudulent testing behavior, such as illegally obtaining live test content before the exam. Four different XGBoost models were trained using different sets of input features based on (a) only dichotomous item responses, (b) only nominal item responses, (c) both dichotomous item responses and response times, and (d) both nominal item responses and response times. The predictive performance of each model was evaluated using the area under the receiving operating characteristic curve and several classification measures such as the false-positive rate, true-positive rate, and precision. For comparison purposes, the results from two person-fit statistics on the same data set were also provided. The results indicated that XGBoost successfully classified the honest test takers and fraudulent test takers with item preknowledge. Particularly, the classification performance of XGBoost was reasonably good when the response time information and item responses were both taken into account.

Acoustic parameters of infant-directed singing in mothers of infants with down syndrome.

This study compared the acoustic parameters and degree of perceived warmth in two types of infant-directed (ID) songs - the lullaby and the playsong - between mothers of infants with Down syndrome (DS) and mothers of typically-developing (TD) infants. Participants included mothers of 15 DS infants and 15 TD infants between 3 and 9 months of age. Each mother's singing voice was digitally recorded while singing to her infant and subjected to feature extraction and data mining. Mothers of DS infants and TD infants sang both lullabies and playsongs with similar frequency. In comparison with mothers of TD infants, mothers of DS infants used a higher maximum pitch and more key changes during playsong. Mothers of DS infants also took more time to establish a rhythmic structure in their singing. These differences suggest mothers are sensitive to the attentional and arousal needs of their DS infants. Mothers of TD infants sang with a higher degree of perceived warmth which does not agree with previous observations of "forceful warmth" in mothers of DS infants. In comparison with lullaby, all mothers sang playsong with higher overall pitch and slower tempo. Playsongs were also distinguished by higher levels of spectral centroid properties related to emotional expressivity, as well as higher degrees of perceived warmth. These similarities help to define specific song types, and suggest that all mothers sing in an expressive manner that can modulate infant arousal, including mothers of DS infants.

Classification Performance of Answer-Copying Indices Under Different Types of IRT Models.

Test fraud has recently received increased attention in the field of educational testing, and the use of comprehensive integrity analysis after test administration is recommended for investigating different types of potential test frauds. One type of test fraud involves answer copying between two examinees, and numerous statistical methods have been proposed in the literature to screen and identify unusual response similarity or irregular response patterns on multiple-choice tests. The current study examined the classification performance of answer-copying indices measured by the area under the receiver operating characteristic (ROC) curve under different item response theory (IRT) models (one- [1PL], two- [2PL], three-parameter [3PL] models, nominal response model [NRM]) using both simulated and real response vectors. The results indicated that although there is a slight increase in the performance for low amount of copying conditions (20%), when nominal response outcomes were used, these indices performed in a similar manner for 40% and 60% copying conditions when dichotomous response outcomes were utilized. The results also indicated that the performance with simulated response vectors was almost identically reproducible with real response vectors.

A Finite Mixture of Nonlinear Random Coefficient Models for Continuous Repeated Measures Data.

Nonlinear random coefficient models (NRCMs) for continuous longitudinal data are often used for examining individual behaviors that display nonlinear patterns of development (or growth) over time in measured variables. As an extension of this model, this study considers the finite mixture of NRCMs that combine features of NRCMs with the idea of finite mixture (or latent class) models. The efficacy of this model is that it allows the integration of intrinsically nonlinear functions where the data come from a mixture of two or more unobserved subpopulations, thus allowing the simultaneous investigation of intra-individual (within-person) variability, inter-individual (between-person) variability, and subpopulation heterogeneity. Effectiveness of this model to work under real data analytic conditions was examined by executing a Monte Carlo simulation study. The simulation study was carried out using an R routine specifically developed for the purpose of this study. The R routine used maximum likelihood with the expectation-maximization algorithm. The design of the study mimicked the output obtained from running a two-class mixture model on task completion data.

Fitting a linear-linear piecewise growth mixture model with unknown knots: A comparison of two common approaches to inference.

A linear-linear piecewise growth mixture model (PGMM) is appropriate for analyzing segmented (disjointed) change in individual behavior over time, where the data come from a mixture of 2 or more latent classes, and the underlying growth trajectories in the different segments of the developmental process within each latent class are linear. A PGMM allows the knot (change point), the time of transition from 1 phase (segment) to another, to be estimated (when it is not known a priori) along with the other model parameters. To assist researchers in deciding which estimation method is most advantageous for analyzing this kind of mixture data, the current research compares 2 popular approaches to inference for PGMMs: maximum likelihood (ML) via an expectation-maximization (EM) algorithm, and Markov chain Monte Carlo (MCMC) for Bayesian inference. Monte Carlo simulations were carried out to investigate and compare the ability of the 2 approaches to recover the true parameters in linear-linear PGMMs with unknown knots. The results show that MCMC for Bayesian inference outperformed ML via EM in nearly every simulation scenario. Real data examples are also presented, and the corresponding computer codes for model fitting are provided in the Appendix to aid practitioners who wish to apply this class of models.

Longitudinal mathematics development of students with learning disabilities and students without disabilities: a comparison of linear, quadratic, and piecewise linear mixed effects models.

Effective instructional planning and intervening rely heavily on accurate understanding of students' growth, but relatively few researchers have examined mathematics achievement trajectories, particularly for students with special needs. We applied linear, quadratic, and piecewise linear mixed-effects models to identify the best-fitting model for mathematics development over elementary and middle school and to ascertain differences in growth trajectories of children with learning disabilities relative to their typically developing peers. The analytic sample of 2150 students was drawn from the Early Childhood Longitudinal Study - Kindergarten Cohort, a nationally representative sample of United States children who entered kindergarten in 1998. We first modeled students' mathematics growth via multiple mixed-effects models to determine the best fitting model of 9-year growth and then compared the trajectories of students with and without learning disabilities. Results indicate that the piecewise linear mixed-effects model captured best the functional form of students' mathematics trajectories. In addition, there were substantial achievement gaps between students with learning disabilities and students with no disabilities, and their trajectories differed such that students without disabilities progressed at a higher rate than their peers who had learning disabilities. The results underscore the need for further research to understand how to appropriately model students' mathematics trajectories and the need for attention to mathematics achievement gaps in policy.

The effects of baseline estimation on the reliability, validity, and precision of CBM-R growth estimates.

This study examined the effect of baseline estimation on the quality of trend estimates derived from Curriculum Based Measurement of Oral Reading (CBM-R) progress monitoring data. The authors used a linear mixed effects regression (LMER) model to simulate progress monitoring data for schedules ranging from 6-20 weeks for datasets with high and low levels of residual variance (poor and good quality datasets respectively). Three observations per day for the first three days of data collection were generated for baseline estimation. As few as one and as many as nine observations were used to calculate baseline. The number of weeks of progress monitoring and the quality of the dataset were highly influential on the reliability, validity, and precision of simulated growth estimates. Results supported the use of using the median of three observations collected on the first day to estimate baseline, particularly when the first observation of that day systematically underestimated student performance. Collecting a large number of observations to estimate baseline does not appear to improve the quality of CBM-R growth estimates.

Curriculum-based measurement of oral reading: multi-study evaluation of schedule, duration, and dataset quality on progress monitoring outcomes.

Curriculum-Based Measurement of Oral Reading (CBM-R) is used to collect time series data, estimate the rate of student achievement, and evaluate program effectiveness. A series of 5 studies were carried out to evaluate the validity, reliability, precision, and diagnostic accuracy of progress monitoring across a variety of progress monitoring durations, schedules, and dataset quality conditions. A sixth study evaluated the relation between the various conditions of progress monitoring (duration, schedule, and dataset quality) and the precision of weekly growth estimates. Model parameters were derived from a large extant progress monitoring dataset of second-grade (n=1517) and third-grade students (n=1561) receiving supplemental reading intervention as part of a Tier II response-to-intervention program. A linear mixed effects regression model was used to simulate true and observed CBM-R progress monitoring data. The validity and reliability of growth estimates were evaluated with squared correlations between true and observed scores along with split-half reliabilities of observed scores. The precision of growth estimates were evaluated with root mean square error between true and observed estimates of growth. Finally, receiver operator curves were used to evaluate the diagnostic accuracy and optimize decision thresholds. Results are interpreted to guide progress monitoring practices and inform future research.