A sequential Moken scaling approach to evaluate response quality in survey research.

Careless responding, where participants do not fully engage with item content, is pervasive in survey research. Left undetected, carelessness can compromise the interpretation and use of survey results, including information about participant locations on the construct, item difficulty, and the psychometric quality of the instrument. We present and illustrate a sequential procedure for evaluating response quality in survey research using indicators from Mokken scale analysis (MSA). We use a real data illustration and a simulation study to compare a sequential procedure to a standalone procedure. We also consider how identifying and removing responses with evidence of poor measurement properties affects item quality indicators. Results suggest that the sequential procedure was effective in identifying potentially problematic response patterns that may not always be captured by traditional methods for identifying careless responders but was not always sensitive to specific carelessness patterns. We discuss implications for research and practice.

Disparities in physical activity between sexual minority and heterosexual women: A systematic review and meta-analysis.

Research examining potential differences in physical activity (PA) between sexual minority women (SMW) and heterosexual women have yielded inconsistent results. OBJECTIVE: Therefore, the purpose of this systematic review and meta-analysis is to examine potential differences in PA between SMW and heterosexual women and to identify potential moderators that may partially explain observed differences in PA. METHODS: All studies were peer reviewed, published in English, and included a continuous measure of PA for SMW and heterosexual women. A standardized mean difference effect size (ES) was used to compare groups, with random effects models used to estimate a mean ES and 95% CI using a 3-level meta-analysis model to adjust for the correlation between effects nested within studies. RESULTS: The cumulative results of 24 effects gathered from 7 studies indicated there was no difference in PA between SMW (n = 1619) and heterosexual women (n = 103,295) (ES = -0.038, 95%CI -0.179 to 0.102, p = 0.576). Despite no mean differences, moderate-high heterogeneity was observed, indicating that the results were not consistent across effects (I2 = 64.8%, Q23 = 36.7, p = 0.035). The difference in PA was associated with age (ß = -0.018, 95%CI -0.034 to -0.003, p = 0.022) and BMI (ß = -0.145, 95%CI -0.228 to -0.061, p = 0.002), with a quadratic relationship observed for both variables. CONCLUSIONS: Although the results of the current analysis did not indicate significant differences in PA behaviors between SMW and heterosexual women, age and BMI modify the association and are curvilinear in nature; such that smaller differences in PA were observed between SMW and heterosexual women when samples were middle-aged and overweight.

Validity of Smartphone Heart Rate Variability Pre- and Post-Resistance Exercise.

The aim was to examine the validity of heart rate variability (HRV) measurements from photoplethysmography (PPG) via a smartphone application pre- and post-resistance exercise (RE) and to examine the intraday and interday reliability of the smartphone PPG method. Thirty-one adults underwent two simultaneous ultrashort-term electrocardiograph (ECG) and PPG measurements followed by 1-repetition maximum testing for back squats, bench presses, and bent-over rows. The participants then performed RE, where simultaneous ultrashort-term ECG and PPG measurements were taken: two pre- and one post-exercise. The natural logarithm of the root mean square of successive normal-to-normal (R-R) differences (LnRMSSD) values were compared with paired-sample t-tests, Pearson product correlations, Cohen's d effect sizes (ESs), and Bland-Altman analysis. Intra-class correlations (ICC) were determined between PPG LnRMSSDs. Significant, small-moderate differences were found for all measurements between ECG and PPG: BasePre1 (ES = 0.42), BasePre2 (0.30), REPre1 (0.26), REPre2 (0.36), and REPost (1.14). The correlations ranged from moderate to very large: BasePre1 (r = 0.59), BasePre2 (r = 0.63), REPre1 (r = 0.63), REPre2 (r = 0.76), and REPost (r = 0.41)-all p < 0.05. The agreement for all the measurements was "moderate" (0.10-0.16). The PPG LnRMSSD exhibited "nearly-perfect" intraday reliability (ICC = 0.91) and "very large" interday reliability (0.88). The smartphone PPG was comparable to the ECG for measuring HRV at rest, but with larger error after resistance exercise.

Many-facet Dichotomous Rasch Model Analysis of the Modern Language Aptitude Test.

Researchers and practitioners have used the Modern Language Aptitude Test (MLAT) to assess language aptitude and identify possible language learning deficiencies in examinees since the 1950s. However, researchers have not assessed its psychometric properties using modern measurement theory methods. We use the dichotomous Rasch model to explore the psychometric properties of the MLAT, including data-model fit indices, item difficulty and student ability calibrations, reliability of separation, and differences in achievement across gender subgroups based on a sample of undergraduate and graduate university students (N=204). Our findings suggest that the MLAT has acceptable psychometric properties such that it can be meaningfully interpreted as a measure of language proficiency. Our findings confirm previous research that language performance across gender groups significantly differs. We found no significant interactions between gender subgroups and the difficulty of the five domains of the assessment. We discuss these results in terms of their implications for research and practice.

Exploring The Psychometric Properties of a Self-Efficacy Scale For High School Students.

In previous studies, researchers have focused on the development and interpretation of measurement tools related to self-efficacy. However, researchers have seldom investigated whether these instruments demonstrate acceptable psychometric properties, including similar item interpretations between subgroups of respondents. The purpose of this study was to explore the extent to which a self-efficacy measure has a consistent interpretation for two self-reported gender subgroups. The researchers utilized Rasch analysis to explore differences in item difficulty between the subgroups. Results suggested differences in item difficulty ordering for certain self-efficacy items. Implications for research and practice are discussed.

Changes in Heart Rate Variability and Fatigue Measures Following Moderate Load Resistance Exercise.

The purpose of this study was to determine the relationship between changes in heart rate variability (HRV), neuromuscular performance, and fatigue biomarkers in response to a resistance exercise bout. The root mean square of successive RR interval differences (RMSSD), neuromuscular performance - isometric handgrip (IHG), countermovement jump (CMJ), mean propulsive velocity (MPV) - metabolic stress (lactate [Lac]) and inflammation (interleukin-6 [IL-6]) were measured in 30 subjects who performed 6×10 back squat (BS), 3×10 bench press (BP), and 3×10 bent-over rows (BR) at 70% of 1-repetition maximum (1RM). The RMSSD, neuromuscular performance, and biomarkers were measured 10 min pre-exercise and 30 min post-exercise (Post30); HRV and Lac were also measured immediately post-exercise (Post0). Pre- versus post-exercise differences were evaluated using paired-samples t-tests. Pearson's correlations were used to determine the association between changes. With the exception of IL-6 (P=0.296) and MPVBP (P=0.678), LnRMSSD, neuromuscular performance, and metabolic stress were different post- compared to pre-exercise. We observed moderate associations between ΔLnRMSSD Post0 and ΔLac Post0 (r = -0.44) and ΔLac Post30 (r = -0.55), respectively. Practitioners should use multiple training load indicators to gain an accurate depiction of recovery.

Cross-Cultural Comparisons of School Leadership using Rasch Measurement.

School leadership influences school conditions and organizational climate; these conditions in turn impact student outcomes. Accordingly, examining differences in principals' perceptions of leadership activities within and across countries may provide insight into achievement differences. The major purpose of this study was to explore differences in the relative difficulty of principals' leadership activities across four countries that reflect Asian and North American national contexts: (1) Hong Kong SAR, (2) Chinese Taipei, (3) the United States, and (4) Canada. We also sought to illustrate the use of Rasch measurement theory as a modern measurement approach to exploring the psychometric properties of a leadership survey, with a focus on differential item functioning. We applied a rating scale formulation of the Many-facet Rasch model to principals' responses to the Leadership Activities Scale in order to examine the degree to which the overall ordering of leadership activities was invariant across the four countries. Overall, the results suggested that there were significant differences in the difficulty ordering of leadership activities across countries, and that these differences were most pronounced between the two continents. Implications are discussed for research and practice.

Student Perceptions of Grammar Instruction in Iranian Secondary Education: Evaluation of an Instrument using Rasch Measurement Theory.

Isolated and integrated grammar instruction are two approaches to grammar teaching that can be implemented within a form-focused instruction (FFI) framework. In both approaches, instructors primarily concentrate on meaning, and the difference is in the timing of instruction on specific language forms. In previous studies, researchers have observed that the match between teachers' and learners' beliefs related to the effectiveness of instructional approaches is an important component in predicting the success of grammar instruction. In this study, we report on the psychometric properties of a questionnaire designed to measure students' perceptions of isolated and integrated FFI taking place in Iranian secondary schools. The Iranian context is interesting with regard to approaches to grammar instruction in light of recent policy reforms that emphasize isolated FFI. Using a combination of principal components analysis and Rasch measurement theory techniques, we observed that Iranian students distinguish among the two forms of grammar instruction. Looking within each approach, we observed significant differences among individual students as well as differences in the difficulty for students to endorse different instructional activities related to both isolated and integrated instruction. Together, our findings highlight the importance of examining students' beliefs about the effectiveness of approaches to grammar instruction within different instructional contexts. We discuss implications for research and practice.

Using Repeated Ratings to Improve Measurement Precision in Incomplete Rating Designs.

When selecting a design for rater-mediated assessments, one important consideration is the number of raters who rate each examinee. In balancing costs and rater-coverage, rating designs are often implemented wherein only a portion of the examinees are rated by each judge, resulting in large amounts of missing data. One drawback to these sparse rating designs is the reduced precision of examinee ability estimates they provide. When increasing the number of raters per examinee is not feasible, another option may be to increase the number of ratings provided by each rater per examinee. This study applies a Rasch model to explore the effect of increasing the number of rating occasions used by raters to judge examinee proficiency. We used a simulation study to approximate a sparse but connected rater network with a sequentially increasing number of repeated ratings per examinee. The generated data were used to explore the influence of repeated ratings on the precision of rater, examinee, and task parameter estimates as measured by parameter standard errors, the correlation of sparse parameter estimates to true estimates, and the root mean square error of parameter estimates. Results suggest that increasing the number of rating occasions significantly improves the precision of examinee and rater parameter estimates. Results also suggest that parameter recovery levels of rater and task estimates are quite robust to reductions in the number of repeated ratings, although examinee parameter estimates are more sensitive to them. Implications for research and practice in the context of rater-mediated assessment designs are discussed.

Developing an Engineering Design Process Assessment using Mixed Methods.

Recent reforms in science education worldwide include an emphasis on engineering design as a key component of student proficiency in the Science, Technology, Engineering, and Mathematics disciplines. However, relatively little attention has been directed to the development of psychometrically sound assessments for engineering. This study demonstrates the use of mixed methods to guide the development and revision of K-12 Engineering Design Process (EDP) assessment items. Using results from a middle-school EDP assessment, this study illustrates the combination of quantitative and qualitative techniques to inform item development and revisions. Overall conclusions suggest that the combination of quantitative and qualitative evidence provides an in-depth picture of item quality that can be used to inform the revision and development of EDP assessment items. Researchers and practitioners can use the methods illustrated here to gather validity evidence to support the interpretation and use of new and existing assessments.

Examining the Psychometric Quality of Multiple-Choice Assessment Items using Mokken Scale Analysis.

The concept of invariant measurement is typically associated with Rasch measurement theory (Engelhard, 2013). Concerned with the appropriateness of the parametric transformation upon which the Rasch model is based, Mokken (1971) proposed a nonparametric procedure for evaluating the quality of social science measurement that is theoretically and empirically related to the Rasch model. Mokken's nonparametric procedure can be used to evaluate the quality of dichotomous and polytomous items in terms of the requirements for invariant measurement. Despite these potential benefits, the use of Mokken scaling to examine the properties of multiple-choice (MC) items in education has not yet been fully explored. A nonparametric approach to evaluating MC items is promising in that this approach facilitates the evaluation of assessments in terms of invariant measurement without imposing potentially inappropriate transformations. Using Rasch-based indices of measurement quality as a frame of reference, data from an eighth-grade physical science assessment are used to illustrate and explore Mokken-based techniques for evaluating the quality of MC items. Implications for research and practice are discussed.

Gendered language attitudes: exploring language as a gendered construct using Rasch measurement theory.

Gendered language attitudes (GLAs) are gender-based perceptions of language varieties based on connections between gender-related and linguistic characteristics of individuals, including the perception of language varieties as possessing degrees of masculinity and femininity. This study combines substantive theory about language learning and gender with a model based on Rasch measurement theory to explore the psychometric properties of a new measure of GLAs. Findings suggest that GLAs is a unidimensional construct and that the items used can be used to describe differences among students in terms of the strength of their GLAs. Implications for research, theory, and practice are discussed. Special emphasis is given to the teaching and learning of languages.

Detecting Rating Scale Malfunctioning With the Partial Credit Model and Generalized Partial Credit Model.

Rating scale analysis techniques provide researchers with practical tools for examining the degree to which ordinal rating scales (e.g., Likert-type scales or performance assessment rating scales) function in psychometrically useful ways. When rating scales function as expected, researchers can interpret ratings in the intended direction (i.e., lower ratings mean "less" of a construct than higher ratings), distinguish between categories in the scale (i.e., each category reflects a unique level of the construct), and compare ratings across elements of the measurement instrument, such as individual items. Although researchers have used these techniques in a variety of contexts, studies are limited that systematically explore their sensitivity to problematic rating scale characteristics (i.e., "rating scale malfunctioning"). I used a real data analysis and a simulation study to systematically explore the sensitivity of rating scale analysis techniques based on two popular polytomous item response theory (IRT) models: the partial credit model (PCM) and the generalized partial credit model (GPCM). Overall, results indicated that both models provide valuable information about rating scale threshold ordering and precision that can help researchers understand how their rating scales are functioning and identify areas for further investigation or revision. However, there were some differences between models in their sensitivity to rating scale malfunctioning in certain conditions. Implications for research and practice are discussed.

The Effects of Rating Designs on Rater Classification Accuracy and Rater Measurement Precision in Large-Scale Mixed-Format Assessments.

In standalone performance assessments, researchers have explored the influence of different rating designs on the sensitivity of latent trait model indicators to different rater effects as well as the impacts of different rating designs on student achievement estimates. However, the literature provides little guidance on the degree to which different rating designs might affect rater classification accuracy (severe/lenient) and rater measurement precision in both standalone performance assessments and mixed-format assessments. Using results from an analysis of National Assessment of Educational Progress (NAEP) data, we conducted simulation studies to systematically explore the impacts of different rating designs on rater measurement precision and rater classification accuracy (severe/lenient) in mixed-format assessments. The results suggest that the complete rating design produced the highest rater classification accuracy and greatest rater measurement precision, followed by the multiple-choice (MC) + spiral link design and the MC link design. Considering that complete rating designs are not practical in most testing situations, the MC + spiral link design may be a useful choice because it balances cost and performance. We consider the implications of our findings for research and practice.

Exploring the Influence of Item Characteristics in a Spatial Reasoning Task.

Well-designed spatial assessments can incorporate multiple sources of complexity that reflect important aspects of spatial reasoning. When these aspects are systematically included in spatial reasoning items, researchers can use psychometric models to examine the impact of each aspect on item difficulty. These methods can then help the researchers to understand the nature and development of spatial reasoning and can also inform the development of new items to better reflect the construct. This study investigated sources of item difficulty for object assembly (OA), a format for the assessment of spatial reasoning, by specifying nine item characteristics that were predicted to contribute to item difficulty. We used data from two focal samples including high-ability students in grades 3 to 7 and undergraduate students who responded to 15 newly developed OA items. Results from the linear logistic test model (LLTM) indicated that eight of the nine identified item characteristics significantly contributed to item difficulty. This suggests that an LLTM approach is useful in examining the contributions of various aspects of spatial reasoning to item difficulty and informing item development for spatial reasoning assessments.

Does Sparseness Matter? Examining the Use of Generalizability Theory and Many-Facet Rasch Measurement in Sparse Rating Designs.

Sparse rating designs, where each examinee's performance is scored by a small proportion of raters, are prevalent in practical performance assessments. However, relatively little research has focused on the degree to which different analytic techniques alert researchers to rater effects in such designs. We used a simulation study to compare the information provided by two popular approaches: Generalizability theory (G theory) and Many-Facet Rasch (MFR) measurement. In previous comparisons, researchers used complete data that were not simulated-thus limiting their ability to manipulate characteristics such as rater effects, and to understand the impact of incomplete data on the results. Both approaches provided information about rating quality in sparse designs, but the MFR approach highlighted rater effects related to centrality and bias more readily than G theory.

Comparing Person-Fit and Traditional Indices Across Careless Response Patterns in Surveys.

Methods to identify carelessness in survey research can be valuable tools in reducing bias during survey development, validation, and use. Because carelessness may take multiple forms, researchers typically use multiple indices when identifying carelessness. In the current study, we extend the literature on careless response identification by examining the usefulness of three item-response theory-based person-fit indices for both random and overconsistent careless response identification: infit MSE outfit MSE, and the polytomous lz statistic. We compared these statistics with traditional careless response indices using both empirical data and simulated data. The empirical data included 2,049 high school student surveys of teaching effectiveness from the Network for Educator Effectiveness. In the simulated data, we manipulated type of carelessness (random response or overconsistency) and percent of carelessness present (0%, 5%, 10%, 20%). Results suggest that infit and outfit MSE and the lz statistic may provide complementary information to traditional indices such as LongString, Mahalanobis Distance, Validity Items, and Completion Time. Receiver operating characteristic curves suggested that the person-fit indices showed good sensitivity and specificity for classifying both over-consistent and under-consistent careless patterns, thus functioning in a bidirectional manner. Carelessness classifications based on low fit values correlated with carelessness classifications from LongString and completion time, and classifications based on high fit values correlated with classifications from Mahalanobis Distance. We consider implications for research and practice.

Acute Responses to High-Intensity Back Squats with Bilateral Blood Flow Restriction.

This study examined the acute effects of high-intensity resistance exercise with blood flow restriction (BFR) on performance and fatigue, metabolic stress, and markers of inflammation (interleukin-6 (IL-6)), muscle damage (myoglobin), angiogenesis (vascular endothelial growth factor (VEGF)). Thirteen resistance-trained participants (four female, 24.8 ± 4.7 years) performed four sets of barbell back-squats (75% 1RM) to failure under two conditions: blood flow restriction (BFR, bilateral 80% occlusion pressure) and control (CTRL). Completed repetitions and pre-post-exercise changes in maximal voluntary isometric contractions, countermovement jump, barbell mean propulsive velocity, and surface electromyography were recorded. Pre-post blood lactate (BLa) and venous blood samples for analysis of IL-6, myoglobin, and VEGF were collected. Ratings of perceived exertion (RPE) and pain were recorded for each set. Fewer repetitions were performed during BFR (25.5 ± 9.6 reps) compared to CTRL (43.4 ± 14.2 reps, p < 0.001), with greater repetitions performed during sets 1, 2, and 4 (p < 0.05) in CTRL. Although RPE between conditions was similar across all sets (p > 0.05), pain was greater in BFR across all sets (p < 0.05). Post-exercise fatigue was comparable between conditions. BLa was significantly greater in CTRL compared to BFR at two minutes (p = 0.001) but not four minutes post-exercise (p = 0.063). IL-6 was significantly elevated following BFR (p = 0.011). Comparable increases in myoglobin (p > 0.05) and no changes in VEGF were observed (p > 0.05). BFR increases the rate of muscular fatigue during high-intensity resistance exercise and acutely enhances IL-6 response, with significantly less total work performed, but increases pain perception, limiting implementation.

Feeding Stressors and Resources Used by Caregivers of Children With Down Syndrome: A Qualitative Analysis.

BACKGROUND: Challenging eating behaviors or feeding difficulties, commonly displayed in children with Down syndrome (DS), may amplify perceived stress in caregivers. If caregivers lack resources on how to accommodate the needs of the child with DS, they may find feeding the child stressful and resort to negative coping strategies. OBJECTIVE: The aim of this study was to understand the feeding stressors, resources, and coping strategies used by caregivers of children with DS. DESIGN: A qualitative analysis of interview transcripts was undertaken, framed around the Transactional Model of Stress and Coping. PARTICIPANTS/SETTING: Between September to November 2021, 15 caregivers of children (aged 2 through 6 years) with DS, were recruited from 5 states located in the Southeast, Southwest, and West regions of the United States. ANALYSIS: Interviews were audio-recorded, transcribed verbatim, and analyzed using deductive thematic analysis and content analysis approaches. RESULTS: Thirteen caregivers reported increased stress around feeding the child with DS. Stressors identified included concern about adequacy of intake and challenges associated with feeding difficulties. Stress related to feeding was higher among caregivers whose child was learning a new feeding skill or in a transitional phase of feeding. Caregivers used both professional and interpersonal resources in addition to problem- and emotion-based coping strategies. CONCLUSIONS: Caregivers identified feeding as a stressful event with higher stress reported during transitional phases of feeding. Caregivers reported that speech, occupational, and physical therapists were beneficial resources to provide support for optimizing nutrition and skill development. These findings suggest that caregiver access to therapists and registered dietitian nutritionists is warranted.

Acute Blood Flow Responses to Varying Blood Flow Restriction Pressures in the Lower Limb.

The purpose of this study was to investigate lower limb blood flow responses under varying blood flow restriction (BFR) pressures based on individualized limb occlusion pressures (LOP) using a commonly used occlusion device. Twenty-nine participants (65.5% female, 23.8 ± 4.7 years) volunteered for this study. An 11.5cm tourniquet was placed around participants' right proximal thigh, followed by an automated LOP measurement (207.1 ± 29.4mmHg). Doppler ultrasound was used to assess posterior tibial artery blood flow at rest, followed by 10% increments of LOP (10-90% LOP) in a randomized order. All data were collected during a single 90-minute laboratory visit. Friedman's and one-way repeated-measures ANOVAs were used to examine potential differences in vessel diameter, volumetric blood flow (VolFlow), and reduction in VolFlow relative to rest (%Rel) between relative pressures. No differences in vessel diameter were observed between rest and all relative pressures (all p < .05). Significant reductions from rest in VolFlow and %Rel were first observed at 50% LOP and 40% LOP, respectively. VolFlow at 80% LOP, a commonly used occlusion pressure in the legs, was not significantly different from 60% (p = .88), 70% (p = .20), or 90% (p = 1.00) LOP. Findings indicate a minimal threshold pressure of 50%LOP may be required to elicit a significant decrease in arterial blood flow at rest when utilizing the 11.5cm Delfi PTSII tourniquet system.