Psychometric reliability, validity, and generalizability of 3MSE scores among American Indian adults: the Strong Heart Study.

OBJECTIVE: Modified Mini-Mental State Examination (3MSE) is often used to screen for dementia, but little is known about psychometric validity in American Indians. METHODS: We recruited 818 American Indians aged 65-95 for 3MSE examinations in 2010-2013; 403 returned for a repeat examination in 2017-2019. Analyses included standard psychometrics inferences for interpretation, generalizability, and extrapolation: factor analysis; internal consistency-reliability; test-retest score stability; multiple indicator multiple cause structural equation models. RESULTS: This cohort was mean age 73, majority female, mean 12 years education, and majority bilingual. The 4-factor and 2nd-order models fit best, with subfactors for orientation and visuo-construction (OVC), language and executive functioning (LEF), psychomotor and working memory (PMWM), verbal and episodic memory (VEM). Factor structure was supported for both research and clinical interpretation, and factor loadings were moderate to high. Scores were generally consistent over mean 7 years. Younger participants performed better in overall scores, but not in individual factors. Males performed better on OVC and LEF, females better on PMWM. Those with more education performed better on LEF and worse on OVC; the converse was true for bilinguals. All differences were significant, but small. CONCLUSION: These findings support use of 3MSE for individual interpretation in clinic and research among American Indians, with moderate consistency, stability, reliability over time. Observed extrapolations across age, sex, education, and bilingual groups suggest some important contextual differences may exist.

Contributions of causal reasoning to early scientific literacy.

Although early causal reasoning has been studied extensively, inconsistency in the tasks used to assess it has clouded our understanding of its structure, development, and relevance to broader developmental outcomes. The current research attempted to bring clarity to these questions by exploring patterns of performance across several commonly used measures of causal reasoning, and their relation to scientific literacy, in a sample of 3- to 5-year-old children from diverse backgrounds (N = 153). A longitudinal confirmatory factor analysis revealed that some measures of causal reasoning (counterfactual reasoning, causal learning, and causal inference), but not all of them (tracking cause-effect associations and resolving confounded evidence), assess a unidimensional factor and that this resulting factor was relatively stable across time. A cross-lagged panel model analysis revealed associations between causal reasoning and scientific literacy across each age tested. Causal reasoning and scientific literacy related to each other concurrently, and each predicted the other in subsequent years. These relations could not be accounted for by children's broader cognitive skills. Implications for early STEM (science, technology, engineering, and math) engagement and success are discussed.

A multi-centre international study of salivary hormone oestradiol and progesterone measurements in ART monitoring.

RESEARCH QUESTION: Ovarian stimulation during IVF cycles involves close monitoring of oestradiol, progesterone and ultrasound measurements of follicle growth. In contrast to blood draws, sampling saliva is less invasive. Here, a blind validation is presented of a novel saliva-based oestradiol and progesterone assay carried out in samples collected in independent IVF clinics. DESIGN: Concurrent serum and saliva samples were collected from 324 patients at six large independent IVF laboratories. Saliva samples were frozen and run blinded. A further 18 patients had samples collected more frequently around the time of HCG trigger. Saliva samples were analysed using an immunoassay developed with Salimetrics LLC. RESULTS: In total, 652 pairs of saliva and serum oestradiol were evaluated, with correlation coefficients ranging from 0.68 to 0.91. In the European clinics, a further 237 of saliva and serum progesterone samples were evaluated; however, the correlations were generally poorer, ranging from -0.02 to 0.22. In the patients collected more frequently, five out of 18 patients (27.8%) showed an immediate decrease in oestradiol after trigger. When progesterone samples were assessed after trigger, eight out of 18 (44.4%) showed a continued rise. CONCLUSIONS: Salivary oestradiol hormone testing correlates well to serum-based assessment, whereas progesterone values, around the time of trigger, are not consistent from patient to patient.

Quantifying Item Invariance for the Selection of the Least Biased Assessment.

An important aspect of educational and psychological measurement and evaluation of individuals is the selection of scales with appropriate evidence of reliability and validity for inferences and uses of the scores for the population of interest. One aspect of validity is the degree to which a scale fairly assesses the construct(s) of interest for members of different subgroups within the population. Typically, this issue is addressed statistically through assessment of differential item functioning (DIF) of individual items, or differential bundle functioning (DBF) of sets of items. When selecting an assessment to use for a given application (e.g., measuring intelligence), or which form of an assessment to use in a given instance, researchers need to consider the extent to which the scales work with all members of the population. Little research has examined methods for comparing the amount or magnitude of DIF/DBF present in two assessments when deciding which assessment to use. The current simulation study examines 6 different statistics for this purpose. Results show that a method based on the random effects item response theory model may be optimal for instrument comparisons, particularly when the assessments being compared are not of the same length.

The Effects of a Nutrition Media Literacy Intervention on Parents' and Youths' Communication about Food.

Interventions addressing links between media exposure and obesity risk for school-age youth have not explicitly addressed the role of family communication about media. Youths' influence attempts on parents to purchase advertised foods can create conflict and negatively affect parental food choices. This study tested whether a family-based media literacy curriculum improves parents' media management skills and decreases youths' susceptibility to appealing but unrealistic food marketing. A matched-group pretest/posttest field experiment of parent-youth dyads with control group (N = 100 dyads, youth M = 11 years of age) tested the six-session curriculum. Hypotheses were analyzed using a Bayesian structural equation model. The curriculum increased parents' active negative mediation to foster youths' critical thinking about food marketing, b* = 0.35, 95% CCI [0.17, 0.50], increased parent Efficacy for making healthy dietary changes for their families, b* = 0.59, 95% CCI [0.41, 0.75], and fostered family discussion about nutrition labels (total effect = 0.22). Additionally, cumulative influences of Perceived Desirability and Wishful Identification on youths' requests for marketed foods were reduced (total effect = 0.04). Media literacy education can empower parents and improve youths' critical thinking to reduce effects of food marketing on families and improve use of media to obtain nutrition information.

Methods for the Comparison of Differential Item Functioning across Assessments.

An important aspect of the educational and psychological evaluation of individuals is the selection of scales with appropriate evidence of reliability and validity for inferences and uses of the scores for the population of interest. One key aspect of validity is the degree to which a scale fairly assesses the construct(s) of interest for members of different subgroups within the population. Typically, this issue is addressed statistically through assessment of differential item functioning (DIF) of individual items, or differential test functioning (DTF) of sets of items within the same measure. When selecting an assessment to use for a given application (e.g., measuring intelligence), or which form of an assessment to use for a test administration, researchers need to consider the extent to which the scales work with all members of the population. Little research has examined methods for comparing the amount or magnitude of DIF/DTF present in two or more assessments when deciding which assessment to use. The current study made use of 7 different statistics for this purpose, in the context of intelligence testing. Results demonstrate that by using a variety of effect sizes, the researcher can gain insights into not only which scales may contain the least amount of DTF, but also how they differ with regard to the way in which the DTF manifests itself.

Ecological Momentary Assessment in Behavioral Research: Addressing Technological and Human Participant Challenges.

BACKGROUND: Ecological momentary assessment (EMA) assesses individuals' current experiences, behaviors, and moods as they occur in real time and in their natural environment. EMA studies, particularly those of longer duration, are complex and require an infrastructure to support the data flow and monitoring of EMA completion. OBJECTIVE: Our objective is to provide a practical guide to developing and implementing an EMA study, with a focus on the methods and logistics of conducting such a study. METHODS: The EMPOWER study was a 12-month study that used EMA to examine the triggers of lapses and relapse following intentional weight loss. We report on several studies that informed the implementation of the EMPOWER study: (1) a series of pilot studies, (2) the EMPOWER study's infrastructure, (3) training of study participants in use of smartphones and the EMA protocol and, (4) strategies used to enhance adherence to completing EMA surveys. RESULTS: The study enrolled 151 adults and had 87.4% (132/151) retention rate at 12 months. Our learning experiences in the development of the infrastructure to support EMA assessments for the 12-month study spanned several topic areas. Included were the optimal frequency of EMA prompts to maximize data collection without overburdening participants; the timing and scheduling of EMA prompts; technological lessons to support a longitudinal study, such as proper communication between the Android smartphone, the Web server, and the database server; and use of a phone that provided access to the system's functionality for EMA data collection to avoid loss of data and minimize the impact of loss of network connectivity. These were especially important in a 1-year study with participants who might travel. It also protected the data collection from any server-side failure. Regular monitoring of participants' response to EMA prompts was critical, so we built in incentives to enhance completion of EMA surveys. During the first 6 months of the 12-month study interval, adherence to completing EMA surveys was high, with 88.3% (66,978/75,888) completion of random assessments and around 90% (23,411/25,929 and 23,343/26,010) completion of time-contingent assessments, despite the duration of EMA data collection and challenges with implementation. CONCLUSIONS: This work informed us of the necessary preliminary steps to plan and prepare a longitudinal study using smartphone technology and the critical elements to ensure participant engagement in the potentially burdensome protocol, which spanned 12 months. While this was a technology-supported and -programmed study, it required close oversight to ensure all elements were functioning correctly, particularly once human participants became involved.

The Global Temperament Project: Parent-reported temperament in infants, toddlers, and children from 59 nations.

Data from 83,423 parent reports of temperament (surgency, negative affectivity, and regulatory capacity) in infants, toddlers, and children from 341 samples gathered in 59 countries were used to investigate the relations among culture, gender, and temperament. Between-nation differences in temperament were larger than those obtained in similar studies of adult personality, and most pronounced for negative affectivity. Nation-level patterns of negative affectivity were consistent across infancy, toddlerhood, and childhood, and patterns of regulatory capacity were consistent between infancy and toddlerhood. Nations that previously reported high extraversion, high conscientiousness, and low neuroticism in adults were found to demonstrate high surgency in infants and children, and countries reporting low adult openness and high adult neuroticism reported high temperamental negative affectivity. Negative affectivity was high in Southern Asia, Western Asia, and South America and low in Northern and Western Europe. Countries in which children were rated as high in negative affectivity had cultural orientations reflecting collectivism, high power distance, and short-term orientation. Surgency was high in Southeastern and Southern Asia and Southern Europe and low in Eastern Asian countries characterized by philosophies of long-term orientation. Low personal income was associated with high negative affectivity. Gender differences in temperament were largely consistent in direction with prior studies, revealing higher regulatory capacity in females than males and higher surgency in males than females, with these differences becoming more pronounced at later ages. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Psychometric Reliability, Validity, and Generalizability of MoCA in American Indian Adults: The Strong Heart Study.

Standardized neuropsychological instruments are used to evaluate cognitive impairment, but few have been psychometrically evaluated in American Indians. We collected Montreal Cognitive Assessment (MoCA) in 403 American Indians 70 to 95 years, as well as age, sex, education, bilingual status, depression symptoms, and other neuropsychological instruments. We evaluated inferences of psychometric validity, including scoring inference using confirmatory factor analysis and structural equation modeling, generalizability inference using reliability coefficient, and extrapolation inference by examining performance across different contexts and substrata. The unidimensional (total score) model had good fit criteria. Internal consistency reliability was high. MoCA scores were positively associated with crystallized cognition (ρ = 0.48, p < .001) and inversely with depression symptoms (ρ = -0.27, p < .001). Significant differences were found by education (d = 0.79, p < .05) depression (d = 0.484, p < .05), and adjudicated cognitive status (p = .0001) strata; however, MoCA was not sensitive or specific in discriminating cognitive impairment from normal cognition (area under the curve <0.5). MoCA scores had psychometric validity in older American Indians, but education and depression are important contextual features for score interpretability. Future research should evaluate cultural or community-specific adaptations, to improve test discriminability in this underserved population.

Psychometric Properties of Controlled Oral Word Association (COWA) Test and Associations With Education and Bilingualism in American Indian Adults: The Strong Heart Study.

The Controlled Oral Word Association (COWA) test is used to assess phonemic fluency and executive function. Formal validation of test scores is important for accurate cognitive evaluation. However, there is a dearth of psychometric validation among American Indian adults. Given high burden of dementia risk and key contextual factors associated with cognitive assessments, this represents a critical oversight. In a large, longitudinal population-based cohort study of adult American Indians, we examined several validity inferences for COWA, including scoring, generalization, and extrapolation inferences, by investigation of factor structure, internal consistency, test-retest reliability, and differential test functioning. We found adequate unidimensional model fit, with high factor loadings. Internal consistency reliability and test-retest reliability were 0.88 and 0.77, respectively, for the full group. COWA scores were lowest among the oldest, lowest education, bilingual speakers; group effects for sex and bilingual status were small; age effect was medium; and education effect was largest. However, Wide Range Achievement Test (WRAT) score effect was stronger than education effect, suggesting better contextualization may be needed. These results support interpretation of total COWA score, including across sex, age, or language use strata.

Cultural values and parental psychology: A multilevel analysis from the Joint Effort Toddler Temperament Consortium.

The present study aims to gain a greater understanding of the manner in which culture may impact parenting and, thus, child development by examining the relationship between cultural values, socialization goals (SGs), and parental ethnotheories (PEs). Specifically, this study examined links between cultural value dimensions (i.e., individualism/collectivism, power distance, masculinity/femininity, uncertainty avoidance, indulgence/restraint, and long-term/short-term orientation; Hofstede et al., 2010) and autonomous as well as relational SGs and PEs. We examined data collected from mothers of toddlers (N = 865) between 17 and 40 months of age (M = 26.88 months, SD = 5.65 months; 52% boys) from 14 nations represented in the Joint Effort Toddler Temperament Consortium. We hypothesized that: (a) Cultural values consistent with independent cultural ideals would be positively associated with SGs and PEs representative of greater autonomy and independence, and (b) Cultural values consistent with interdependent ideals would be positively associated with SGs and PEs representative of greater interrelatedness. Multilevel modeling was used to regress parental psychology on Hofstede's cultural values. Support for these hypotheses was somewhat mixed; higher ratings of culture-level indulgence were associated with higher autonomous PEs, as well as with higher relational and autonomous SGs. Furthermore, higher ratings of culture-level masculinity were associated with lower relational PEs and with lower autonomous SGs. The results suggest differences in the effects for cultural values associated with parenting versus cultural values associated with child outcomes and highlight considerations related to dichotomous cultural frameworks. The findings help explain both individual- and country-level variations in aspects of parental psychology. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Applicability and Efficiency of a Computerized Adaptive Test for the Washington Assessment of the Risks and Needs of Students.

The Washington Assessment of the Risks and Needs of Students (WARNS) is a computer-based assessment created to help courts, schools, and youth service providers determine an adolescent's risks and needs that may lead to truancy, drop out, or delinquency from school. Users are advised to consider the WARNS total score to work with youth. A total score estimate based on fewer items than the full item set may result in less respondent burden, administration time, and fatigue, while not hindering accurate decisions. This simulation study examined the applicability and efficiency of a computerized adaptive test (CAT) to estimate a WARNS total score under a unidimensional item response theory model. The results demonstrate that the CAT provides an accurate estimate of students' risks and needs and reduces the number of items administered for each examinee compared with the existing version. Future directions and limitations of CAT development with the WARNS are discussed.

A Comparison of Confirmatory Factor Analysis and Network Models for Measurement Invariance Assessment When Indicator Residuals are Correlated.

Social science research is heavily dependent on the use of standardized assessments of a variety of phenomena, such as mood, executive functioning, and cognitive ability. An important assumption when using these instruments is that they perform similarly for all members of the population. When this assumption is violated, the validity evidence of the scores is called into question. The standard approach for assessing the factorial invariance of the measures across subgroups within the population involves multiple groups confirmatory factor analysis (MGCFA). CFA models typically, but not always, assume that once the latent structure of the model is accounted for, the residual terms for the observed indicators are uncorrelated (local independence). Commonly, correlated residuals are introduced after a baseline model shows inadequate fit and inspection of modification indices ensues to remedy fit. An alternative procedure for fitting latent variable models that may be useful when local independence does not hold is based on network models. In particular, the residual network model (RNM) offers promise with respect to fitting latent variable models in the absence of local independence via an alternative search procedure. This simulation study compared the performances of MGCFA and RNM for measurement invariance assessment when local independence is violated, and residual covariances are themselves not invariant. Results revealed that RNM had better Type I error control and higher power compared to MGCFA when local independence was absent. Implications of the results for statistical practice are discussed.

Assessment of the Risks and Needs of Middle School Students: Invariance Properties Related to Gender and Ethnicity.

The middle school version of the Washington Assessment of Risks and Needs of Students (msWARNS) is a self-report instrument designed for use by school personnel to identify barriers to school attendance and school success for sixth- to eighth-grade students. It measures six domains relevant to improving school outcomes that include aggression-defiance, depression-anxiety, substance use, peer deviance, home environment, and school engagement. In the present study, a bifactor S - 1 model, for which the aggression-defiance domain was the reference factor for the general factor and the other domains constituted the subfactors, had good fit and better fit than several other alternative models. Results of multigroup confirmatory factor analysis revealed invariance across different groups defined by gender and race/ethnicity (Native American, African American, Hispanic, and White), with a sample of referred middle school students (N = 2,356; ages 10-15 years). Reliability analyses support the use of the general factor to guide decision-making, the reliable use of the depression-anxiety factor for providing additional insights, and the remaining factors for guiding communication, as part of an assessment and intervention program for middle school students.

Latent profile similarity of middle and high school youth risk and needs.

Research concerning school success and completion has grown increasingly complex with the number of proposed associated risk and needs domains. As the number of domains expands, various data analytical techniques have been employed to understand them, including the modeling of latent profiles, to better understand how risks and needs aggregate at the level of individual persons. Latent profile analysis helps identify individuals' subgroups based on salient combinations of characteristics. The present study used latent profile analysis and a systematic profile similarity approach to examine the profiles across middle and high school student cohorts. The study replicates the profiles of previous work with high school students and extends this to middle school students. We used two independent cohorts to replicate a 3-profile solution for middle and high school samples. Results supported a similar 3-profile solution for both samples, with minor discrepancies. Results are discussed with respect to the replication and extension of the 3-profile model and its application to efforts to improve outcomes for youth in both grade level cohorts.

Measuring Food Security among American Indian and Alaska Native Adults: Validity Evidence Supports the Use of the US Department of Agriculture Module.

BACKGROUND: Inequities in access, availability, and affordability of nutritious foods produced by settler colonialism contribute to high rates of food insecurity among American Indian and Alaska Native (AI/AN) households. Efforts to understand the influences of food security programming among AI/AN individuals in the United States are constrained by the absence of validity evidence for food security assessments for this population. OBJECTIVE: This study assessed whether AI/AN adult responses on the Food Security Survey Module provide an accurate assessment of food security prevalence, especially when compared with other racial and ethnic groups. DESIGN: A correlational design with the cross-sectional 2019 National Health Interview Survey was used to address the research objective. PARTICIPANTS AND SETTING: The 2019 National Health Interview Survey contains a sample (N = 30,052) representative of the resident civilian noninstitutionalized population. MAIN OUTCOME MEASURES: The primary outcome was food security, as characterized by the 10-item US Department of Agriculture Adult Food Security Survey Module. The module evaluates whether insufficient finances result in perceived food shortages and a reduction in the quantity and/or quality of food intake during the prior 30 days. STATISTICAL ANALYSES PERFORMED: Data were analyzed by racial and ethnic subsamples to assess scale dimensionality (confirmatory factor analysis), Item Response Theory item analysis, differential item functioning, and external validity (χ2 tests). RESULTS: Results supported the use of the 10-item module for racial and ethnic groups. However, differential item functioning effect sizes exceeded criteria for the Asian, AI/AN, and Hispanic respondents when compared with White respondents. Food security was not significantly related to all expected correlates in the AI/AN subsample. CONCLUSIONS: Compelling evidence is presented for validity of the FSSM scores in determining food security status of AI/AN adults. Qualitative inquiry that explores how culture influences the way food security is conceptualized and experienced is warranted.

Assessing spoken word recognition in children who are deaf or hard of hearing: a translational approach.

Under natural conditions, listeners use both auditory and visual speech cues to extract meaning from speech signals containing many sources of variability. However, traditional clinical tests of spoken word recognition routinely employ isolated words or sentences produced by a single talker in an auditory-only presentation format. The more central cognitive processes used during multimodal integration, perceptual normalization, and lexical discrimination that may contribute to individual variation in spoken word recognition performance are not assessed in conventional tests of this kind. In this article, we review our past and current research activities aimed at developing a series of new assessment tools designed to evaluate spoken word recognition in children who are deaf or hard of hearing. These measures are theoretically motivated by a current model of spoken word recognition and also incorporate "real-world" stimulus variability in the form of multiple talkers and presentation formats. The goal of this research is to enhance our ability to estimate real-world listening skills and to predict benefit from sensory aid use in children with varying degrees of hearing loss.

Relations between bedtime parenting behaviors and temperament across 14 cultures.

Objectives: The present study examined parental sleep-supporting practices during toddlerhood in relation to temperament across 14 cultures. We hypothesized that passive sleep-supporting techniques (e.g., talking, cuddling), but not active techniques (e.g., walking, doing an activity together), would be associated with less challenging temperament profiles: higher Surgency (SUR) and Effortful Control (EC) and lower Negative Emotionality (NE), with fine-grained dimensions exhibiting relationships consistent with their overarching factors (e.g., parallel passive sleep-supporting approach effects for dimensions of NE). Methods: Caregivers (N = 841) across 14 cultures (M = 61 families per site) reported toddler (between 17 and 40 months of age; 52% male) temperament and sleep-supporting activities. Utilizing linear multilevel regression models and group-mean centering procedures, we assessed the role of between- and within-cultural variance in sleep-supporting practices in relation to temperament. Results: Both within-and between-culture differences in passive sleep-supporting techniques were associated with temperament attributes, (e.g., lower NE at the between-culture level; higher within-culture EC). For active techniques only within-culture effects were significant (e.g., demonstrating a positive association with NE). Adding sleep-supporting behaviors to the regression models accounted for significantly more between-culture temperament variance than child age and gender alone. Conclusion: Hypotheses were largely supported. Findings suggest parental sleep practices could be potential targets for interventions to mitigate risk posed by challenging temperament profiles (e.g., reducing active techniques that are associated with greater distress proneness and NE).

A Motivational-Developmental Free Response Assessment Through a Bifactor Lens.

The Motivational-Developmental Assessment (MDA) measures a university student's motivational and developmental attributes by utilizing overlapping constructs measured across four writing prompts. The MDA's format may lead to the violation of the local item independence (LII) assumption for unidimensional item response theory (IRT) scoring models, or the uncorrelated errors assumption for scoring models in classical test theory (CTT) due to the measurement of overlapping constructs within a prompt. This assumption violation is known as a testlet effect, which can be viewed as a method effect. The application of a unidimensional IRT or CTT model to score the MDA can result in imprecise parameter estimates when this effect is ignored. To control for this effect in the MDA responses, we first examined the presence of local dependence via a restricted bifactor model and Yen's Q3 statistic. Second, we applied bifactor models to account for the testlet effect in the responses, as this effect is modeled as an additional latent variable in a factor model. Results support the presence of local dependence in two of the four MDA prompts, and the use of the restricted bifactor model to account for the testlet effect in the responses. Modeling the testlet effect through the restricted bifactor model supports a scoring inference in a validation argument framework. Implications are discussed.

Item response theory analysis of the Triarchic Psychopathy Measure.

This study investigated item and scale functioning in the triarchic psychopathy measure (TriPM) using an item response theory (IRT) analysis. TriPM data collected from 937 (410 men and 527 women) college students and community members were analyzed. A graded response model was utilized to analyze the items comprising the TriPM's three scales. Exploratory factor analyses (EFAs) supported the unidimensionality of the Disinhibition and Meanness scales, whereas Boldness was best represented by two domains. Each TriPM scale was found to include some items that had limited ability to differentiate between respondents possessing varying levels of the trait being measured. Across all scales, 34 items (61%) yielded evidence of significantly different responding between men and women possessing similar levels of the underlying trait, or differential item functioning (DIF). The scoring format (i.e., directly scored vs. reverse scored) also influenced item functioning, especially when the given scale included a large majority of one scoring type. Overall, the IRT and DIF results suggest that the TriPM effectively identifies disinhibition and meanness in individuals possessing high levels of those traits, and boldness at lower levels, but does not seem to identify these traits equally well in men and women. Possible revisions to the Boldness scale as well as ideas for addressing female and male DIF were provided. (PsycInfo Database Record (c) 2021 APA, all rights reserved).