Psycho-Educational Assessments: Theory and Practice.

Psycho-educational assessments, such as intelligence tests, cognitive test batteries, and behavioral measures, serve as invaluable tools for school psychologists and educators [...].

Preschool Children's Loose Parts Play and the Relationship to Cognitive Development: A Review of the Literature.

Play is an integrative process, and the skills acquired in it-overcoming impulses, behavior control, exploration and discovery, problem-solving, reasoning, drawing conclusions, and attention to processes and outcomes are foundational cognitive structures that drive learning and motivation. Loose parts play is a prominent form of play that many scholars and educators explicitly endorse for cognitive development (e.g., divergent thinking, problem-solving). It is unique among play types because children can combine different play types and natural or manufactured materials in one occurrence. While educators and policymakers promote the benefits of loose parts play, no previous research has explored the direct relationship between preschool-age children's indoor loose parts play experiences and cognitive development. We address this gap by bringing together the relevant literature and synthesizing the empirical studies on common play types with loose parts, namely object and exploratory, symbolic and pretend, and constructive play. We also focus on studies that examine children's experiences through loose parts, highlighting the impact of different play types on learning through the reinforcement of cognitive skills, such as executive function, cognitive self-regulation, reasoning, and problem-solving. By examining the existing literature and synthesizing empirical evidence, we aim to deepen our understanding of the relationship between children's play with loose parts and its impact on cognitive development. Ultimately, pointing out the gaps in the literature that would add to the body of knowledge surrounding the benefits of play for cognitive development and inform educators, policymakers, and researchers about the significance of incorporating loose parts play into early childhood education.

A Psychometric Network Analysis of CHC Intelligence Measures: Implications for Research, Theory, and Interpretation of Broad CHC Scores "Beyond g".

For over a century, the structure of intelligence has been dominated by factor analytic methods that presume tests are indicators of latent entities (e.g., general intelligence or g). Recently, psychometric network methods and theories (e.g., process overlap theory; dynamic mutualism) have provided alternatives to g-centric factor models. However, few studies have investigated contemporary cognitive measures using network methods. We apply a Gaussian graphical network model to the age 9-19 standardization sample of the Woodcock-Johnson Tests of Cognitive Ability-Fourth Edition. Results support the primary broad abilities from the Cattell-Horn-Carroll (CHC) theory and suggest that the working memory-attentional control complex may be central to understanding a CHC network model of intelligence. Supplementary multidimensional scaling analyses indicate the existence of possible higher-order dimensions (PPIK; triadic theory; System I-II cognitive processing) as well as separate learning and retrieval aspects of long-term memory. Overall, the network approach offers a viable alternative to factor models with a g-centric bias (i.e., bifactor models) that have led to erroneous conclusions regarding the utility of broad CHC scores in test interpretation beyond the full-scale IQ, g.

Item Selection With Collaborative Filtering in On-The-Fly Multistage Adaptive Testing.

An important design feature in the implementation of both computerized adaptive testing and multistage adaptive testing is the use of an appropriate method for item selection. The item selection method is expected to select the most optimal items depending on the examinees' ability level while considering other design features (e.g., item exposure and item bank utilization). This study introduced collaborative filtering (CF) as a new method for item selection in the on-the-fly assembled multistage adaptive testing framework. The user-based CF (UBCF) and item-based CF (IBCF) methods were compared to the maximum Fisher information method based on the accuracy of ability estimation, item exposure rates, and item bank utilization under different test conditions (e.g., item bank size, test length, and the sparseness of training data). The simulation results indicated that the UBCF method outperformed the traditional item selection methods regarding measurement accuracy. Also, the IBCF method showed the most superior performance in terms of item bank utilization. Limitations of the current study and the directions for future research are discussed.

Exploring Factors That Affected Student Well-Being during the COVID-19 Pandemic: A Comparison of Data-Mining Approaches.

COVID-19-related school closures caused unprecedented and prolonged disruption to daily life, education, and social and physical activities. This disruption in the life course affected the well-being of students from different age groups. This study proposed analyzing student well-being and determining the most influential factors that affected student well-being during the COVID-19 pandemic. With this aim, we adopted a cross-sectional study designed to analyze the student data from the Responses to Educational Disruption Survey (REDS) collected between December 2020 and July 2021 from a large sample of grade 8 or equivalent students from eight countries (n = 20,720), including Burkina Faso, Denmark, Ethiopia, Kenya, the Russian Federation, Slovenia, the United Arab Emirates, and Uzbekistan. We first estimated a well-being IRT score for each student in the REDS student database. Then, we used 10 data-mining approaches to determine the most influential factors that affected the well-being of students during the COVID-19 outbreak. Overall, 178 factors were analyzed. The results indicated that the most influential factors on student well-being were multifarious. The most influential variables on student well-being were students' worries about contracting COVID-19 at school, their learning progress during the COVID-19 disruption, their motivation to learn when school reopened, and their excitement to reunite with friends after the COVID-19 disruption.

Examining Humans' Problem-Solving Styles in Technology-Rich Environments Using Log File Data.

This study investigated how one's problem-solving style impacts his/her problem-solving performance in technology-rich environments. Drawing upon experiential learning theory, we extracted two behavioral indicators (i.e., planning duration for problem solving and human-computer interaction frequency) to model problem-solving styles in technology-rich environments. We employed an existing data set in which 7516 participants responded to 14 technology-based tasks of the Programme for the International Assessment of Adult Competencies (PIAAC) 2012. Clustering analyses revealed three problem-solving styles: Acting indicates a preference for active explorations; Reflecting represents a tendency to observe; and Shirking shows an inclination toward scarce tryouts and few observations. Explanatory item response modeling analyses disclosed that individuals with the Acting style outperformed those with the Reflecting or the Shirking style, and this superiority persisted across tasks with different difficulties.

An explanatory mixture IRT model for careless and insufficient effort responding in self-report measures.

Careless and insufficient effort responding (C/IER) on self-report measures results in responses that do not reflect the trait to be measured, thereby posing a major threat to the quality of survey data. Reliable approaches for detecting C/IER aid in increasing the validity of inferences being made from survey data. First, once detected, C/IER can be taken into account in data analysis. Second, approaches for detecting C/IER support a better understanding of its occurrence, which facilitates designing surveys that curb the prevalence of C/IER. Previous approaches for detecting C/IER are limited in that they identify C/IER at the aggregate respondent or scale level, thereby hindering investigations of item characteristics evoking C/IER. We propose an explanatory mixture item response theory model that supports identifying and modelling C/IER at the respondent-by-item level, can detect a wide array of C/IER patterns, and facilitates a deeper understanding of item characteristics associated with its occurrence. As the approach only requires raw response data, it is applicable to data from paper-and-pencil and online surveys. The model shows good parameter recovery and can well handle the simultaneous occurrence of multiple types of C/IER patterns in simulated data. The approach is illustrated on a publicly available Big Five inventory data set, where we found later item positions to be associated with higher C/IER probabilities. We gathered initial supporting validity evidence for the proposed approach by investigating agreement with multiple commonly employed indicators of C/IER.

Rapid assessment of communication consistency: sentiment analysis of public health briefings during the COVID-19 pandemic.

Background: A key component of the initial public health response to the COVID-19 pandemic involved the use of mass media briefings led by public health officials to communicate updates during a time of great uncertainty and rapidly changing information. This study aims to examine the consistency of communications expressed during the public health briefings to generate novel insights about the type, direction, and strength of public health messages. The data source included 131 readily accessible public health briefings alongside the provincial and national new confirmed case counts during the first two waves of rapidly increasing cases during the pandemic in Alberta, Canada. We employed sentiment analysis as a text mining technique to explore the types and frequency of words in public health briefings conveying positive and negative sentiments. Using statistical analyses and data visualizations, we examined how public health messaging shifted with case trends. Results: Our findings indicate consistent public health messaging in terms of sentiments regardless of case count fluctuations, an association of specific words with conveying positive and negative sentiments, and a focus on particular message patterns at different points during the first two waves of the COVID-19 pandemic. Conclusion: Our findings demonstrate the practical implications and methodological advantages of using sentiment analysis as a data analytics tool for rapidly and objectively assessing the consistency of health communications during a public health crisis.

Conformity, obedience, and the Better than Average Effect in health professional students.

Background: Compliance, through conformity and obedience to authority, can produce negative outcomes for patient safety, as well as education. To date, educational interventions for dealing with situations of compliance or positive deviance have shown variable results. Part of the challenge for education on compliance may result from disparities between learners' expectations about their potential for engaging in positive deviance and the actual likelihood of engaging in positive deviance. More specifically, students may demonstrate a Better Than Average Effect (BTAE), the tendency for people to believe they are comparatively better than the average across a wide range of behaviours and skills. Methods: Four vignettes were designed and piloted using cognitive interviews, to investigate the BTAE. Conformity and obedience to authority were each addressed with two vignettes. The vignettes were included in a survey distributed to Canadian health professional students across multiple programs at several different institutions during the Winter 2019 semester. Self-evaluation of behaviour was investigated using a one-sample proportion test. Demographic data were investigated using logistic regression to identify predictors of the BTAE. Results: Participants demonstrated the BTAE for expected behaviour compared to peers for situations of conformity and obedience to authority. Age, sex, and program year were identified as potential predictors for exhibiting the BTAE. Conclusions: This study demonstrated that health professional students expect that they will behave better than average in compliance scenarios. Health professional students are not exempt from this cognitive bias in self-assessment. The results have implications for education on compliance, positive deviance, and patient safety.

Contexte: La conformité, par le biais du respect des normes et de l'obéissance à l'autorité, peut avoir des effets négatifs tant pour la sécurité des patients que pour l'éducation des médecins. À ce jour, les interventions éducatives portant sur la conformité ou la déviance positive se sont soldées par des résultats mitigés. Une des difficultés auxquelles se heurte l'enseignement de la conformité provient de l'écart entre les attentes des apprenants quant à la possibilité d'appliquer la déviance positive et la probabilité qu'ils adoptent réellement ce comportement. Plus précisément, les étudiants peuvent faire preuve de supériorité illusoire, c'est-à-dire la tendance à se croire supérieur à la moyenne des gens par rapport à un éventail de comportements et d'habiletés. Méthodes: Quatre vignettes, deux sur la conformité et deux sur l'obéissance à l'autorité, ont été conçues et mises à l'essai dans le cadre d'entretiens cognitifs afin d'étudier la présence du phénomène de supériorité illusoire. Les vignettes faisaient partie d'une enquête menée auprès d'étudiants de différents programmes en sciences de la santé offerts par divers établissements canadiens au cours de la session d'hiver 2019. L'auto-évaluation du comportement a été examinée à l'aide d'un test de proportion pour échantillon unique. Les données démographiques ont été étudiées à l'aide d'une régression logistique pour identifier les prédicteurs de la supériorité illusoire. Résultats: L'effet de supériorité illusoire par rapport aux pairs était présent chez les participants en ce qui concerne le comportement qu'on attend d'eux dans des situations en lien avec la conformité et l'obéissance à l'autorité. L'âge, le sexe et l'année de formation ont été repérés comme prédicteurs potentiels de la manifestation de supériorité illusoire. Conclusions: Cette étude montre que les étudiants en sciences de la santé s'attendent à avoir un comportement supérieur à la moyenne dans les situations où la conformité est en jeu. Ils ne sont pas exempts de ce biais cognitif dans leur auto-évaluation. Les résultats obtenus ont des implications pour l'éducation en matière de conformité, de déviance positive et de sécurité des patients.

Linguistic Influences on Cognitive Test Performance: Examinee Characteristics Are More Important than Test Characteristics.

Consideration of the influence of English language skills during testing is an understandable requirement for fair and valid cognitive test interpretation. Several professional standards and expert recommendations exist to guide psychologists as they attempt to engage in best practices when assessing English learners (ELs). Nonetheless, relatively few evidence-based recommendations for practice have been specified for psychologists. To address this issue, we used a mixed-effects modeling approach to examine the influences of test characteristics (i.e., test directions) and examinee characteristics (i.e., expressive and receptive language abilities) on cognitive test performance. Our results suggest that language abilities appear to have a significant influence on cognitive test performance, whereas test characteristics do not influence performance, after accounting for language abilities. Implications for practice include the assessment of expressive and receptive language abilities of EL students prior to administering, scoring, and interpreting cognitive test scores.

Building an intelligent recommendation system for personalized test scheduling in computerized assessments: A reinforcement learning approach.

The introduction of computerized formative assessments in the classroom has opened a new area of effective progress monitoring with more accessible test administrations. With computerized formative assessments, all students could be tested at the same time and with the same number of test administrations within a school year. Alternatively, the decision for the number and frequency of such tests could be made by teachers based on their observations and personal judgments about students. However, this often results in rigid test scheduling that fails to take into account the pace at which students acquire knowledge. To administer computerized formative assessments efficiently, teachers should be provided with systematic guidance regarding effective test scheduling based on each student's level of progress. In this study, we introduce an intelligent recommendation system that can gauge the optimal number and timing of testing for each student. We discuss how to build an intelligent recommendation system using a reinforcement learning approach. Then, we present a case study with a large sample of students' test results in a computerized formative assessment. We show that the intelligent recommendation system can significantly reduce the number of testing for the students by eliminating unnecessary test administrations where students do not show significant progress (i.e., growth). Also, the proposed recommendation system is capable of identifying the optimal test time for students to demonstrate adequate progress from one test administration to another. Implications for future research on personalized assessment scheduling are discussed.

Risk and Resilience: How Is the Health of Older Adults and Immigrant People Living in Canada Impacted by Climate- and Air Pollution-Related Exposures?

BACKGROUND: In the rapidly shifting Canadian climate, an ageing population, and increased migration, a greater understanding of how local climate and air pollution hazards impact older adults and immigrant populations will be necessary for mitigating and adapting to adverse health impacts. OBJECTIVES: To explore the reported health impacts of climate change and air pollution exposures in older adults and immigrant people living in Canada, identify known factors influencing risk and resilience in these populations and gaps in the literature. METHODS: We searched for research focused on older adults and immigrants living in Canada, published from 2010 onward, where the primary exposures were related to climate or air pollution. We extracted data on setting, exposures, health outcomes, and other relevant contextual factors. RESULTS AND DISCUSSION: We identified 52 eligible studies, most focused in Ontario and Quebec. Older people in Canada experience health risks due to climate and air pollution exposures. The extent of the risk depends on multiple factors. We found little information about the climate- and air pollution-related health impacts experienced by immigrant communities. CONCLUSIONS: Further research about climate- and air pollution-related exposures, health, and which factors promote or reduce resiliency in Canada's older adults and immigrant communities is necessary.

A Polytomous Scoring Approach to Handle Not-Reached Items in Low-Stakes Assessments.

In low-stakes assessments, some students may not reach the end of the test and leave some items unanswered due to various reasons (e.g., lack of test-taking motivation, poor time management, and test speededness). Not-reached items are often treated as incorrect or not-administered in the scoring process. However, when the proportion of not-reached items is high, these traditional approaches may yield biased scores and thereby threatening the validity of test results. In this study, we propose a polytomous scoring approach for handling not-reached items and compare its performance with those of the traditional scoring approaches. Real data from a low-stakes math assessment administered to second and third graders were used. The assessment consisted of 40 short-answer items focusing on addition and subtraction. The students were instructed to answer as many items as possible within 5 minutes. Using the traditional scoring approaches, students' responses for not-reached items were treated as either not-administered or incorrect in the scoring process. With the proposed scoring approach, students' nonmissing responses were scored polytomously based on how accurately and rapidly they responded to the items to reduce the impact of not-reached items on ability estimation. The traditional and polytomous scoring approaches were compared based on several evaluation criteria, such as model fit indices, test information function, and bias. The results indicated that the polytomous scoring approaches outperformed the traditional approaches. The complete case simulation corroborated our empirical findings that the scoring approach in which nonmissing items were scored polytomously and not-reached items were considered not-administered performed the best. Implications of the polytomous scoring approach for low-stakes assessments were discussed.

Age and Sex Invariance of the Woodcock-Johnson IV Tests of Cognitive Abilities: Evidence from Psychometric Network Modeling.

The Woodcock-Johnson IV Tests of Cognitive Abilities (WJ IV COG) is a comprehensive assessment battery designed to assess broad and narrow cognitive abilities, as defined by the Cattell-Horn-Carroll (CHC) theory of intelligence. Previous studies examined the invariance of the WJ assessments across sex and age groups using factor analytic methods. Psychometric network modeling is an alternative methodology that can address both direct and indirect relationships among the observed variables. In this study, we employed psychometric network modeling to examine the invariance of the WJ IV COG across sex and age groups. Using a normative sample (n = 4212 participants) representative of the United States population, we tested the extent to which the factorial structure of the WJ IV COG aligned with CHC theory for the school-aged sample. Next, we used psychometric network modeling as a data-driven method to investigate whether the network structure of the WJ IV COG remains similar across different sex and age (age 6 to 19, inclusively) groups. Our results showed that the WJ IV COG maintained the same network structure across all age and sex groups, although the network structure at younger ages indicated weaker relationships among some subtests. Overall, the results provide construct validity evidence for the WJ IV COG, based on both theoretical and data-driven methods.

Effect of Detailed OSCE Score Reporting on Learning and Anxiety in Medical School.

INTRODUCTION: There is growing literature on increasing feedback from Objective Structured Clinical Examinations (OSCEs) and one approach is a score report. The purpose of this study was to implement and evaluate a score report for a second and fourth-year medical school OSCE. METHODS: We developed an electronic OSCE score report that displayed comments and performance by domain within and across stations (checklist items and rating scales were tagged to each domain). Our initial pilot released the score report after pass/fail decisions but subsequent iterations released the score report the same day as the exam. Our evaluation approach included both student surveys and focus groups. RESULTS: Students felt the OSCE score report was accurate, identified strengths and weaknesses, and would likely cause them to take future action, with second-year students more likely to act on the report than fourth year students. The thematic analysis revealed barriers and enablers to utilizing feedback as well as the power of the score report to reduce anxiety. CONCLUSIONS: Our OSCE score report was simple to develop and implement the same day as an OSCE with an overall positive response from students with respect to accuracy and ability to use the information for future learning.

A multi-method exploratory study of health professional students' experiences with compliance behaviours.

BACKGROUND: Research in healthcare, including students as participants, has begun to document experiences with negative compliance, specifically conformity and obedience. There is a growing body of experimental and survey literature, however, currently lacking is a direct measure of the frequency at which health professional students have negative experiences with conformity and obedience integrated with psychological factors, the outcomes of negative compliance, and students' perceptions. METHODS: To develop empirical knowledge about the frequency of negative compliance and student perceptions during health professional education a multi-methods survey approach was used. The survey was administered to health professional students across ten disciplines at four institutions. RESULTS: The results indicated students regularly experience obedience and conformity and are influenced by impression management and displacement of responsibility. Moral distress was identified as a consistent negative outcome. Student self-reported experiences aligned with the empirical findings. CONCLUSIONS: The findings of the present study demonstrate the pervasiveness of experiences with negative compliance during health professional's education along with some attendant psychological factors. The findings have educational and practical implications, as well as pointing to the need for further integration of social and cognitive psychology in explaining compliance in healthcare. The results are likely generalizable to a population level however replication is encouraged to better understand the true frequency of negative compliance at a health professional population level.

Evaluating the Performances of Missing Data Handling Methods in Ability Estimation From Sparse Data.

Large amounts of missing data could distort item parameter estimation and lead to biased ability estimates in educational assessments. Therefore, missing responses should be handled properly before estimating any parameters. In this study, two Monte Carlo simulation studies were conducted to compare the performance of four methods in handling missing data when estimating ability parameters. The methods were full-information maximum likelihood (FIML), zero replacement, and multiple imputation with chain equations utilizing classification and regression trees (MICE-CART) and random forest imputation (MICE-RFI). For the two imputation methods, missing responses were considered as a valid response category to enhance the accuracy of imputations. Bias, root mean square error, and the correlation between true ability parameters and estimated ability parameters were used to evaluate the accuracy of ability estimates for each method. Results indicated that FIML outperformed the other methods under most conditions. Zero replacement yielded accurate ability estimates when missing proportions were very high. The performances of MICE-CART and MICE-RFI were quite similar but these two methods appeared to be affected differently by the missing data mechanism. As the number of items increased and missing proportions decreased, all the methods performed better. In addition, the information on missing data could improve the performance of MICE-RFI and MICE-CART when the data set is sparse and the missing data mechanism is missing at random.

From experiences to expectations: A quantitative study on the fear of childbirth among multigravida women.

There is a strong relationship between women's previous birth experiences and their predetermined expectations. Childbirth expectations play an important role in the women's response to the birthing experience and postpartum period. The negative emotions and expectations, such as fear of childbirth, may result in negative experiences in subsequent childbirths. The aim of this study is to examine the relationship between previous birth experiences and the fear of childbirth in current pregnancy. A sample of 309 healthy women with normal pregnancies was recruited for this study. A chi-square test of independence and a multinomial logistic regression were used to explain the association between previous birth experiences and the fear of childbirth in current pregnancy. The level of childbirth fear that pregnant women feel appears to differ based upon their previous pregnancy experiences. Pregnant women who describe their previous births as happy and proud tend to experience a moderate level of childbirth fear about their current pregnancies, whereas pregnant women who remember their previous births as either fearful or painful have lower levels of childbirth fear. The level of childbirth fear tends to decrease as pregnant women have more children. Gestational week does not seem to be influential on the level of childbirth fear. According to the findings of this study, counterintuitively there is a negative relationship between the previous birth experience and childbirth fear. The level of childbirth fear is lower for pregnant women who remember their previous births as a negative experience compared to those who remember their previous births positively.

Examining the relation of personality factors to substance use disorder by explanatory item response modeling of DSM-5 symptoms.

This paper explores how personality factors affect substance use disorders (SUDs) using explanatory item response modeling (EIRM). A total of 606 Chinese illicit drug users participated in our study. After removing the cases with missing values on the covariate measures, a final sample of 573 participants was used for data analysis. The Diagnostic and Statistical Manual of Mental Disorders (DSM-5) was used to measure the illicit drug users' SUD level. Four personality factors-anxiety sensitivity, impulsivity, sensation seeking and hopelessness-along with gender and alcohol use were included in EIRM as person covariates. The results indicated that gender, alcohol use, and their interaction significantly predicted the SUD level. The only personality factor that strongly predicted the SUD level was sensation seeking. In addition, the interaction between gender and hopelessness was also found to be a significant predictor of the SUD level, indicating that the negative effect of hopelessness on SUD is stronger for women than for men. The findings suggest that sensation seeking plays an important role in influencing SUDs, and thus, it should be considered when designing intervention or screening procedures for potential illicit drug users. In addition, several DSM-5 SUD symptoms were found to exhibit differential effects by gender, alcohol use, and personality factors. The possible explanations were discussed.

Retrofitting Diagnostic Classification Models to Responses From IRT-Based Assessment Forms.

Developing a diagnostic tool within the diagnostic measurement framework is the optimal approach to obtain multidimensional and classification-based feedback on examinees. However, end users may seek to obtain diagnostic feedback from existing item responses to assessments that have been designed under either the classical test theory or item response theory frameworks. Retrofitting diagnostic classification models to existing assessments designed under other psychometric frameworks could be a plausible approach to obtain more actionable scores or understand more about the constructs themselves. This study (a) discusses the possibility and problems of retrofitting, (b) proposes a step-by-step retrofitting framework, and (c) explores the information one can gain from retrofitting through an empirical application example. While retrofitting may not always be an ideal approach to diagnostic measurement, this article aims to invite discussions through presenting the possibility, challenges, process, and product of retrofitting.