Assessment fidelity of a language screening instrument for 4-year-olds.

AIM: The aim of the study was to explore the assessment fidelity of Språkfyran, a language screening instrument for four-year-old children. Språkfyran is a mandatory part of the healthcare program within the Swedish Child Health Service (CHS) and is offered to all four-year-olds in the region Scania in Sweden. METHODS: The study was based on structured observations of twenty-four specialist CHS nurses' adherence to the Språkfyran protocol during screening. RESULTS: All the observed nurses deviated from the test protocol. There was a large variation in the number of deviations from the test protocol per nurse, with the highest number of deviations occurring for three specific testing items. Significantly more deviations were made with four-year-old bilingual children as opposed to four-year-old monolingual children. Half of the nurses did not use the test protocol. CONCLUSIONS: There is a clear need to improve the assessment fidelity of Språkfyran. Both the training that the nurses are offered, and the development of the test, are essential in securing the aim of high-quality work within the CHS. Support from experts in child speech-language development and disorders is suggested to be available at the CHS in Sweden.

Evaluating implementation outcomes of a measure of social vulnerability in adults with intellectual disabilities.

BACKGROUND: A test identified as valid and accurate in research will not automatically be considered appropriate by those involved in its use, or even be used in the first place. The Social Vulnerability Test-22 items [TV-22] is a measure specially designed for adults with intellectual disabilities (ID). This study aims to evaluate the implementation outcomes of the TV-22; more precisely its acceptability (e.g., complexity), appropriateness (e.g., perceived relevance) and the assessment fidelity (i.e., adherence to assessment guidelines) by special education practitioners. PROCEDURES: Thirty-one practitioners (8 psychologists, 11 educators, 12 special education center managers) administered the TV-22 during an interview with an adult with ID. Semi-structured interviews were conducted to collect practitioners' opinions on the acceptability and the appropriateness of the TV-22 for their clinical practice. Quantitative analyses were performed to assess the fidelity of the assessments and the influence of some personal factors. RESULTS: The results indicate a good appropriateness, a reasonable acceptability, - but a low assessment fidelity of the TV-22 by some practitioners. Psychologists stand out for a more rigorous use of the test. IMPLICATIONS: Results highlight the importance of evaluating implementation outcomes when a new measure is developed to ensure its appropriateness and correct use by stakeholders.

Examiner error in curriculum-based measurement of oral reading.

Although curriculum based measures of oral reading (CBM-R) have strong technical adequacy, there is still a reason to believe that student performance may be influenced by factors of the testing situation, such as errors examiners make in administering and scoring the test. This study examined the construct-irrelevant variance introduced by examiners using a cross-classified multilevel model. We sought to determine the extent of variance in student CBM-R scores attributable to examiners and, if present, the extent to which it was moderated by students' grade level and English learner (EL) status. Fit indices indicated that a cross-classified random effects model (CCREM) best fits the data with measures nested within students, students nested within schools, and examiners crossing schools. Intraclass correlations of the CCREM revealed that roughly 16% of the variance in student CBM-R scores was associated between examiners. The remaining variance was associated with the measurement level, 3.59%; between students, 75.23%; and between schools, 5.21%. Results were moderated by grade level but not by EL status. The discussion addresses the implications of this error for low-stakes and high-stakes decisions about students, teacher evaluation systems, and hypothesis testing in reading intervention research.