The Validity of a Holistically Scored Retell Protocol for Determining the Reading Comprehension of Middle School Students.

ABSTRACT

In this study, the authors examined the validity of a holistically scored retell within a confirmatory factor analysis framework by comparing the fit of a three-factor model of reading with the data from a diverse sample of seventh and eighth graders. The final model demonstrated adequate fit, χ(2)(32) = 97.316; comparative fit index = .96; Tucker-Lewis index = .94; and root mean square error of approximation = .08. Retell's chi-square difference, Δχ(2)(1) = 16.652, p < .001, and factor loading (.250, p < .001) were higher for the comprehension construct. Similarly, retell's correlation to comprehension measures (r = .155-.257, p < .01) was stronger than its relationship to measures of fluency (r = .158-.183, p < .01) or word identification (r = .132, p < .05). However, retell had a large residual variance (.938) and low interrater reliability (κ = .37), suggesting that improvements to the instrument are needed. Despite overall latent differences, retell did not demonstrate differential item functioning.