Using Confidence Intervals of the Item and Test Information Functions to Test Differential Item and Test Functioning: Visual and Statistical Analyses.

ABSTRACT

The purpose of the present paper was twofold (a) to use 95% confidence intervals of the item and test information functions as a means of visualizing differences between groups on the information provided at the item and test levels, and, (b) to statistically compare item and test information functions as a method for evaluating differential item and differential test functioning. Participants were 2,305 high school students who took a Mathematics National entrance examination in Saudi Arabia. Item and test information functions, conditional standard errors of measurement and reliability were estimated for both males and females. Differences between groups became evident when plotting 95% confidence intervals of the item and test information functions and the visual findings were confirmed using population-based Z-tests of point estimates using a Monte-Carlo simulation. It was concluded that differential group behavior at the item and test levels can be evidenced using information functions and inferential tests of significance can be constructed using the bootstrap distribution. The current procedure involves both item difficulties and discrimination indices and provides increased sensitivity over the traditional methods relying on item difficulties only.