The effect of missing data and imputation on the detection of bias in cognitive testing using differential item functioning methods.

BACKGROUND: Item response theory (IRT) methods for addressing differential item functioning (DIF) can detect group differences in responses to individual items (e.g., bias). IRT and DIF-detection methods have been used increasingly often to identify bias in cognitive test performance by characteristics (DIF grouping variables) such as hearing impairment, race, and educational attainment. Previous analyses have not considered the effect of missing data on inferences, although levels of missing cognitive data can be substantial in epidemiologic studies. METHODS: We used data from Visit 6 (2016-2017) of the Atherosclerosis Risk in Communities Neurocognitive Study (N = 3,580) to explicate the effect of artificially imposed missing data patterns and imputation on DIF detection. RESULTS: When missing data was imposed among individuals in a specific DIF group but was unrelated to cognitive test performance, there was no systematic error. However, when missing data was related to cognitive test performance and DIF group membership, there was systematic error in DIF detection. Given this missing data pattern, the median DIF detection error associated with 10%, 30%, and 50% missingness was -0.03, -0.08, and -0.14 standard deviation (SD) units without imputation, but this decreased to -0.02, -0.04, and -0.08 SD units with multiple imputation. CONCLUSIONS: Incorrect inferences in DIF testing have downstream consequences for the use of cognitive tests in research. It is therefore crucial to consider the effect and reasons behind missing data when evaluating bias in cognitive testing.

Assessing Bias in Cognitive Testing for Older Adults with Sensory Impairment: An Analysis of Differential Item Functioning in the Baltimore Longitudinal Study on Aging (BLSA) and the Atherosclerosis Risk in Communities Neurocognitive Study (ARIC-NCS).

OBJECTIVES: Vision and hearing impairments affect 55% of people aged 60+ years and are associated with lower cognitive test performance; however, tests rely on vision, hearing, or both. We hypothesized that scores on tests that depend on vision or hearing are different among those with vision or hearing impairments, respectively, controlling for underlying cognition. METHODS: Leveraging cross-sectional data from the Baltimore Longitudinal Study of Aging (BLSA) and the Atherosclerosis Risk in Communities Neurocognitive Study (ARIC-NCS), we used item response theory to test for differential item functioning (DIF) by vision impairment (better eye presenting visual acuity worse than 20/40) and hearing impairment (better ear .5-4 kHz pure-tone average > 25 decibels). RESULTS: We identified DIF by vision impairment for tests whose administrations do not rely on vision [e.g., Delayed Word Recall both in ARIC-NCS: .50 logit difference between impaired and unimpaired (p = .04) and in BLSA: .62 logits (p = .02)] and DIF by hearing impairment for tests whose administrations do not rely on hearing [Digit Symbol Substitution test in BLSA: 1.25 logits (p = .001) and Incidental Learning test in ARIC-NCS: .35 logits (p = .001)]. However, no individuals had differences between unadjusted and DIF-adjusted measures of greater than the standard error of measurement. CONCLUSIONS: DIF by sensory impairment in cognitive tests was independent of administration characteristics, which could indicate that elevated cognitive load among persons with sensory impairment plays a larger role in test performance than previously acknowledged. While these results were unexpected, neither of these samples are nationally representative and each has unique selection factors; thus, replication is critical.

Sound localization with hearing protectors.

A preliminary investigation was conducted of the ability of normal-hearing subjects to localize sound while wearing hearing protective devices. Performance with Bilsom 2392 level-dependent stereophonic muffs with limited amplification was compared with that of conventional level-independent E-A-R 3000 muffs and E-A-R plugs. Sound localization was assessed using an array of six speakers, positioned in the horizontal plane, 1 m from the subject at azimuth angles of 30 degrees, 90 degrees, 150 degrees, 210 degrees, 270 degrees and 330 degrees. The stimulus was an 80 dB SPL 300-ms one-third octave noise band centered at 500 Hz or 4000 Hz. The experiment was conducted in quiet and in a 65 dB SPL white noise background. The results indicated that at 500 Hz, the wearing of protectors resulted in a 20% decrease in the accuracy of speaker identification but the three types of device were indistinguishable. At 4000 Hz, performance was best in the unoccluded condition and relatively worst with the level-dependent muff. Error patterns depended on the interaction of stimulus frequency, speaker azimuth and protector. Front/back errors in localizing the high-frequency stimulus were greatest with the conventional muff, whereas right/left errors were greatest with the level-dependent muff. Protected listening was generally unaffected by the presence of a noise background for both of the frequencies tested.

Auditory perception with level-dependent hearing protectors. The effects of age and hearing loss.

Auditory perception with hearing protectors was assessed in three groups of subjects, two with normal hearing, but differing in age, and one with moderate bilateral sensorineural hearing loss. Individuals were tested with the ears unoccluded, and fitted with each of two level-dependent ear muffs and their conventional level-independent counterparts. One of the former devices provided limited amplification. In each of these five ear conditions, the threshold of audibility for one-third octave noise bands centered at 500, 1,000, 2,000 and 4,000 Hz, consonant discrimination, and word recognition were measured in quiet and in a continuous impulse noise background. The results showed that the attenuation of sounds (i.e. the difference between protected and unoccluded thresholds) in quiet did not vary as a function of age or hearing loss for any of the four protectors. In noise, the difference between protected and unoccluded listening was close to zero, as long as hearing was normal. With hearing loss as a factor, there was a significant increment in the protected threshold, the amount determined by the device. Word recognition in quiet was adversely affected in normal-hearing listeners by the three attenuating devices but improved in noise relative to unoccluded listening. Amplification had a deleterious effect for both consonant discrimination and word recognition in noise. In hearing-impaired listeners, speech perception was impeded by all four muffs but less so in quiet with limited amplification.