Getting serious about test-retest reliability: a critique of retest research and some recommendations.

ABSTRACT

PURPOSE: To focus attention on the need for rigorous and carefully designed test-retest reliability assessments for new patient-reported outcomes and to encourage retest researchers to be thoughtful, ambitious, and creative in their retest efforts. METHODS: The paper outlines key challenges that confront retest researchers, calls attention to some limitations in meeting those challenges, and describes some strategies to improve retest research. RESULTS: Modest retest coefficients are often reported as acceptable, and many important decisions-such as the retest interval-appear not to be evidence-based. Retest assessments are seldom undertaken before a measure has been finalized, which rules out using retest data to select strong, reproducible items. CONCLUSIONS: Strategies for improving retest research include seeking input from patients or experts regarding the stability of the construct to support decisions about the retest interval, analyzing item-level retest data to identify items to revise or discard, establishing a priori standards of acceptability for reliability coefficients, using large, heterogeneous, and representative retest samples and collecting follow-up data to better understand consistent and inconsistent responses over time.