Choice Consistency in Discrete Choice Experiments: Does Numeracy Skill Matter?

ABSTRACT

OBJECTIVES: This study investigated the relationship between numeracy skills (NS) and choice consistency in discrete choice experiments (DCEs). METHODS: A DCE was conducted to explore patients' preferences for kidney transplantation in Italy. Patients completed the DCE and answered 3-item numeracy questions. A heteroskedastic multinomial logit model was used to investigate the effect of numeracy on choice consistency. RESULTS: Higher NS were associated with greater choice consistency, increasing the scale to 1.63 (P < .001), 1.39 (P < .001), and 1.18 (P < .001) for patients answering 3 of 3, 2 of 3, and 1 of 3 questions correctly, respectively, compared with those with no correct answers. This corresponded to 63%, 39%, and 18% more consistent choices, respectively. Accounting for choice consistency resulted in varying willingness-to-wait (WTW) estimates for kidney transplant attributes. Patients with the lowest numeracy (0/3) were willing to wait approximately 42 months [95% CI 29.37, 54.68] for standard infectious risk, compared with 33 months [95% CI 28.48, 38.09] for 1 of 3, 28 months [95% CI 25.13, 30.32] for 2 of 3, and 24 months [95% CI 20.51, 27.25] for 3 of 3 correct answers. However, WTW differences for an additional year of graft survival and neoplastic risk were not statistically significant across numeracy levels. Supplementary analyses of 2 additional DCEs on COVID-19 vaccinations and rheumatoid arthritis, conducted online, supported these findings: higher NS were associated with more consistent choices across different disease contexts and survey formats. CONCLUSIONS: The findings suggested that combining patients with varying NS could bias WTW estimates, highlighting the need to consider numeracy in DCE data analysis and interpretation.