The Performance of Kaizen Tasks Across Three Online Discrete Choice Experiment Surveys: An Evidence Synthesis.

BACKGROUND: Kaizen is a Japanese term for continuous improvement (kai ~ change, zen ~ good). In a kaizen task, a respondent makes sequential choices to improve an object's profile, revealing a preference path. Including kaizen tasks in a discrete choice experiment has the advantage of collecting greater preference evidence than pick-one tasks, such as paired comparisons. OBJECTIVE AND METHODS: So far, three online discrete choice experiments have included kaizen tasks: the 2020 US COVID-19 vaccination (CVP) study, the 2021 UK Children's Surgery Outcome Reporting (CSOR) study, and the 2023 US EQ-5D-Y-3L valuation (Y-3L) study. In this evidence synthesis, we describe the performance of the kaizen tasks in terms of response behaviors, conditional logit and Zermelo-Bradley-Terry (ZBT) estimates, and their standard errors in each of the surveys. RESULTS: Comparing the CVP and Y-3L, including hold-outs (i.e., attributes shared by all alternatives) seems to reduce positional behavior by half. The CVP tasks excluded multi-level improvements; therefore, we could not estimate logit main effects directly. In the CSOR, only 12 of the 21 logit estimates are significantly positive (p < 0.05), possibly due to the fixed attribute order. All Y-3L estimates are significantly positive, and their predictions are highly correlated (Pearson: logit 0.802, ZBT 0.882) and strongly agree (Lin: logit 0.744, ZBT 0.852) with the paired-comparison probabilities. CONCLUSIONS: These discrete choice experiments offer important lessons for future studies: (1) include warm-up tasks, hold-outs, and multi-level improvements; (2) randomize the attribute order (i.e., up-down) at the respondent level; and (3) recruit smaller samples of respondents than traditional discrete choice experiments with only pick-one tasks.

Scale and rate heterogeneity in the EQ-5D-5L valuation.

OBJECTIVES: To estimate values on a quality-adjusted life year (QALY) scale using individual preference evidence, choice analyses typically include ancillary parameters, such as scale factors and discount rates. These parameters potentially differ among respondents. In this study, we investigated how allowing heterogeneity in scale and rate affects the estimation of EQ-5D-5L values. METHODS: Using the first wave of the 2016 EQ-5D-5L valuation study (N = 1017), we estimated a conditional logit (CL) model and three mixed logit models: random scale, random rate, and bivariate. Prior to the exploratory study, we hypothesized that scale and rate are correlated and that allowing heterogeneity in both parameters decreases the number of insignificant incremental effects. We confirmed the exploratory findings by re-estimating these models using paired comparison responses from a second wave (N = 1229). RESULTS: Scale and rate exhibited significant heterogeneity and were positively correlated. As hypothesized, allowing this heterogeneity improved the face validity of the EQ-5D-5L value set by reducing the number of insignificant incremental effects (from 6 to 2 p-values > 0.05; out of 20). Nevertheless, the CL and bivariate mixed logit estimates are highly correlated and concordant (Pearson correlation coefficient of 0.897, Spearman correlation coefficient of 0.888, Lin's concordance coefficient of 0.763). CONCLUSIONS: Allowing this heterogeneity adds three parameters to the estimation (two variances and a correlation) and improves the face validity of the EQ-5D-5L values. This finding may influence experimental design and choice analysis in health valuation more generally.

Child health valuation protocol for a discrete choice experiment comparing paired comparison and kaizen tasks and estimating US EQ-5D-Y-3L values on an experience scale.

INTRODUCTION: A decade ago, the first national valuation study of the EQ-5D-Y-3L (Y-3L) involved a discrete choice experiment (DCE) that asked 4155 US adult respondents to complete 40 paired comparisons, choosing between two dying children. Instead of choosing between dying children, the respondents in this novel protocol are asked whether 'being in a coma' is better or worse than experiencing 'health problems' (ie, experience scale) and how they would relieve health problems (ie, kaizen tasks). Our aims are to compare the preference evidence of the paired comparison and kaizen tasks and to conduct a DCE for the valuation of Y-3L profiles on an experience scale. METHODS AND ANALYSIS: Under this protocol, we will conduct an online survey that collects preference evidence from 600 US adult respondents on the health of a 10-year-old child for a week. Across all scenarios, each child will be described as either being 'in a coma' or having 'health problems', namely five three-level attributes (Y-3L). In this DCE, each respondent will be randomly assigned to one of four D-efficient blocks, including five coma comparisons (ie, Y-3L vs coma), 10 paired comparisons (Y-3L vs Y-3L) and 10 kaizen tasks (preference paths). In addition to comparing evidence by task (aim 2), the analysis plan includes the estimation of main-effects conditional logit models to create a Y-3L value set on an 'experience scale' where positive (negative) experiences have positive (negative) values (0 is 'being in a coma' and 1 is full health). ETHICS AND DISSEMINATION: The institutional review board (IRB) (Advarra) determined that this project (Pro00072276) is exempt from IRB oversight based on DHHS 45 CFR 46.104(d)(2) and is not subject to requirements for continuing review. The results will be prepared for publication in peer-reviewed journals and presented at scientific meetings. The data and code will be made available on reasonable request.