A comparison of two experimental design approaches in applying conjoint analysis in patient-centered outcomes research: a randomized trial.

BACKGROUND: While the application of conjoint analysis and discrete-choice experiments in health are now widely accepted, a healthy debate exists around competing approaches to experimental design. There remains, however, a paucity of experimental evidence comparing competing design approaches and their impact on the application of these methods in patient-centered outcomes research. OBJECTIVES: Our objectives were to directly compare the choice-model parameters and predictions of an orthogonal and a D-efficient experimental design using a randomized trial (i.e., an experiment on experiments) within an application of conjoint analysis studying patient-centered outcomes among outpatients diagnosed with schizophrenia in Germany. METHODS: Outpatients diagnosed with schizophrenia were surveyed and randomized to receive choice tasks developed using either an orthogonal or a D-efficient experimental design. The choice tasks elicited judgments from the respondents as to which of two patient profiles (varying across seven outcomes and process attributes) was preferable from their own perspective. The results from the two survey designs were analyzed using the multinomial logit model, and the resulting parameter estimates and their robust standard errors were compared across the two arms of the study (i.e., the orthogonal and D-efficient designs). The predictive performances of the two resulting models were also compared by computing their percentage of survey responses classified correctly, and the potential for variation in scale between the two designs of the experiments was tested statistically and explored graphically. RESULTS: The results of the two models were statistically identical. No difference was found using an overall chi-squared test of equality for the seven parameters (p = 0.69) or via uncorrected pairwise comparisons of the parameter estimates (p-values ranged from 0.30 to 0.98). The D-efficient design resulted in directionally smaller standard errors for six of the seven parameters, of which only two were statistically significant, and no differences were found in the observed D-efficiencies of their standard errors (p = 0.62). The D-efficient design resulted in poorer predictive performance, but this was not significant (p = 0.73); there was some evidence that the parameters of the D-efficient design were biased marginally towards the null. While no statistical difference in scale was detected between the two designs (p = 0.74), the D-efficient design had a higher relative scale (1.06). This could be observed when the parameters were explored graphically, as the D-efficient parameters were lower. CONCLUSIONS: Our results indicate that orthogonal and D-efficient experimental designs have produced results that are statistically equivalent. This said, we have identified several qualitative findings that speak to the potential differences in these results that may have been statistically identified in a larger sample. While more comparative studies focused on the statistical efficiency of competing design strategies are needed, a more pressing research problem is to document the impact the experimental design has on respondent efficiency.

Can patients diagnosed with schizophrenia complete choice-based conjoint analysis tasks?

BACKGROUND: Schizophrenia is a severe mental illness associated with hallucinations, delusions, apathy, poor social functioning, and impaired cognition. Researchers and funders have been hesitant to focus efforts on treatment preferences of patients with schizophrenia because of the perceived cognitive burden that research methods, such as conjoint analysis, place on them. OBJECTIVE: The objective of this study was to test if patients diagnosed with schizophrenia were able to complete a choice-based conjoint analysis (often referred to as discrete-choice experiments) and to test if meaningful trade-offs were being made. METHODS: German outpatients diagnosed with schizophrenia were eligible to participate in this study if they were aged 18-65 years, had received treatment for at least 1 year and were not experiencing acute symptoms. Conjoint analysis tasks were based on six attributes, each with two levels, which were identified via a literature review and focus groups. A psychologist in a professional interview facility presented each respondent with the eight tasks with little explanation. All interviews were recorded, transcribed, and analyzed to verify that respondents understood the tasks. Preferences were assessed using logistic regression, with a correction for clustering. RESULTS: We found evidence that the 21 patients diagnosed with schizophrenia participating in the study could complete conjoint analysis tasks in a meaningful way. Patients not only related to the scenarios presented in conjoint tasks, but explicitly stated that they used their own preferences to judge which scenarios were better. Statistical analysis confirmed all hypotheses about the attributes (i.e. all attributes had the expected sign). Having a supportive physician, not feeling slowed, and improvements in stressful situations (p < 0.01) were the most important attributes. CONCLUSIONS: We found that patients diagnosed with schizophrenia can complete conjoint analysis tasks, that they base their decisions on their own preferences, and that patients make trade-offs between attributes.

A discrete-event simulation of smoking-cessation strategies based on varenicline pivotal trial data.

BACKGROUND: Smoking is the leading cause of preventable death in the US. While one in five individuals smoke, and 70% of these indicate a desire to quit, <5% of unaided quit attempts succeed. Cessation aids can double or triple the odds of successfully quitting. Models of smoking-cessation behaviour can elucidate the implications of individual abstinence patterns to allow better tailoring of quit attempts to an individual's characteristics. OBJECTIVE: The objectives of this study were to develop and validate a discrete-event simulation (DES) to evaluate the benefits of smoking abstinence using data from the pooled pivotal clinical trials of varenicline versus bupropion or placebo for smoking cessation and to provide a foundation for the development of a lifetime smoking-cessation model. METHODS: The DES model simulated the outcome of a single smoking-cessation attempt over 1 year, in accordance with the clinical trial timeframes. Pharmaceutical costs were assessed from the perspective of a healthcare payer. The model randomly sampled patient profiles from the pooled varenicline clinical trials. All patients were physically and mentally healthy adult smokers who were motivated to quit abruptly. The model allowed for comparisons of up to five distinct treatment approaches for smoking cessation. In the current analyses, three interventions corresponding to the clinical trials were evaluated, which included brief counselling plus varenicline 1.0 mg twice daily (bid) or bupropion SR 150 mg bid versus placebo (i.e. brief counselling only). The treatment periods in the clinical trials were 12 weeks (target quit date: day 8), with a 40-week non-treatment follow-up, and counselling continuing over the entire 52-week period in all treatment groups. The main outcome modelled was the continuous abstinence rate (CAR; defined as complete abstinence from smoking and confirmed by exhaled carbon monoxide ≤ 10 ppm) at end of treatment (weeks 9-12) and long-term follow-up (weeks 9-52), and total time abstinent from smoking over the course of 52 weeks. The model also evaluated costs and cost-effectiveness outcomes. RESULTS: For the varenicline, bupropion and placebo cohorts, respectively, the model predicted CARs for weeks 9-12 of 44.3%, 30.4% and 18.6% compared with observed rates of 44.0%, 29.7% and 17.7%; over weeks 9-52, predicted CARs in the model compared with observed rates in the pooled clinical studies were 22.9%, 16.4% and 9.4% versus 22.4%, 15.4% and 9.3%, respectively. Total mean abstinence times accrued in the model varenicline, bupropion and placebo groups, respectively, were 3.6, 2.6 and 1.5 months and total pharmaceutical treatment costs were $US261, $US442 and $US0 (year 2008 values) over the 1-year model period. Using cost per abstinent-month achieved as a measure of cost effectiveness, varenicline dominated bupropion and yielded an incremental cost-effectiveness ratio of $US124 compared with placebo. CONCLUSION: The model accurately replicated abstinence patterns observed in the clinical trial data using individualized predictions and indicated that varenicline was more effective and may be less costly than bupropion. This simulation incorporated individual predictions of abstinence and relapse, and provides a framework for lifetime modelling that considers multiple quit attempts over time in diverse patient populations using a variety of quit attempt strategies.

Identifying patient-relevant endpoints among individuals with schizophrenia: an application of patient-centered health technology assessment.

OBJECTIVES: Schizophrenia imposes a great burden on society, and while evaluation should play an important role in informing society's efforts to alleviate these burdens, it is unclear what "endpoints" should be chosen as the objective of such analyses. The objectives of the study were to elicit endpoints directly from patients with schizophrenia, to ascertain whether patients are sufficiently cognoscente to express what endpoints are and are not important to them and to rank the relevant endpoints. METHODS: We applied principles of patient-centered health technology assessment to identify and value endpoints from the patient's perspective. Focus groups were conducted to elicit endpoints, using interpretive phenomalogical analysis (IPA) to guide the collection, analysis and interpretation of data. Patient interviews were subsequently used to elicit patient preference over endpoints. Respondents were presented with cards outlining the endpoints and asked to remove irrelevant cards. They where then asked to identify and rank their five most relevant endpoints in order of importance. Interviews were recorded for the purposed of triangulation, and data was analyzed using descriptive statistics. Patients were recruited from five geographically diverse cities in Germany. Eligibility required a diagnosis of schizophrenia by a physician and treatment with an antipsychotic medication for at least one year. Respondents were excluded if they were experiencing an acute episode. RESULTS: Thirteen endpoints emerged as important from the focus groups spanning side-effects, functional status, processes of care and clinical outcomes. Respondents could clearly identify relevant and irrelevant endpoints, and rank which factors were important to them. Triangulation between field notes of the ranking exercise and recordings confirmed that rankings were not arbitrary, but justified from the respondents' point of view. CONCLUSIONS: Patients with schizophrenia can express preferences over endpoints. Our results show that qualitative methods such as IPA can be used to identify factors, but ranking exercises provide a more robust method for ranking the importance of endpoints. Future research involving patients with schizophrenia ranking outcomes is needed to identify variations across patients and methods such as conjoint analysis could prove beneficial in identifying acceptable tradeoffs across endpoints.

Things are Looking up Since We Started Listening to Patients: Trends in the Application of Conjoint Analysis in Health 1982-2007.

Clinical and healthcare decision makers have repeatedly endorsed patient-centered care as a goal of the health system. However, traditional methods of evaluation reinforce societal views, and research focusing on views of patients is often referred to as 'soft science.' Conjoint analysis presents a scientifically rigorous research tool that can be used to understand patient preferences and inform decision making. This paper documents applications of conjoint analysis in medicine and systematically reviews this literature in order to identify publication trends and the range of topics to which conjoint analysis has been applied. In addition, we document important methodological aspects such as sample size, experimental design, and method of analysis.Publications were identified through a MEDLINE search using multiple search terms for identification. We classified each article into one of three categories: clinical applications (n = 122); methodological contributions (n = 56); and health system applications (n = 47). Articles that did not use or adequately discuss conjoint analysis methods (n = 164) were discarded. We identified a near exponential increase in the application of conjoint analyses over the last 10 years of the study period (1997-2007). Over this period, the proportion of applications on clinical topics increased from 40% of articles published in MEDLINE from 1998 to 2002, to 64% of articles published from 2003 to 2007 (p = 0.002).The average sample size among articles focusing on health system applications (n = 556) was significantly higher than clinical applications (n = 277) [p = 0.001], although this 2-fold difference was primarily due to a number of outliers reporting sample sizes in the thousands. The vast majority of papers claimed to use orthogonal factorial designs, although over a quarter of papers did not report their design properties. In terms of types of analysis, logistic regression was favored among clinical applications (28%), while probit was most commonly used among health systems applications (38%). However, 25% of clinical applications and 33% of health systems articles failed to report what regression methods were used. We used the International Classification of Diseases - version 9 (ICD-9) coding system to categorize clinical applications, with approximately 26% of publications focusing on neoplasm. Program planning and evaluation applications accounted for 22% of the health system articles.While interest in conjoint analysis in health is likely to continue, better guidelines for conducting and reporting conjoint analyses are needed.