Equivalence of electronic and paper administration of patient-reported outcome measures: a systematic review and meta-analysis of studies conducted between 2007 and 2013.

OBJECTIVE: To conduct a systematic review and meta-analysis of the equivalence between electronic and paper administration of patient reported outcome measures (PROMs) in studies conducted subsequent to those included in Gwaltney et al's 2008 review. METHODS: A systematic literature review of PROM equivalence studies conducted between 2007 and 2013 identified 1,997 records from which 72 studies met pre-defined inclusion/exclusion criteria. PRO data from each study were extracted, in terms of both correlation coefficients (ICCs, Spearman and Pearson correlations, Kappa statistics) and mean differences (standardized by the standard deviation, SD, and the response scale range). Pooled estimates of correlation and mean difference were estimated. The modifying effects of mode of administration, year of publication, study design, time interval between administrations, mean age of participants and publication type were examined. RESULTS: Four hundred thirty-five individual correlations were extracted, these correlations being highly variable (I2 = 93.8) but showing generally good equivalence, with ICCs ranging from 0.65 to 0.99 and the pooled correlation coefficient being 0.88 (95% CI 0.87 to 0.88). Standardised mean differences for 307 studies were small and less variable (I2 = 33.5) with a pooled standardised mean difference of 0.037 (95% CI 0.031 to 0.042). Average administration mode/platform-specific correlations from 56 studies (61 estimates) had a pooled estimate of 0.88 (95% CI 0.86 to 0.90) and were still highly variable (I2 = 92.1). Similarly, average platform-specific ICCs from 39 studies (42 estimates) had a pooled estimate of 0.90 (95% CI 0.88 to 0.92) with an I2 of 91.5. After excluding 20 studies with outlying correlation coefficients (≥3SD from the mean), the I2 was 54.4, with the equivalence still high, the overall pooled correlation coefficient being 0.88 (95% CI 0.87 to 0.88). Agreement was found to be greater in more recent studies (p < 0.001), in randomized studies compared with non-randomised studies (p < 0.001), in studies with a shorter interval (<1 day) (p < 0.001), and in respondents of mean age 28 to 55 compared with those either younger or older (p < 0.001). In terms of mode/platform, paper vs Interactive Voice Response System (IVRS) comparisons had the lowest pooled agreement and paper vs tablet/touch screen the highest (p < 0.001). CONCLUSION: The present study supports the conclusion of Gwaltney's previous meta-analysis showing that PROMs administered on paper are quantitatively comparable with measures administered on an electronic device. It also confirms the ISPOR Taskforce´s conclusion that quantitative equivalence studies are not required for migrations with minor change only. This finding should be reassuring to investigators, regulators and sponsors using questionnaires on electronic devicesafter migration using best practices. Although there is data indicating that migrations with moderate changes produce equivalent instrument versions, hence do not require quantitative equivalence studies, additional work is necessary to establish this. Furthermore, there is the need to standardize migration practices and reporting practices (i.e. include copies of tested instrument versions and screenshots) so that clear recommendations regarding equivalence testing can be made in the future.raising questions about the necessity of conducting equivalence testing moving forward.

Psychometric evaluation of the Sinonasal Outcome Test-16 and activity impairment assessment in acute bacterial sinusitis.

OBJECTIVE: To validate the Sinonasal Outcome Test-16 and Activity Impairment Assessment in patients with acute bacterial sinusitis. STUDY DESIGN: Data were used from a phase III clinical trial designed to evaluate the efficacy and safety of moxifloxacin 400 mg once daily for 5 consecutive days in the treatment of acute bacterial sinusitis. The psychometric properties and factor structure of the 2 measures were assessed. SETTING: Participants were given the measures to self-complete using either a telephone voice response system or a paper-and-pencil format. SUBJECTS AND METHODS: Three hundred seventy-four patients with acute bacterial sinusitis were used in the analysis. Patients received either a placebo or 400 mg moxifloxacin once daily. Patients were then reviewed at test of cure and follow-up. All analyses were conducted on a combined sample of placebo and active treatment patients. RESULTS: The Sinonasal Outcome Test-16 was associated with minimal missing data at baseline but a higher proportion by test of cure. There was no evidence of floor or ceiling effects and no significant skew. The Activity Impairment Assessment also had low missing data at baseline and no obvious floor or ceiling effects, but the data were not normally distributed. Both measures had good internal consistency. Convergent and divergent validity as well as sensitivity and the minimally important difference are also reported. CONCLUSION: The measures both have good psychometric properties and are suitable for use with patients with acute bacterial sinusitis. Both instruments are sensitive. The minimal important difference estimates for the Sinonasal Outcome Test-16 are quite high but are similar to estimates reported previously.

Utility and work productivity data for economic evaluation of breast cancer therapies in the Netherlands and Sweden.

OBJECTIVE: This study aimed to estimate utility values in laypeople and productivity loss for women with breast cancer in Sweden and the Netherlands. METHODS: To capture utilities, validated health state vignettes were used, which were translated into Dutch and Swedish. They described progressive disease, stable disease, and 7 grade 3/4 adverse events. One hundred members of the general public in each country rated the states using the visual analog scale and time trade-off method. To assess productivity, women who had recently completed or were currently receiving treatment for early or advanced breast cancer (the Netherlands, n = 161; Sweden, n = 52) completed the Work Productivity and Activity Impairment-General Health (WPAI-GH) questionnaire. Data were analyzed using means (SD). RESULTS: The utility study showed that the Swedish sample rated progressive and stable disease (mean, 0.61 [0.07] and 0.81 [0.05], respectively) higher than did the Dutch sample (0.49 [0.06] and 0.69 [0.05]). The health states incorporating the toxicities in both countries produced similar mean scores. Results of the WPAI-GH showed that those currently receiving treatment reported productivity reductions of 69% (the Netherlands) and 72% (Sweden); those who had recently completed therapy reported reductions of 41% (the Netherlands) and 40% (Sweden). CONCLUSIONS: The differences in the utility scores between the 2 countries underline the importance of capturing country-specific values. The significant impact of adverse events on health-related quality of life was also highlighted. The WPAI-GH results demonstrated how the negative impact of breast cancer on productivity persists after women have completed their treatment.