Best Practices for the Electronic Implementation and Migration of Patient-Reported Outcome Measures.

While the use of electronic methods to collect patient-reported outcome data in clinical trials continues to increase, it remains the case that many patient-reported outcome measures (PROMs) have originally been developed and validated on paper. Careful consideration during the move from paper PROMs to electronic format is required to preserve the integrity of the measure and ensure a "faithful migration." Relevant literature has long called out the importance of following migration best practices during this process; nevertheless, such best practices are distributed across multiple documents. This article consolidates and builds upon existing electronic PROM implementation best practice recommendations to provide a comprehensive, up-to-date, single point of reference. It reflects the current consensus based on the significant advances in technology capabilities and knowledge gleaned from the growing evidence base on electronic migration and implementation, to balance the need for maintaining the integrity of the measure while optimizing respondent usability. It also specifies whether the practice is rooted in evidence or expert consensus, to enable those using these best practices to make informed and considered decisions when conducting migration.

Updated Recommendations on Evidence Needed to Support Measurement Comparability Among Modes of Data Collection for Patient-Reported Outcome Measures: A Good Practices Report of an ISPOR Task Force.

The ISPOR Task Force on measurement comparability between modes of data collection for patient-reported outcome measures (PROMs) has updated the good practice recommendations from the 2009 ISPOR electronic patient-reported outcome and 2014 patient-reported outcome mixed modes Good Research Practices Task Force reports in light of accumulated evidence of measurement comparability among different modes of PROM data collection. Furthermore, with the increasing use of electronic formats of clinical outcome assessments in clinical trials and the US Food and Drug Administration's encouragement of electronic data collection, this new task force report provides stakeholders with best practice recommendations reflecting the current body of evidence and enables them to respond to future developments in research and technology. This task force recommends an evidence-based approach to determine whether new research is needed to evaluate measurement comparability for a given questionnaire or technology. The suitability of existing evidence depends upon whether it satisfactorily demonstrates that the change in data collection mode has not affected the PROM's measurement properties. In cases where sufficient evidence of measurement comparability exists and best practices for faithful migration are followed, this task force concludes that further testing of measurement comparability among the data collection modes is unnecessary, including cases of "mixing modes" within clinical trials such as bring your own device designs.

Recommendations for the Electronic Migration and Implementation of Clinician-Reported Outcome Assessments in Clinical Trials.

OBJECTIVES: Although best practices from electronic patient-reported outcome (PRO) measures are transferable, the migration of clinician-reported outcome (ClinRO) assessments to electronic modes requires recommendations that address their unique properties, such as the user (eg, clinician), and complexity associated with programming of clinical content. Faithful migration remains essential to ensuring that the content and psychometric properties of the original scale (ie, validated reference) are preserved, such that clinicians completing the ClinRO assessments interpret and respond to the items the same way regardless of data collection mode. The authors present a framework for how to "faithfully" migrate electronic ClinRO assessments for successful deployment in clinical trials. METHODS: Critical Path Institute's Electronic PRO Consortium and PRO Consortium convened a consensus panel of representatives from member firms to develop recommendations for electronic migration and implementation of ClinRO assessments in clinical trials based on industry standards, regulatory guidelines where available, and relevant literature. The recommendations were reviewed and approved by all member firms from both consortia. CONSENSUS RECOMMENDATIONS: Standard, minimal electronic modifications for ClinRO assessments are described. This article also outlines implementation steps, including planning, startup, electronic clinical outcome assessment system development, training, and deployment. The consensus panel proposes that functional clinical testing by a clinician or clinical outcome assessment expert, as well as copyright holder review of screenshots (if possible) are sufficient to support minimal modifications during migration. Additional evidence generation is proposed for modifications that deviate significantly from the validated reference.

Selection of and Evidentiary Considerations for Wearable Devices and Their Measurements for Use in Regulatory Decision Making: Recommendations from the ePRO Consortium.

BACKGROUND: Wearable devices offer huge potential to collect rich sources of data to provide insights into the effects of treatment interventions. Despite this, at the time of writing this report, limited regulatory guidance on the use of wearables in clinical trial programs has been published. OBJECTIVES: To present recommendations from the Critical Path Institute's Electronic Patient-Reported Outcome Consortium regarding the selection and evaluation of wearable devices and their measurements for use in regulatory trials and to support labeling claims. METHODS: The evaluation group was composed of Critical Path Institute's clinical outcome assessment (COA) scientists and COA specialists from pharmaceutical trial eCOA solution providers, including COA development and validation specialists. The resulting recommendations were drawn from a broad range of backgrounds, perspectives, and expertise that enriched the development of this report. Recommendations were developed through analysis of existing regulatory guidance relating to COA development and use in clinical trials, medical device certification/clearance regulations, literature-reported best practice, and practical experience of wearable technology application in clinical trials. RESULTS: We identify the essential properties of fit-for-purpose wearables and propose evidence needed to support their use. In addition, we overview the activities required to establish clinical endpoints derived from wearables data. CONCLUSIONS: Using this framework, we believe there is enough current understanding to promote the appropriate use of wearables in study protocols. We hope this will provide a basis for discussion among clinical trial stakeholders and catalyze the development of more robust regulatory guidance.

Equivalence of electronic and paper administration of patient-reported outcome measures: a systematic review and meta-analysis of studies conducted between 2007 and 2013.

OBJECTIVE: To conduct a systematic review and meta-analysis of the equivalence between electronic and paper administration of patient reported outcome measures (PROMs) in studies conducted subsequent to those included in Gwaltney et al's 2008 review. METHODS: A systematic literature review of PROM equivalence studies conducted between 2007 and 2013 identified 1,997 records from which 72 studies met pre-defined inclusion/exclusion criteria. PRO data from each study were extracted, in terms of both correlation coefficients (ICCs, Spearman and Pearson correlations, Kappa statistics) and mean differences (standardized by the standard deviation, SD, and the response scale range). Pooled estimates of correlation and mean difference were estimated. The modifying effects of mode of administration, year of publication, study design, time interval between administrations, mean age of participants and publication type were examined. RESULTS: Four hundred thirty-five individual correlations were extracted, these correlations being highly variable (I2 = 93.8) but showing generally good equivalence, with ICCs ranging from 0.65 to 0.99 and the pooled correlation coefficient being 0.88 (95% CI 0.87 to 0.88). Standardised mean differences for 307 studies were small and less variable (I2 = 33.5) with a pooled standardised mean difference of 0.037 (95% CI 0.031 to 0.042). Average administration mode/platform-specific correlations from 56 studies (61 estimates) had a pooled estimate of 0.88 (95% CI 0.86 to 0.90) and were still highly variable (I2 = 92.1). Similarly, average platform-specific ICCs from 39 studies (42 estimates) had a pooled estimate of 0.90 (95% CI 0.88 to 0.92) with an I2 of 91.5. After excluding 20 studies with outlying correlation coefficients (≥3SD from the mean), the I2 was 54.4, with the equivalence still high, the overall pooled correlation coefficient being 0.88 (95% CI 0.87 to 0.88). Agreement was found to be greater in more recent studies (p < 0.001), in randomized studies compared with non-randomised studies (p < 0.001), in studies with a shorter interval (<1 day) (p < 0.001), and in respondents of mean age 28 to 55 compared with those either younger or older (p < 0.001). In terms of mode/platform, paper vs Interactive Voice Response System (IVRS) comparisons had the lowest pooled agreement and paper vs tablet/touch screen the highest (p < 0.001). CONCLUSION: The present study supports the conclusion of Gwaltney's previous meta-analysis showing that PROMs administered on paper are quantitatively comparable with measures administered on an electronic device. It also confirms the ISPOR Taskforce´s conclusion that quantitative equivalence studies are not required for migrations with minor change only. This finding should be reassuring to investigators, regulators and sponsors using questionnaires on electronic devicesafter migration using best practices. Although there is data indicating that migrations with moderate changes produce equivalent instrument versions, hence do not require quantitative equivalence studies, additional work is necessary to establish this. Furthermore, there is the need to standardize migration practices and reporting practices (i.e. include copies of tested instrument versions and screenshots) so that clear recommendations regarding equivalence testing can be made in the future.raising questions about the necessity of conducting equivalence testing moving forward.

Long-acting olanzapine versus long-acting risperidone for schizophrenia in Spain - a cost-effectiveness comparison.

BACKGROUND: In schizophrenia, medication adherence is critical to achieve better patient outcomes and to avoid relapses, which are responsible for a significant proportion of total healthcare costs for this chronic illness. The aim of this study was to assess the cost-effectiveness of olanzapine long-acting injection (OLAI) compared with risperidone long-acting injection (RLAI) in patients with schizophrenia in Spain. METHODS: A discrete event simulation (DES) model was developed from a Spanish healthcare system perspective to estimate clinical and economic outcomes for patients with schizophrenia over a five-year period. Patients who had earlier responded to oral medication and have a history of relapse due to adherence problems were considered. Identical model populations were treated with either OLAI or RLAI. In the absence of a head-to-head clinical trial, discontinuation and relapse rates were obtained from open-label studies. The model accounted for age, gender, risks of relapse and discontinuation, relapse management, hospitalization, treatment switching and adverse events. Direct medical costs for the year 2011 and outcomes including relapse avoided, life years (LYs), and quality-adjusted life years (QALYs) were discounted at a rate of 3%. RESULTS: When comparing RLAI and OLAI, the model predicts that OLAI would decrease 5-year costs by €2,940 (Standard Deviation between replications 300.83), and result in a QALY and LY gains of 0.07 (SD 0.019) and 0.04 (SD 0.025), respectively. Patients on OLAI had fewer relapses compared to RLAI (1.392 [SD 0.035] vs. 1.815 [SD 0.035]) and fewer discontinuations (1.222 [SD 0.031] vs. 1.710 [SD 0.039]). Sensitivity analysis indicated that the study was robust and conclusions were largely unaffected by changes in a wide range of parameters. CONCLUSIONS: The present evaluation results in OLAI being dominant over RLAI, meaning that OLAI represents a more effective and less costly alternative compared to RLAI in the treatment of patients with schizophrenia in the Spanish setting.

Comparability of a provisioned device versus bring your own device for completion of patient-reported outcome measures by participants with chronic obstructive pulmonary disease: qualitative interview findings.

BACKGROUND: There is interest in participants using their own smartphones or tablets ("bring your own device"; BYOD) to complete patient-reported outcome (PRO) measures in clinical studies. Our study aimed to qualitatively evaluate participants' experience using a provisioned device (PD) versus their own smartphone (BYOD) for this purpose. METHODS: Participants with chronic obstructive pulmonary disease (COPD) were recruited for this observational, cross-over study and completed PRO measures daily on one device type for 15 days, then switched to the other device type to complete the same measures for another 15 days. After each 15-day period, semi-structured interviews were conducted about their experience with the device. RESULTS: Of 64 participants enrolled, the final qualitative analysis populations comprised those who participated in an interview without protocol violations. Thus, the qualitative longitudinal population (LP) included n = 57 (89%), while the qualitative cross-sectional population (CSP) included n = 60 (94%). CSP participants found both device types easy to use. Twenty CSP participants (33%) reported missing data entry on at least one day when using PD, and 24 (40%) reported missing at least one day when using BYOD. In the LP, preference for one of the device types was somewhat evenly split; 45.6% (n = 26) preferred PD and 50.9% (n = 29) preferred BYOD. The most common reason for preferring PD was that it was "dedicated" to the study; the "convenience" of carrying a single device was the main reason for preferring BYOD. CONCLUSION: The findings from the interviews demonstrated few differences in participants' experience completing PRO measures on a PD versus BYOD. Our study supports the use of BYOD as a potential addition to PD for collecting PRO data and contributes evidence that BYOD may be employed to collect PRO data in demographically diverse patient populations.

Comparability of a provisioned device versus bring your own device for completion of patient-reported outcome measures by participants with chronic obstructive pulmonary disease: quantitative study findings.

OBJECTIVE: To quantitatively compare equivalence and compliance of patient-reported outcome (PRO) data collected via provisioned device (PD) versus bring your own device (BYOD). METHODS: Participants with stable chronic obstructive pulmonary disease (COPD) completed the EXAcerbations of Chronic Pulmonary Disease Tool (EXACT®) daily and COPD Assessment Test™ (CAT) and Patient Global Impression of Severity (PGIS) of COPD weekly on either PD or BYOD for 15 days, then switched device types for 15 days. EXACT was scored using the Evaluating Respiratory Symptoms in COPD (E-RS®: COPD) algorithm and equivalence assessed using intraclass correlation coefficients (ICCs) adjusting for cross-over sequence, period, and time. Two one-sided tests (TOSTs) used ICC adjusted means with 10%, 20%, and 40% of total score tested as equivalence margins. Compliance and comfort with technology were assessed. Equivalence across 3 device screen sizes was assessed following the second completion period. RESULTS: Participants (N = 64) reported high comfort with technology, with 79.7% reporting being "quite a bit" or "very" comfortable. Weekly compliance was high (BYOD = 89.7-100%; PD = 76.9-100%). CAT and E-RS: COPD scores correlated well with PGIS (r > 0.50) and demonstrated equivalence between PD and BYOD completion (ICC = 0.863-0.908). TOST equivalence was achieved within 10% of the total score (p > 0.05). PRO measure scores were equivalent across 3 different screen sizes (ICC = 0.972-0.989). CONCLUSIONS: Measure completion was high and scores equivalent between PD and BYOD, supporting use of BYOD in addition to PD for collecting PRO data in COPD studies and in demographically diverse patient populations.

In vitro fertilisation with recombinant follicle stimulating hormone requires less IU usage compared with highly purified human menopausal gonadotrophin: results from a European retrospective observational chart review.

BACKGROUND: Previous studies have reported conflicting results for the comparative doses of recombinant follicle stimulating hormone (rFSH) and highly purified human menopausal gonadotrophin (hMG-HP) required per cycle of in vitro fertilisation (IVF); the aim of this study was to determine the average total usage of rFSH versus hMG-HP in a 'real-world' setting using routine clinical practice. METHODS: This retrospective chart review of databases from four European countries investigated gonadotrophin usage, oocyte and embryo yield, and pregnancy outcomes in IVF cycles (± intra-cytoplasmic sperm injection) using rFSH or hMG-HP alone. Included patients met the National Institute for Health and Clinical Excellence (NICE) guideline criteria for IVF and received either rFSH or hMG-HP. Statistical tests were conducted at 5% significance using Chi-square or t-tests. RESULTS: Of 30,630 IVF cycles included in this review, 74% used rFSH and 26% used hMG-HP. A significantly lower drug usage per cycle for rFSH than hMG-HP (2072.53 +/- 76.73 IU vs. 2540.14 +/- 883.08 IU, 22.6% higher for hMG-HP; p < 0.01) was demonstrated. The median starting dose was also significantly lower for rFSH than for hMG-HP (150 IU vs. 225 IU, 50% higher for hMG-HP, p < 0.01). The average oocyte yield per IVF cycle in patients treated with rFSH was significantly greater than with hMG-HP (10.80 +/- 6.02 vs. 9.77 +/- 5.53; p < 0.01), as was the average mature oocyte yield (8.58 +/- 5.27 vs. 7.72 +/- 4.59; p < 0.01). No significant differences were observed in pregnancy outcomes including spontaneous abortion between the two treatments. There was a significantly higher rate of OHSS (all grades) with rFSH (18.92% vs. 14.09%; p < 0.0001). The hospitalisation rate due to OHSS was low but significantly higher in the rFSH group (1.07% of cycles started vs. 0.67% of cycles started with rFSH and hMG-HP, respectively; p = 0.002). CONCLUSIONS: Based on these results, IVF treatment cycles with rFSH yield statistically more oocytes (and more mature oocytes), using significantly less IU per cycle, versus hMG-HP. The incidence of all OHSS and hospitalisations due to OHSS was significantly higher in the rFSH cycles compared to the hMG-HP cycles. However, the absolute incidence of hospitalisations due to OHSS was similar to that reported previously. These results suggest that the perceived required dosage with rFSH is currently over-estimated, and the higher unit cost of rFSH may be offset by a lower required dosage compared with hMG-HP.

Best Practices for Avoiding Paper Backup When Implementing Electronic Approaches to Patient-Reported Outcome Data Collection in Clinical Trials.

Electronic data capture is fast becoming the preferred method of collecting patient-reported outcome (PRO) data in clinical trials. Data collection can be site-based (clinical study site), and typically collected on a tablet, or field-based (subject's typical environment such as home, school, or workplace), and most often accomplished with handheld devices, such as a smartphone. While site and study subject compliance with protocol-specific data collection procedures using these devices is critical to trial success, so is the robustness of the device hardware and the software these devices use to capture the trial data. Technology failures and/or site or subject resistance to the electronic data capture protocol may lead a subject to record data on paper, which can result in undesirable data challenges. As such, both site and subject compliance issues and technology-related factors must be anticipated to adhere to the ePRO data collection plan. The objective of this paper is to provide the technology industry's best practice recommendations for optimizing ePRO data collection in clinical trials by proposing the inclusion of a planned approach to data collection that includes viable electronic backup strategies so that defaulting to a paper-based backup becomes unnecessary.

An analysis of the radioactive contamination due to radon in a granite processing plant and its decontamination by ventilation.

This work focuses on studying concentration distribution of 222Rn radioisotope in a granite processing plant. Using Computational Fluid Dynamic Techniques (CFD), the exposure of the workers to radiation was assessed and, in order to minimise this exposure, different decontamination scenarios using ventilation were analysed. Natural ventilation showed not sufficient to maintain radon concentration below acceptable limits, so a forced ventilation was used instead. Position of the granite blocks also revealed as a determining factor in the radioactive level distribution. Thus, a correct layout of the stored material and an adequate ventilation system can guarantee free of exposure to radiation zones within the studied workshop. This leads to a drastic fall in the exposure of the workers and consequently minimises their risk of developing aggressive illness like lung cancer.

Considerations for Requiring Subjects to Provide a Response to Electronic Patient-Reported Outcome Instruments.

The increase in the use of electronic patient-reported outcome (ePRO) instruments has presented study teams with considerations not previously encountered with paper. Specifically, in an effort to minimize missing data, there is now the opportunity of requiring subjects to provide a response to an item before allowing the subject to proceed to the next item. While the ability to require subjects to respond to ePRO items would seem to guarantee a complete data set, it raises questions about the conditions under which it is appropriate to require subjects to respond to the items in an instrument. This article provides guidance on the circumstances under which allowing a subject to opt out of responding to ePRO items may be appropriate. Three main scenarios are discussed: (1) requiring subjects to complete all items in all the instruments in the study, (2) allowing subjects to opt out of at least some selective items that do not support key primary or secondary endpoints, and (3) allowing subjects to opt out of responding to any or all items in the study. For either of the 2 scenarios allowing the subject to opt out of responding to an item, the use of programmed edit checks is highly recommended to confirm that the subject intended to "skip" or "opt out of" the item. This ensures that, at the end of the study, the database contains an explicit data point indicating when a subject has actively decided to skip an item. While this article is focused on patient-reported outcomes, the issues raised could also apply to other clinical outcome assessments, such as clinician- and observer-reported outcomes.

Capturing Patient-Reported Outcome (PRO) Data Electronically: The Past, Present, and Promise of ePRO Measurement in Clinical Trials.

Patient-reported outcomes (PROs) are an important means of evaluating the treatment benefit of new medical products. It is recognized that PRO measures should be used when assessing concepts best known by the patient or best measured from the patient's perspective. As a result, there is growing emphasis on well defined and reliable PRO measures. In addition, advances in technology have significantly increased electronic PRO (ePRO) data collection capabilities and options in clinical trials. The movement from paper-based to ePRO data capture has enhanced the integrity and accuracy of clinical trial data and is encouraged by regulators. A primary distinction in the types of ePRO platforms is between telephone-based interactive voice response systems and screen-based systems. Handheld touchscreen-based devices have become the mainstay for remote (i.e., off-site, unsupervised) PRO data collection in clinical trials. The conventional approach is to provide study subjects with a handheld device with a device-based proprietary software program. However, an emerging alternative for clinical trials is called bring your own device (BYOD). Leveraging study subjects' own Internet-enabled mobile devices for remote PRO data collection (via a downloadable app or a Web-based data collection portal) has become possible due to the widespread use of personal smartphones and tablets. However, there are a number of scientific and operational issues that must be addressed before BYOD can be routinely considered as a practical alternative to conventional ePRO data collection methods. Nevertheless, the future for ePRO data collection is bright and the promise of BYOD opens a new chapter in its evolution.

"Bring Your Own Device" (BYOD): The Future of Field-Based Patient-Reported Outcome Data Collection in Clinical Trials?

Field-based patient-reported outcome (PRO) assessments, including measures of signs, symptoms, and events that are administered outside of the research clinic, can be critical in evaluating the efficacy and safety of new medical treatments. Collection of this type of data commonly involves providing subjects with stand-alone electronic devices, such as smartphones, that they can use to respond to assessments in their home or work environment. Although this approach has proven useful, it is also limited in several ways: For example, provisioning stand-alone devices can be costly for sponsors, and requiring subjects to carry a device that is exclusively dedicated to the study can be burdensome. The "Bring Your Own Device" (BYOD) approach, in which subjects use their own smartphone or Internet-enabled device to complete field-based PRO assessments, addresses many of these concerns. However, the BYOD model has its own limitations that should be considered. In this article, representatives of the ePRO Consortium review operational, privacy/security, and scientific/regulatory considerations regarding BYOD. We hope that this review will allow researchers to make informed decisions when choosing methods to collect field-based PRO data in future clinical trials. Additionally, we hope that the discussion in this article will establish a research agenda for further examination of BYOD approaches.