Test-retest reliability of a mobile application of the patient reported outcomes burdens and experiences (PROBE) study.

INTRODUCTION: The Patient Reported Outcomes, Burdens, and Experiences (PROBE) questionnaire is a patient-reported outcome tool that assesses quality of life and disease burden in people with haemophilia (PWH). AIM: To assesses the test-retest reliability of PROBE when completed using the mobile phone application. METHODS: We recruited PWH, including carriers, and individuals with no bleeding disorders who attended haemophilia-related workshops or via social media. Participants completed PROBE three times (twice on the app: T1 and T2, and once on the web, T3). Test-retest reliability was analysed for T1 versus T2 (app to app, time period one) and T2 versus T3 (app to web, time period two). RESULTS: We enrolled 48 participants (median age = 56 [range 27-78] years). Eighteen participants (37.5%) were PWH and seven (14.6%) were carriers. On general health domain questions, we found almost perfect agreement, except for a question on the frequency of use of pain medication in the last 12 months [Kappa coefficient (κ) .72 and .37 for time period one and two, respectively] and any use of pain medications (κ .75) for time period two. For haemophilia-related questions, we found substantial to perfect agreement, except for the questions on the number of joint bleeds in the previous 6 months for time period one (κ .49) and the number of bleeds in the previous two weeks for time period two (κ .34). CONCLUSIONS: The results demonstrate the reliability of the PROBE app. The app can be used interchangeably with the paper and web platforms for PROBE administration.

User-Centered Development and Testing of the Online Patient-Reported Outcomes, Burdens, and Experiences (PROBE) Survey and the myPROBE App and Integration With the Canadian Bleeding Disorder Registry: Mixed Methods Study.

BACKGROUND: The Patient-Reported Outcomes, Burdens, and Experiences (PROBE) questionnaire is a tool for assessing the quality of life and disease burden in people living with hemophilia. OBJECTIVE: The objectives of our study were (1) to assess the needs of relevant stakeholders involved in the use of PROBE, (2) to develop the software infrastructure needed to meet these needs, and (3) to test the usability of the final product. METHODS: We conducted a series of semistructured interviews of relevant stakeholders, including PROBE investigators, people with hemophilia, and representatives of the sponsor. Based on these, we developed an online survey and a mobile app for iOS and Android. A user group evaluated the final product using the System Usability Scale (SUS) and an open feedback framework. RESULTS: The online survey was updated, and the myPROBE app for mobile devices and a new application programming interface were developed. The app was tested and modified according to user feedback over multiple cycles. The final version of the app was released in July 2019. Seventeen users aged 23 to 67 years evaluated the final version of the app using the SUS. The median (first, third quartile) SUS score for the app was 85 (68, 88) out of 100. The newly introduced functionalities were as follows: (1) capability to longitudinally track repeated fillings of the questionnaire at different time points by the same participant (as opposed to anonymous completion); (2) linking of the questionnaire with hemophilia registries, starting with the Canadian Bleeding Disorders Registry as a proof of concept; (3) removing or adding questions as needed; and (4) sending notifications to the users (eg, reminders). A new secure database was built for securely storing personal information separately from the questionnaire data. The PROBE online survey is currently available in 96 countries and 34 languages. CONCLUSIONS: The online survey was updated successfully, and the myPROBE app was developed, with a SUS score of 85 (out of 100). The app has been released in 81 countries and 34 languages. This will facilitate data collection for research and advocacy purposes, and the use of this tool in everyday clinical practice.