Development and Validation of a Vocabulary Measure in the Mobile Toolbox.

OBJECTIVE: We describe the development of a new computer adaptive vocabulary test, Mobile Toolbox (MTB) Word Meaning, and validity evidence from 3 studies. METHOD: Word Meaning was designed to be a multiple-choice synonym test optimized for self-administration on a personal smartphone. The items were first calibrated online in a sample of 7,525 participants to create the computer-adaptive test algorithm for the Word Meaning measure within the MTB app. In Study 1, 92 participants self-administered Word Meaning on study-provided smartphones in the lab and were administered external measures by trained examiners. In Study 2, 1,021 participants completed the external measures in the lab and Word Meaning was self-administered remotely on their personal smartphones. In Study 3, 141 participants self-administered Word Meaning remotely twice with a 2-week delay on personal iPhones. RESULTS: The final bank included 1363 items. Internal consistency was adequate to good across samples (ρxx = 0.78 to 0.81, p < .001). Test-retest reliability was good (ICC = 0.65, p < .001), and the mean theta score was not significantly different upon the second administration. Correlations were moderate to large with measures of similar constructs (ρ = 0.67-0.75, p < .001) and non-significant with measures of dissimilar constructs. Scores demonstrated small to moderate correlations with age (ρ = 0.35 to 0.45, p < .001) and education (ρ = 0.26, p < .001). CONCLUSION: The MTB Word Meaning measure demonstrated evidence of reliability and validity in three samples. Further validation studies in clinical samples are necessary.

Evaluating a web-based personalized decision report for total knee or hip replacement: Lessons learned from patients.

RATIONALE: Patient-reported outcomes (PROs) are increasingly used in the context of clinical care, but evaluation of patients' perspectives of PRO-based applications in routine care remains limited. AIMS AND OBJECTIVES: This paper investigates patients' acceptability of a personalized web-based decision report for total knee or hip replacement and identifies opportunities to refine the report. METHOD: This qualitative evaluation was embedded in a pragmatic cluster randomized trial of the report. We interviewed 25 patients with knee and hip osteoarthritis about their experiences using the personalized decision report in the context of a surgical consultation. The web-based report contained current descriptive PRO scores of pain, function and general physical health; tailored predicted postoperative PRO scores (i.e., personalized likely outcomes based on actual knee or hip replacement outcomes of similar patients in a national registry); and information about alternative nonoperative treatments. Two trained researchers analysed the interview data qualitatively using a combination of inductive and deductive coding. RESULTS: We identified three major categories for evaluation: content of report, presentation of data in report and engagement with report. Patients generally liked the report overall but specifically valued different pages of the report based on where they were in the surgical decision-making process. Patients identified areas of confusion in data presentation related to graph orientation, terminology and interpretation of T-scores. Patients also highlighted support needs to meaningfully engage with the information in the report. CONCLUSION: Our findings highlight areas of opportunity to further refine this personalized web-based decision report and similar patient-facing PRO applications for routine clinical care. Specific examples include additional tailoring of reports via filterable web-based dashboards and scalable educational supports to facilitate more independent patient understanding and use.

Patients' perspectives on the benefits of feedback on patient-reported outcome measures in a web-based personalized decision report for hip and knee osteoarthritis.

BACKGROUND: Applications of patient-reported outcome measures (PROMs) for individual patient management are expanding with the support of digital tools. Providing PROM-based information to patients can potentially improve care experiences and outcomes through informing and activating patients. This study explored patients' perspectives on the benefits of receiving feedback on PROMs in the context of a web-based personalized decision report to guide care for their hip or knee osteoarthritis. METHODS: This qualitative descriptive interview study was nested in a pragmatic clinical trial of a personalized report, which includes descriptive PROM scores and predicted postoperative PROM scores. Patients completed a semi-structured interview within 6 weeks of an office visit with an orthopaedic surgeon. Only patients who reported receiving the report and reviewing it with the surgeon and/or a health educator were included. Data were iteratively analyzed using a combination of deductive and inductive coding strategies. RESULTS: Twenty-five patients aged 49-82 years (60% female, 72% surgical treatment decision) participated and described three primary benefits of the PROM feedback within the report: 1. Gaining Information About My Health Status, including data teaching new information, confirming what was known, or providing a frame of reference; 2. Fostering Communication Between Patient and Surgeon, encompassing use of the data to set expectations, ask and answer questions, and facilitate shared understanding; and 3. Increasing My Confidence and Trust, relating to the treatment outcomes, treatment decision, and surgeon. CONCLUSIONS: Patients identified actual and hypothetical benefits of receiving feedback on PROM scores in the context of a web-based decision report, including advantages for those who had already made a treatment decision before seeing the surgeon. Findings provide insight into patients' perspectives on how digital PROM data can promote patient-centered care. Results should be considered in the context of the homogeneous sample and complex trial. While participants perceived value in this personalized report, questions remain regarding best practices in patient-facing data presentation and engagement. TRIAL REGISTRATION: ClinicalTrials.gov, NCT03102580. Registered on 5 April 2017.

Challenges in Participant Engagement and Retention Using Mobile Health Apps: Literature Review.

BACKGROUND: Mobile health (mHealth) apps are revolutionizing the way clinicians and researchers monitor and manage the health of their participants. However, many studies using mHealth apps are hampered by substantial participant dropout or attrition, which may impact the representativeness of the sample and the effectiveness of the study. Therefore, it is imperative for researchers to understand what makes participants stay with mHealth apps or studies using mHealth apps. OBJECTIVE: This study aimed to review the current peer-reviewed research literature to identify the notable factors and strategies used in adult participant engagement and retention. METHODS: We conducted a systematic search of PubMed, MEDLINE, and PsycINFO databases for mHealth studies that evaluated and assessed issues or strategies to improve the engagement and retention of adults from 2015 to 2020. We followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Notable themes were identified and narratively compared among different studies. A binomial regression model was generated to examine the factors affecting retention. RESULTS: Of the 389 identified studies, 62 (15.9%) were included in this review. Overall, most studies were partially successful in maintaining participant engagement. Factors related to particular elements of the app (eg, feedback, appropriate reminders, and in-app support from peers or coaches) and research strategies (eg, compensation and niche samples) that promote retention were identified. Factors that obstructed retention were also identified (eg, lack of support features, technical difficulties, and usefulness of the app). The regression model results showed that a participant is more likely to drop out than to be retained. CONCLUSIONS: Retaining participants is an omnipresent challenge in mHealth studies. The insights from this review can help inform future studies about the factors and strategies to improve participant retention.