Psychometric properties of the Telepsych User Experience Scale (TUES).

PURPOSE: To develop and psychometrically evaluate a brief measure of the telemental health experience among people receiving psychiatric and psychological care-the Telepsych User Experience Scale (TUES). METHODS: The TUES was administered at 6 months to 364 study participants who screened positive for posttraumatic stress disorder and/or bipolar disorder and used telepsych services. The factor structure of the TUES was examined using exploratory and confirmatory factor analyses. Psychometric testing of the final scale examined (1) internal reliability, (2) criterion validity, (3) convergent validity, and (4) test-retest reliability using omega coefficients, negative binomial regression, and correlations, respectively. A week after the 12-month follow-up survey was completed, a retest was administered to 29 patients to assess reliability. FINDINGS: Factor analytic methods indicated a single latent factor (user experience) and correlated error variance (method effect of item wording) for 2 items. To enhance clinical utility, we removed the 2 negatively worded items, resulting in a 5-item scale. Confirmatory analyses indicated excellent fit of the final model, which retained the best performing items from each hypothesized construct. The TUES demonstrated evidence of internal consistency (omega = 0.88-0.90), convergent validity, (r = 0.58), and criterion validity through telepsych engagement (incidence rate ratio = 1.19, P < .001), though test-retest reliability was unacceptable (r = 0.41). CONCLUSION: The TUES is a pragmatic instrument with evidence of validity and internal reliability. Replication is necessary, but this initial psychometric evaluation suggests the TUES is a promising, brief yet comprehensive measure of telemental health user experience with clinical populations in rural settings.

Psychometric Properties of the Assessment of Perceived Access to Care (APAC) Instrument.

Valid measures of perceived access are needed to measure whether health care systems are providing adequate access. This research reports on the psychometric properties of the Assessment of Perceived Access to Care (APAC), which was administered to 1004 Community Health Center patients screening positive for psychiatric disorders. Known-group validity was good, with 6 of the 8 hypothesized associations between social determinants of access and perceived access being significant (P < .01). Better access was significantly (P < .01) correlated with more outpatient mental health visits, indicating good convergent validity. The test-retest Pearson correlation coefficient (0.64) was statistically significant (P < .01). The APAC has acceptable psychometric properties.

Care manager perspectives on integrating an mHealth app system into clinical workflows: A mixed methods study.

OBJECTIVE: mHealth can be a valuable means of monitoring symptoms and supporting care for rural patients, but barriers to implementation remain. This study aimed to examine care manager perspectives on the adoption, use and impact of an mHealth system deployed within a pragmatic Collaborative Care trial for rural patients with PTSD and/or Bipolar Disorder. METHOD: Sixteen care managers at 12 Federally Qualified Health Centers in 3 states participated in semi-structured interviews. Interviews were transcribed, coded, and thematically analyzed using the Unified Theory of Adoption and Use of Technology as a conceptual framework. App metadata was used to assess the frequency of a care manager reported phenomenon, clinically disengaged app use. RESULTS: 4 themes were identified: infrastructural limitations; redundant and incompatible clinical and mHealth workflows; cross platform and web access; and patient engagement and clinically disengaged app use. Most users had a period of consistently submitting symptom measures via the app while disengaged from care for >4 weeks.

Applying the Principles for Digital Development: Case Study of a Smartphone App to Support Collaborative Care for Rural Patients With Posttraumatic Stress Disorder or Bipolar Disorder.

BACKGROUND: Despite a proliferation of patient-facing mobile apps for mental disorders, there is little literature guiding efforts to incorporate mobile tools into clinical care delivery and integrate patient-generated data into care processes for patients with complex psychiatric disorders. OBJECTIVE: The aim of this study was to seek to gain an understanding of how to incorporate a patient-provider mobile health (mHealth) platform to support the delivery of integrated primary care-based mental health services (Collaborative Care) to rural patients with posttraumatic stress disorder and/or bipolar disorder. METHODS: Using the Principles for Digital Development as a framework, we describe our experience designing, developing, and deploying a mobile system to support Collaborative Care. The system consists of a patient-facing smartphone app that integrates with a Web-based clinical patient registry used by behavioral health care managers and consulting psychiatrists. Throughout development, we engaged representatives from the system's two user types: (1) providers, who use the Web-based registry and (2) patients, who directly use the mobile app. We extracted mobile metadata to describe the early adoption and use of the system by care managers and patients and report preliminary results from an in-app patient feedback survey that includes a System Usability Scale (SUS). RESULTS: Each of the nine Principles for Digital Development is illustrated with examples. The first 10 patients to use the smartphone app have completed symptom measures on average every 14 days over an average period of 20 weeks. The mean SUS score at week 8 among four patients who completed this measure was 91.9 (range 72.5-100). We present lessons learned about the technical and training requirements for integration into practice that can inform future efforts to incorporate health technologies to improve care for patients with psychiatric conditions. CONCLUSIONS: Adhering to the Principles for Digital Development, we created and deployed an mHealth system to support Collaborative Care for patients with complex psychiatric conditions in rural health centers. Preliminary data among the initial users support high system usability and show promise for sustained use. On the basis of our experience, we propose five additional principles to extend this framework and inform future efforts to incorporate health technologies to improve care for patients with psychiatric conditions: design for public health impact, add value for all users, test the product and the process, acknowledge disruption, and anticipate variability.