Establishing a Working Definition of User Experience for eHealth Interventions of Self-reported User Experience Measures With eHealth Researchers and Adolescents: Scoping Review.

BACKGROUND: Across eHealth intervention studies involving children, adolescents, and their parents, researchers have measured user experience to assist with intervention development, refinement, and evaluation. To date, no widely accepted definitions or measures of user experience exist to support a standardized approach for evaluation and comparison within or across interventions. OBJECTIVE: We conduct a scoping review with subsequent Delphi consultation to identify how user experience is defined and measured in eHealth research studies, characterize the measurement tools used, and establish working definitions for domains of user experience that could be used in future eHealth evaluations. METHODS: We systematically searched electronic databases for published and gray literature available from January 1, 2005, to April 11, 2019. We included studies assessing an eHealth intervention that targeted any health condition and was designed for use by children, adolescents, and their parents. eHealth interventions needed to be web-, computer-, or mobile-based, mediated by the internet with some degree of interactivity. We required studies to report the measurement of user experience as first-person experiences, involving cognitive and behavioral factors reported by intervention users. We appraised the quality of user experience measures in included studies using published criteria: well-established, approaching well-established, promising, or not yet established. We conducted a descriptive analysis of how user experience was defined and measured in each study. Review findings subsequently informed the survey questions used in the Delphi consultations with eHealth researchers and adolescent users for how user experience should be defined and measured. RESULTS: Of the 8634 articles screened for eligibility, 129 articles and 1 erratum were included in the review. A total of 30 eHealth researchers and 27 adolescents participated in the Delphi consultations. On the basis of the literature and consultations, we proposed working definitions for 6 main user experience domains: acceptability, satisfaction, credibility, usability, user-reported adherence, and perceived impact. Although most studies incorporated a study-specific measure, we identified 10 well-established measures to quantify 5 of the 6 domains of user experience (all except for self-reported adherence). Our adolescent and researcher participants ranked perceived impact as one of the most important domains of user experience and usability as one of the least important domains. Rankings between adolescents and researchers diverged for other domains. CONCLUSIONS: Findings highlight the various ways in which user experience has been defined and measured across studies and what aspects are most valued by researchers and adolescent users. We propose incorporating the working definitions and available measures of user experience to support consistent evaluation and reporting of outcomes across studies. Future studies can refine the definitions and measurement of user experience, explore how user experience relates to other eHealth outcomes, and inform the design and use of human-centered eHealth interventions.

Design and Delivery Features That May Improve the Use of Internet-Based Cognitive Behavioral Therapy for Children and Adolescents With Anxiety: A Realist Literature Synthesis With a Persuasive Systems Design Perspective.

BACKGROUND: Internet-based cognitive behavioral therapy (iCBT) is a persuasive system as its design combines therapeutic content, technological features, and interactions between the user and the program to reduce anxiety for children and adolescents. How iCBT is designed and delivered differs across programs. Although iCBT is considered an effective approach for treating child and adolescent anxiety, rates of program use (eg, module completion) are highly variable for reasons that are not clear. As the extent to which users complete a program can impact anxiety outcomes, understanding what iCBT design and delivery features improve program use is critical for optimizing treatment effects. OBJECTIVE: The objectives of this study were to use a realist synthesis approach to explore the design and delivery features of iCBT for children and adolescents with anxiety as described in the literature and to examine their relationship to program use outcomes. METHODS: A search of published and gray literature was conducted up to November 2017. Prespecified inclusion criteria identified research studies, study protocols, and program websites on iCBT for child and adolescent anxiety. Literature was critically appraised for relevance and methodological rigor. The persuasive systems design (PSD) model, a comprehensive framework for designing and evaluating persuasive systems, was used to guide data extraction. iCBT program features were grouped under 4 PSD categories-Primary task support, Dialogue support, System credibility support, and Social support. iCBT design (PSD Mechanisms) and delivery features (Context of use) were linked to program use (Outcomes) using meta-ethnographic methods; these relationships were described as Context-Mechanism-Outcome configurations. For our configurations, we identified key PSD features and delivery contexts that generated moderate-to-high program use based on moderate-to-high quality evidence found across multiple iCBT programs. RESULTS: A total of 44 documents detailing 10 iCBT programs were included. Seven iCBT programs had at least one document that scored high for relevance; most studies were of moderate-to-high methodological rigor. We developed 5 configurations that highlighted 8 PSD features (Tailoring, Personalization [Primary task supports]; Rewards, Reminders, Social role [Dialogue supports]; and Trustworthiness, Expertise, Authority [System credibility supports]) associated with moderate-to-high program use. Important features of delivery Context were adjunct support (a face-to-face, Web- or email-based communications component) and whether programs targeted the prevention or treatment of anxiety. Incorporating multiple PSD features may have additive or synergistic effects on program use. CONCLUSIONS: The Context-Mechanism-Outcome configurations we developed suggest that, when delivered with adjunct support, certain PSD features contribute to moderate-to-high use of iCBT prevention and treatment programs for children and adolescents with anxiety. Standardization of the definition and measurement of program use, formal testing of individual and combined PSD features, and use of real-world design and testing methods are important next steps to improving how we develop and deliver increasingly useful treatments to target users.

The Gobbledygook in Online Parent-Focused Information about Child and Adolescent Mental Health.

Online parent-focused informational resources play a vital step in parent decision-making about initiating child and adolescent mental health care, but their usefulness may depend on how easily the resource content can be understood. The purpose of this cross-sectional study was to examine the readability and reliability of parent-focused mental health resources provided on Canadian websites. After meeting inclusion criteria, 50 documents retrieved during the search in September 2016 from websites using online health information searching strategies that would be typically employed by parents underwent analysis. Document readability was assessed using Health Canada recommended instruments: Simple Measure of Gobbledygook (SMOG), Flesch-Kincaid Grade Level, and the Flesch Reading-Ease scale. Reliability was assessed using Health on the Net Foundation Code of Conduct (HONcode). Our analysis revealed that all included documents exceeded the 8th grade reading level. The mean ± SD readability scores were SMOG 11.65 ± 1.10 and FKGL 10.03 ± 1.49. Reading-ease scores showed that 42% (n = 21) of the documents were "Difficult" to read. Factorial ANOVA revealed no significant difference in readability across mental health topic areas or organizational sectors (p = > 0.05). Twenty-four percent of documents came from sites with HONcode certification. Findings suggest that almost all child and adolescent mental health information that would typically be found online by parents in Canada had readability scores that were too high for average parents to read and exceeded Health Canada recommended reading levels. Being able to locate resources online can be significantly precluded if a parent cannot understand and use information to mobilize them to accessing mental health care for their families.

How do eHealth Programs for Adolescents With Depression Work? A Realist Review of Persuasive System Design Components in Internet-Based Psychological Therapies.

BACKGROUND: Major depressive disorders are common among adolescents and can impact all aspects of their daily life. Traditional therapies, cognitive behavioral therapy (CBT), and interpersonal psychotherapy (IPT) have been delivered face-to-face. However, Internet-based (online) delivery of these therapies is emerging as an option for adolescents. Internet-based CBT and IPT involve therapeutic content, interaction between the user and the system, and different technological features embedded into the online program (eg, multimedia). Studies of Internet-based CBT and IPT for adolescent depression differ on all three aspects, and variable, positive therapy effects have been reported. A better understanding of the treatment conditions that influence therapy outcomes is important to designing and evaluating these novel therapies. OBJECTIVE: Our aim was to examine the technological and program delivery features of Internet-based CBT and IPT for adolescent depression and to document their potential relation to treatment outcomes and program use. METHODS: We performed a realist synthesis. We started with an extensive search of published and gray literature. We included intervention studies that evaluated Internet-based CBT or IPT for adolescent depression. We included mixed-methods and qualitative studies, theoretical papers, and policy/implementation documents if they included a focus on how Internet-based psychological therapy is proposed to work for adolescents with depression/depressive symptoms. We used the Mixed-Methods Appraisal Tool to assess the methodological quality of studies. We used the Persuasive System Design (PSD) model as a framework for data extraction and analysis to examine how Internet-based CBT and IPT, as technology-based systems, influence the attitudes and behaviors of system users. PSD components described for the therapies were linked to reported outcomes using a cross-case comparison method and thematic synthesis. RESULTS: We identified 19 Internet-based CBT programs in 59 documents. Of those, 71% (42/59) were of moderate to high quality. The PSD features surface credibility (competent "look and feel"), dialogue support (online program + in-person support), liking and similarity (esthetics and content appeal to adolescent users), the reduction and tunneling of therapeutic content (reducing online content into simple tasks, guiding users), and use of self-monitoring were present in therapies that resulted in improved therapy engagement, satisfaction, and adherence, as well as symptom and functional impairments. CONCLUSIONS: When incorporated into Internet-based CBT for adolescent depression, PSD features may improve adolescent adherence, satisfaction, and depression-related outcomes. Testing of these features using hypothesis-driven dismantling approaches is recommended to advance our understanding of how these features contribute to therapy effectiveness.

Emergency Department Use and Postvisit Care for Anxiety and Stress Disorders Among Children: A Population-Based Cohort Study in Alberta, Canada.

OBJECTIVE: We examined sociodemographic trends in emergency department ED visits and postvisit health care for anxiety and stress disorders. METHODS: Emergency department visits (N = 11,289) by children younger than 18 years were extracted from population-based administrative databases from Alberta, Canada (2002-2011). We examined health services use by demographics and socioeconomic proxy (First Nations status, family subsidy type: government, human services program, none). We calculated visit rates and described physician visits after discharge (n = 8075 children). Multivariable survival analyses (with 95% confidence intervals [CIs]) estimate time to first physician visit and ED return. RESULTS: During 2002-2011, visit rates increased by age, First Nations, and subsidy status. The largest increase was for children (all ages) from families receiving government subsidy (491.43/100,000, P < 0.001). Thirty days after an index visit, most physician follow-up visits were made by children aged 15 to 17 years (61.0%) and from families receiving no subsidy (66.5%). The median time to physician follow-up for First Nations children was 32 days (95% CI, 27-37) compared with 19 days for children from families receiving government subsidy whose median time was shortest (95% CI, 18-23). Children (all ages) in the government and human services program subsidy groups and those who had First Nations status returned earlier to the ED compared with children in the no subsidy group. CONCLUSIONS: Adolescents had high ED use and physician follow-up, whereas First Nations children and those from families receiving subsidy (all ages) had high ED use and low physician follow-up. Efforts to improve disorder identification and treatment among high ED user groups and low physician follow-up user groups are needed.

Decision-making under explicit risk is impaired in multiple sclerosis: relationships with ventricular width and disease disability.

BACKGROUND: Decision-making is an essential function of everyday life. Decision-making under explicit risk requires developing advantageous decision strategies based on fixed outcomes (e.g., probabilities of winning or losing a bet). Decision-making and its neural substrates have been rarely studied in MS. We expected performance in decision-making under risk to be lowered in MS patients, and negatively correlated with disease-related disability, cognition, and ventricular width. METHODS: Three groups were included: 32 MS patients and 20 healthy controls were examined with conventional neuropsychological tests and the Game-of-Dice Task (GDT) assessing decision-making under explicit risk. Linear 2-D ventricular width was assessed on MS patients' clinical MRIs and compared to a third group, 20 non-MS neurological control patients. RESULTS: Compared to healthy controls, MS patients showed impaired GDT and neuropsychological performance, depending on the MS-subtype (relapsing-remitting (RR), n = 22; secondary progressive, n = 10) and disability severity among RR-MS patients. In MS patients, GDT performance correlated with processing speed, intercaudate ratio, and third ventricle ratio (p's < 0.05). Mediation analysis showed that the link between GDT performance and processing speed was fully explained by ventricular size. CONCLUSION: Decision-making under explicit risk was reduced in MS patients, but only those with more pronounced disability. Independent of processing speed, decision-making under explicit risk correlates inversely with central atrophy in MS.

Predicting time to emergency department return for anxiety disorders and acute stress reactions in children and adolescents: a cohort study.

PURPOSE: While studies indicate that few children with anxiety disorders receive specialized mental health care, significant knowledge gaps exist for the mental health care trajectory outside of specialized care. We examined whether physician-based care after an emergency department (ED) visit for mental health care predicted time to ED return. METHODS: We conducted a retrospective cohort study using administrative data from Alberta, Canada. Included in the cohort were 8075 children <18 years discharged from an ED (104 EDs in total), April 2002-September 2010, following a visit for an anxiety disorder or acute stress reaction. We used a multivariable Cox proportional hazards model to estimate time to ED return for mental health care (censored at 90 days). The variables of interest were: number and type of physician visits 30 days after index visit. We adjusted for sociodemographics, clinical acuity, comorbidity, and recent history of ED mental health care in the model. RESULTS: Within 30 days of ED discharge, multiple physician follow-up visits were associated with shorter return time (adjusted HR 1.24, 95 % CI 1.08-1.43). A single physician follow-up visit was associated with longer return time (adjusted HR 0.68, 95 % CI 0.58-0.79). Physician follow-up visits for mental health care were associated with shorter return time (adjusted HR 2.5, 95 % CI 2.21-2.83). CONCLUSIONS: Following an index visit, ED return by children with anxiety disorders or acute stress reactions is associated with specific characteristics of subsequent physician visits. Improving physician use of evidence-based treatment and family access to coordinated services may reduce costly hospital-based care.

A randomized controlled trial evaluating the effect of an internet-based cognitive-behavioral program on anxiety symptoms in a community-based sample of adolescents.

Adolescents' use of online resources to self-manage anxiety is growing. The objective of the current trial was to assess the effectiveness of an online, primarily self-led cognitive behavioral therapy (CBT) program in reducing anxiety symptoms compared to an active comparator, access to anxiety resources on a static website. A total of 563 adolescents (13-19 years) with self-identified anxiety concerns were enrolled. Self-reported anxiety symptoms were assessed pre- and post-intervention (6 weeks). Adolescents were further assessed 3 months post-intervention. Other outcomes assessed at the three time-points were quality of life (QOL) and healthcare utilization. Both interventions reduced anxiety symptoms after use. Group differences in symptom change were not significant post-intervention (p = 0.16), but were at 3 months (favouring online CBT; p = 0.04) with male participants reporting more symptom change (p = 0.03). Across time-points, as anxiety symptoms decreased, QOL increased (p < 0.001). Among participants that provided healthcare utilization before and after intervention use, the greatest changes in use were among online CBT users particularly for mental health provider visits (psychiatrist, -41.0 % vs. +18.5 %; social worker, -42.5 % vs. -22.1 %), hospital-based care (emergency department visits, -80.0 % vs. +79.4 %; hospital admissions, -76.1 % vs. +42.9 %), and use of self-help or alternative treatments (-60.0 % vs. +6.6 %). Results suggest that, over time, use of online CBT by adolescents can result in improved anxiety symptoms and fewer use of other healthcare resources compared to traditional online information seeking.

Examining the Usage, User Experience, and Perceived Impact of an Internet-Based Cognitive Behavioral Therapy Program for Adolescents With Anxiety: Randomized Controlled Trial.

BACKGROUND: Internet-based cognitive behavioral therapy (iCBT) increases treatment access for adolescents with anxiety; however, completion rates of iCBT programs are typically low. Understanding adolescents' experiences with iCBT, what program features and changes in anxiety (minimal clinically important difference [MCID]) are important to them, may help explain and improve iCBT program use and impact. OBJECTIVE: Within a randomized controlled trial comparing a six-session iCBT program for adolescent anxiety, Being Real, Easing Anxiety: Tools Helping Electronically (Breathe), with anxiety-based resource webpages, we aimed to (1) describe intervention use among adolescents allocated to Breathe or webpages and those who completed postintervention assessments (Breathe or webpage respondents); (2) describe and compare user experiences between groups; and (3) calculate an MCID for anxiety and explore relationships between iCBT use, experiences, and treatment response among Breathe respondents. METHODS: Enrolled adolescents with self-reported anxiety, aged 13 to 19 years, were randomly allocated to Breathe or webpages. Self-reported demographics and anxiety symptoms (Multidimensional Anxiety Scale for Children-2nd edition [MASC-2]) were collected preintervention. Automatically-captured Breathe or webpage use and self-reported symptoms and experiences (User Experience Questionnaire for Internet-based Interventions) were collected postintervention. Breathe respondents also reported their perceived change in anxiety (Global Rating of Change Scale [GRCS]) following program use. Descriptive statistics summarized usage and experience outcomes, and independent samples t tests and correlations examined relationships between them. The MCID was calculated using the mean MASC-2 change score among Breathe respondents reporting somewhat better anxiety on the GRCS. RESULTS: Adolescents were mostly female (382/536, 71.3%), aged 16.6 years (SD 1.7), with very elevated anxiety (mean 92.2, SD 18.1). Intervention use was low for adolescents allocated to Breathe (mean 2.2 sessions, SD 2.3; n=258) or webpages (mean 2.1 visits, SD 2.7; n=278), but was higher for Breathe (median 6.0, range 1-6; 81/258) and webpage respondents (median 2.0, range 1-9; 148/278). Total user experience was significantly more positive for Breathe than webpage respondents (P<.001). Breathe respondents reported program design and delivery factors that may have challenged (eg, time constraints and program support) or facilitated (eg, demonstration videos, self-management activities) program use. The MCID was a mean MASC-2 change score of 13.8 (SD 18.1). Using the MCID, a positive treatment response was generated for 43% (35/81) of Breathe respondents. Treatment response was not correlated with respondents' experiences or use of Breathe (P=.32 to P=.88). CONCLUSIONS: Respondents reported positive experiences and changes in their anxiety with Breathe; however, their reports were not correlated with program use. Breathe respondents identified program design and delivery factors that help explain their experiences and use of iCBT and inform program improvements. Future studies can apply our measures to compare user experiences between internet-based interventions, interpret treatment outcomes and improve treatment decision making for adolescents with anxiety. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT02970734 https://clinicaltrials.gov/ct2/show/NCT02970734.

Potential Reduction of Symptoms With the Use of Persuasive Systems Design Features in Internet-Based Cognitive Behavioral Therapy Programs for Children and Adolescents With Anxiety: A Realist Synthesis.

BACKGROUND: Internet-based cognitive behavioral therapy (iCBT) for children and adolescents is a persuasive system that combines 3 major components to therapy-therapeutic content, technological features, and interactions between the user and program-intended to reduce users' anxiety symptoms. Several reviews report the effectiveness of iCBT; however, iCBT design and delivery components differ widely across programs, which raise important questions about how iCBT effects are produced and can be optimized. OBJECTIVE: The objective of this study was to review and synthesize the iCBT literature using a realist approach with a persuasive systems perspective to (1) document the design and delivery components of iCBT and (2) generate hypotheses as to how these components may explain changes in anxiety symptoms after completing iCBT. METHODS: A multi-strategy search identified published and gray literature on iCBT for child and adolescent anxiety up until June 2019. Documents that met our prespecified inclusion criteria were appraised for relevance and methodological rigor. Data extraction was guided by the persuasive systems design (PSD) model. The model describes 28 technological design features, organized into 4 categories that help users meet their health goals: primary task support, dialogue support, system credibility support, and social support. We generated initial hypotheses for how PSD (mechanisms) and program delivery (context of use) features were linked to symptom changes (outcomes) across iCBT programs using realist and meta-ethnographic techniques. These hypothesized context-mechanism-outcome configurations were refined during analysis using evidence from the literature to improve their explanatory value. RESULTS: A total of 63 documents detailing 15 iCBT programs were included. A total of six iCBT programs were rated high for relevance, and most studies were of moderate-to-high methodological rigor. A total of 11 context-mechanism-outcome configurations (final hypotheses) were generated. Configurations primarily comprised PSD features from the primary task and dialogue support categories. Several key PSD features (eg, self-monitoring, simulation, social role, similarity, social learning, and rehearsal) were consistently reported in programs shown to reduce anxiety; many features were employed simultaneously, suggesting synergy when grouped. We also hypothesized the function of PSD features in generating iCBT impacts. Adjunct support was identified as an important aspect of context that may have complemented certain PSD features in reducing users' anxiety. CONCLUSIONS: This synthesis generated context-mechanism-outcome configurations (hypotheses) about the potential function, combination, and impact of iCBT program components thought to support desired program effects. We suggest that, when delivered with adjunct support, PSD features may contribute to reduced anxiety for child and adolescent users. Formal testing of the 11 configurations is required to confirm their impact on anxiety-based outcomes. From this we encourage a systematic and deliberate approach to iCBT design and evaluation to increase the pool of evidence-based interventions available to prevent and treat children and adolescents with anxiety.