Time to act mature-Gearing eHealth evaluations towards technology readiness levels.

It is challenging to design a proper eHealth evaluation. In our opinion, the evaluation of eHealth should be a continuous process, wherein increasingly mature versions of the technology are put to the test. In this article, we present a model for continuous eHealth evaluation, geared towards technology maturity. Technology maturity can be determined best via Technology Readiness Levels, of which there are nine, divided into three phases: the research, development, and deployment phases. For each phase, we list and discuss applicable activities and outcomes on the end-user, clinical, and societal front. Instead of focusing on a single perspective, we recommend to blend the end-user, health and societal perspective. With this article we aim to contribute to the methodological debate on how to create the optimal eHealth evaluation design.

Patient Values Associated with an Exergame Supporting COPD Treatment.

Exercise games (exergames) can help COPD patients stay active and prevent exacerbation. When evaluating such exergames, patient values are an important variable to take into account. In this study, seven COPD patients used an exergame technology at the physiotherapist for six months. Their values regarding treatment and the exergames were identified in interviews. Values were very stable throughout the study, and closely interconnected. Personal Guidance and Independence were important values. Additionally, patients sometimes held conflicting values that they prioritized differently at different times or based on specific events. As the study identified important values that appeared stable over a period of time, albeit with different prioritization, they are worth considering when designing new technology. However, values cannot be looked at in isolation because of the strong connection between values.

Improving usability benchmarking for the eHealth domain: The development of the eHealth UsaBility Benchmarking instrument (HUBBI).

BACKGROUND: Currently, most usability benchmarking tools used within the eHealth domain are based on re-classifications of old usability frameworks or generic usability surveys. This makes them outdated and not well suited for the eHealth domain. Recently, a new ontology of usability factors was developed for the eHealth domain. It consists of eight categories: Basic System Performance (BSP), Task-Technology Fit (TTF), Accessibility (ACC), Interface Design (ID), Navigation & Structure (NS), Information & Terminology (IT), Guidance & Support (GS) and Satisfaction (SAT). OBJECTIVE: The goal of this study is to develop a new usability benchmarking tool for eHealth, the eHealth UsaBility Benchmarking Instrument (HUBBI), that is based on a new ontology of usability factors for eHealth. METHODS: First, a large item pool was generated containing 66 items. Then, an online usability test was conducted, using the case study of a Dutch website for general health advice. Participants had to perform three tasks on the website, after which they completed the HUBBI. Using Partial Least Squares Structural Equation Modelling (PLS-SEM), we identified the items that assess each factor best and that, together, make up the HUBBI. RESULTS: A total of 148 persons participated. Our selection of items resulted in a shortened version of the HUBBI, containing 18 items. The category Accessibility is not included in the final version, due to the wide range of eHealth services and their heterogeneous populations. This creates a constantly different role of Accessibility, which is a problem for a uniform benchmarking tool. CONCLUSIONS: The HUBBI is a new and comprehensive usability benchmarking tool for the eHealth domain. It assesses usability on seven domains (BSP, TTF, ID, NS, IT, GS, SAT) in which a score per domain is generated. This can help eHealth developers to quickly determine which areas of the eHealth system's usability need to be optimized.

Contextual Health Information Behavior in the Daily Lives of People with Type 2 Diabetes: A Diary Study in Scotland.

Changes in lifestyle can have positive effects on treating type 2 diabetes (T2D), like sporting or healthy eating. Therefore, a person diagnosed with T2D is often advised to make healthy choices throughout the day, in addition to other interventions such as medication. To do this, he or she needs health information to support decision-making. Literature describes ample categorizations of types of (health) information behavior and theoretical models that explain the factors that drive people to search for, encounter or avoid information. However, there are few longitudinal studies about triggers and factors in daily life that affect health information behavior (HIB). This study was set up to identify triggers, actions and outcomes for active, passive and avoidant HIB situations in daily life among Scots with Type 2 diabetes (T2D) to identify points of attention for communication strategies. Twelve participants took part in a four-week diary study. Every day, participants received an online diary form to describe active, passive or avoidant HIB situations. Data collection resulted in 53 active, 120 passive and 25 avoidant diary entries. Seven active HIB contexts (e.g., experiencing symptoms, cooking dinner, sports training) and five passive HIB contexts (e.g., home, work, medical facility) were identified. Four motivations for avoidance were found (e.g., time constraints, no health trigger). These results can be used to supplement the theoretical models of health information behavior. Furthermore, health professionals can use these results to support their clients with T2D in the self-management of their health, by guiding them to trustworthy sources of health information and lowering barriers for searching health information.

Conceptualizing Usability for the eHealth Context: Content Analysis of Usability Problems of eHealth Applications.

BACKGROUND: Usability tests can be either formative (where the aim is to detect usability problems) or summative (where the aim is to benchmark usability). There are ample formative methods that consider user characteristics and contexts (ie, cognitive walkthroughs, interviews, and verbal protocols). This is especially valuable for eHealth applications, as health conditions can influence user-system interactions. However, most summative usability tests do not consider eHealth-specific factors that could potentially affect the usability of a system. One of the reasons for this is the lack of fine-grained frameworks or models of usability factors that are unique to the eHealth domain. OBJECTIVE: In this study, we aim to develop an ontology of usability problems, specifically for eHealth applications, with patients as primary end users. METHODS: We analyzed 8 data sets containing the results of 8 formative usability tests for eHealth applications. These data sets contained 400 usability problems that could be used for analysis. Both inductive and deductive coding were used to create an ontology from 6 data sets, and 2 data sets were used to validate the framework by assessing the intercoder agreement. RESULTS: We identified 8 main categories of usability factors, including basic system performance, task-technology fit, accessibility, interface design, navigation and structure, information and terminology, guidance and support, and satisfaction. These 8 categories contained a total of 21 factors: 14 general usability factors and 7 eHealth-specific factors. Cohen κ was calculated for 2 data sets on both the category and factor levels, and all Cohen κ values were between 0.62 and 0.67, which is acceptable. Descriptive analysis revealed that approximately 69.5% (278/400) of the usability problems can be considered as general usability factors and 30.5% (122/400) as eHealth-specific usability factors. CONCLUSIONS: Our ontology provides a detailed overview of the usability factors for eHealth applications. Current usability benchmarking instruments include only a subset of the factors that emerged from our study and are therefore not fully suited for summative evaluations of eHealth applications. Our findings support the development of new usability benchmarking tools for the eHealth domain.

Designing a stakeholder-inclusive service model for an eHealth service to support older adults in an active and social life.

BACKGROUND: Service model design is slowly being recognized among eHealth developers as a valuable method for creating durable implementation strategies. Nonetheless, practical guidelines and case-studies that inform the community on how to design a service model for an eHealth innovation are lacking. This study describes the development of a service model for an eHealth service, titled 'SALSA', which intends to support older adults with a physically active and socially inclusive lifestyle. METHODS: The service model for the SALSA service was developed in eight consecutive rounds, using a mixed-methods approach. First, a stakeholder salience analysis was conducted to identify the most relevant stakeholders. In rounds 2-4, in-depth insights about implementation barriers, facilitators and workflow processes of these stakeholders were gathered. Rounds 5 and 6 were set up to optimize the service model and receive feedback from stakeholders. In rounds 7 and 8, we focused on future implementation and integrating the service model with the technical components of the eHealth service. RESULTS: While the initial goal was to create one digital platform for the eHealth service, the results of the service modelling showed how the needs of two important stakeholders, physiotherapists and sports trainers, were too different for integrating them in one platform. Therefore, the decision was made to create two platforms, one for preventive (senior sports activities) and one for curative (physical rehabilitation) purposes. CONCLUSIONS: A service model shows the interplay between service model design, technical development and business modelling. The process of service modelling helps to align the interests of the different stakeholders to create support for future implementation of an eHealth service. This study provides clear documentation on how to conduct service model design processes which can enable future learning and kickstart new research. Our results show the potential that service model design has for service development and innovation in health care.

Embodied Conversational Agent Appearance for Health Assessment of Older Adults: Explorative Study.

BACKGROUND: Embodied conversational agents (ECAs) have great potential for health apps but are rarely investigated as part of such apps. To promote the uptake of health apps, we need to understand how the design of ECAs can influence the preferences, motivation, and behavior of users. OBJECTIVE: This is one of the first studies that investigates how the appearance of an ECA implemented within a health app affects users' likeliness of following agent advice, their perception of agent characteristics, and their feeling of rapport. In addition, we assessed usability and intention to use. METHODS: The ECA was implemented within a frailty assessment app in which three health questionnaires were translated into agent dialogues. In a within-subject experiment, questionnaire dialogues were randomly offered by a young female agent or an older male agent. Participants were asked to think aloud during interaction. Afterward, they rated the likeliness of following the agent's advice, agent characteristics, rapport, usability, and intention to use and participated in a semistructured interview. RESULTS: A total of 20 older adults (72.2 [SD 3.5] years) participated. The older male agent was perceived as more authoritative than the young female agent (P=.03), but no other differences were found. The app scored high on usability (median 6.1) and intention to use (median 6.0). Participants indicated they did not see an added value of the agent to the health app. CONCLUSIONS: Agent age and gender little influence users' impressions after short interaction but remain important at first glance to lower the threshold to interact with the agent. Thus, it is important to take the design of ECAs into account when implementing them into health apps.

Tailoring Persuasive Electronic Health Strategies for Older Adults on the Basis of Personal Motivation: Web-Based Survey Study.

BACKGROUND: Persuasive design, in which the aim is to change attitudes and behaviors by means of technology, is an important aspect of electronic health (eHealth) design. However, selecting the right persuasive feature for an individual is a delicate task and is likely to depend on individual characteristics. Personalization of the persuasive strategy in an eHealth intervention therefore seems to be a promising approach. OBJECTIVE: This study aimed to develop a method that allows us to model motivation in older adults with respect to leading a healthy life and a strategy for personalizing the persuasive strategy of an eHealth intervention, based on this user model. METHODS: We deployed a Web-based survey among older adults (aged >60 years) in the Netherlands. In the first part, we administered an adapted version of the revised Sports Motivation Scale (SMS-II) as input for the user models. Then, we provided each participant with a selection of 5 randomly chosen mock-ups (out of a total of 11), each depicting a different persuasive strategy. After showing each strategy, we asked participants how much they appreciated it. The survey was concluded by addressing demographics. RESULTS: A total of 212 older adults completed the Web-based survey, with a mean age of 68.35 years (SD 5.27 years). Of 212 adults, 45.3% were males (96/212) and 54.7% were female (116/212). Factor analysis did not allow us to replicate the 5-factor structure for motivation, as targeted by the SMS-II. Instead, a 3-factor structure emerged with a total explained variance of 62.79%. These 3 factors are intrinsic motivation, acting to derive satisfaction from the behavior itself (5 items; Cronbach alpha=.90); external regulation, acting because of externally controlled rewards or punishments (4 items; Cronbach alpha=.83); and a-motivation, a situation where there is a lack of intention to act (2 items; r=0.50; P<.001). Persuasive strategies were appreciated differently, depending on the type of personal motivation. In some cases, demographics played a role. CONCLUSIONS: The personal type of motivation of older adults (intrinsic, externally regulated, and/or a-motivation), combined with their educational level or living situation, affects an individual's like or dislike for a persuasive eHealth feature. We provide a practical approach for profiling older adults as well as an overview of which persuasive features should or should not be provided to each profile. Future research should take into account the coexistence of multiple types of motivation within an individual and the presence of a-motivation.

Assessing usability of eHealth technology: A comparison of usability benchmarking instruments.

BACKGROUND: It is generally assumed that usability benchmarking instruments are technology agnostic. The same methods for usability evaluations are used for digital commercial, educational, governmental and healthcare systems. However, eHealth technologies have unique characteristics. They need to support patients' health, provide treatment or monitor progress. Little research is done on the effectiveness of different benchmarks (qualitative and quantitative) within the eHealth context. OBJECTIVES: In this study, we compared three usability benchmarking instruments (logging task performance, think aloud and the SUS, the System Usability Scale) to assess which metric is most indicative of usability in an eHealth technology. Also, we analyzed how these outcome variables (task completion, system usability score, serious and critical usability issues) interacted with the acceptance factors Perceived benefits, Usefulness and Intention to use. METHODS: A usability evaluation protocol was set up that incorporated all three benchmarking methods. This protocol was deployed among 36 Dutch participants and across three different eHealth technologies: a gamified application for older adults (N = 19), an online tele-rehabilitation portal for healthcare professionals (N = 9), and a mobile health app for adolescents (N = 8). RESULTS: The main finding was that task completion, compared to the SUS, had stronger correlations with usability benchmarks. Also, serious and critical issues were stronger correlated to task metrics than the SUS. With regard to acceptance factors, there were no significant differences between the three usability benchmarking instruments. CONCLUSIONS: With this study, we took a first step in examining how to improve usability evaluations for eHealth. The results show that listing usability issues from think aloud protocols remains one of the most effective tools to explain the usability for eHealth. Using the SUS as a stand-alone usability metric for eHealth is not recommended. Preferably, the SUS should be combined with task metrics, especially task completion. We recommend to develop a usability benchmarking instrument specifically for eHealth.