Methods, Indicators, and End-User Involvement in the Evaluation of Digital Health Interventions for the Public: Scoping Review.

BACKGROUND: Digital health interventions (DHIs) have the potential to enable public end users, such as citizens and patients, to manage and improve their health. Although the number of available DHIs is increasing, examples of successfully established DHIs in public health systems are limited. To counteract the nonuse of DHIs, they should be comprehensively evaluated while integrating end users. Unfortunately, there is a wide variability and heterogeneity according to the approaches of evaluation, which creates a methodological challenge. OBJECTIVE: This scoping review aims to provide an overview of the current established processes for evaluating DHIs, including methods, indicators, and end-user involvement. The review is not limited to a specific medical field or type of DHI but offers a holistic overview. METHODS: This scoping review was conducted following the JBI methodology for scoping reviews based on the framework by Arksey & O'Malley and complies with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. Three scientific databases (PubMed, Scopus, and Science Direct) were searched in April 2023. English and German studies between 2008 and 2023 were considered when evaluating DHIs that explicitly address public end users. The process of study selection was carried out by several researchers to avoid reviewer bias. RESULTS: The search strategy identified 9618 publications, of which 160 were included. Among these included articles, 200 evaluations were derived and analyzed. The results showed that there is neither a consensus on the methods to evaluate DHIs nor a commonly agreed definition or usage of the evaluated indicators, which results in a broad variety of evaluation practices. This aligns with observations of the existing literature. It was found that there is a lack of references to existing frameworks for the evaluation of DHIs. The majority of the included studies referred to user-centered approaches and involved end users in the evaluation process. As assistance for people developing and evaluating DHIs and as a basis for thinking about appropriate ways to evaluate DHIs, a results matrix was created where the findings were combined per DHI cluster. Additionally, general recommendations for the evaluators of DHIs were formulated. CONCLUSIONS: The findings of this scoping review offer a holistic overview of the variety and heterogeneity according to the approaches of evaluation of DHIs for public end users. Evaluators of these DHIs should be encouraged to reference established frameworks or measurements for justification. This would ease the transferability of the results among similar evaluation studies within the digital health sector, thereby enhancing the coherence and comparability of research in this area.

Measuring the Digital Competence of Health Professionals: Scoping Review.

BACKGROUND: Digital competence is listed as one of the key competences for lifelong learning and is increasing in importance not only in private life but also in professional life. There is consensus within the health care sector that digital competence (or digital literacy) is needed in various professional fields. However, it is still unclear what exactly the digital competence of health professionals should include and how it can be measured. OBJECTIVE: This scoping review aims to provide an overview of the common definitions of digital literacy in scientific literature in the field of health care and the existing measurement instruments. METHODS: Peer-reviewed scientific papers from the last 10 years (2013-2023) in English or German that deal with the digital competence of health care workers in both outpatient and inpatient care were included. The databases ScienceDirect, Scopus, PubMed, EBSCOhost, MEDLINE, OpenAIRE, ERIC, OAIster, Cochrane Library, CAMbase, APA PsycNet, and Psyndex were searched for literature. The review follows the JBI methodology for scoping reviews, and the description of the results is based on the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist. RESULTS: The initial search identified 1682 papers, of which 46 (2.73%) were included in the synthesis. The review results show that there is a strong focus on technical skills and knowledge with regard to both the definitions of digital competence and the measurement tools. A wide range of competences were identified within the analyzed works and integrated into a validated competence model in the areas of technical, methodological, social, and personal competences. The measurement instruments mainly used self-assessment of skills and knowledge as an indicator of competence and differed greatly in their statistical quality. CONCLUSIONS: The identified multitude of subcompetences illustrates the complexity of digital competence in health care, and existing measuring instruments are not yet able to reflect this complexity.

An acceptance analysis of a sexual health education digital tool in resource-poor regions of Kenya: an UTAUT based survey study.

BACKGROUND: Unplanned pregnancies and sexually transmitted diseases are a concern in Sub-Saharan Africa, particularly in low-income areas. Access to sexual health information is limited, partly due to the absence of comprehensive sex education in the national school curriculum and social taboos. In response to these challenges, this study introduces a web-based prototype, designed to provide essential sexual health information, targeting 18 to 35-year-old Kenyans, focusing on contraception, menstruation, and female genital mutilation. METHOD: Aiming to investigate young adults' behavioral intention to use a digital tool for sexuality education, by analyzing factors affecting acceptance and usability in low-income and resource-poor regions in Kenya. To explore the acceptability and use of the developed digital tool, this study used a modified version of the Unified Theory of Acceptance and Use of Technology (UTAUT), complemented by the System Usability Scale (SUS) questionnaire. For statistical analysis, a Structural Equation Model (SEM) including Confirmatory Factor Analysis (CFA) and Linear Regression was used. Regarding the reporting of the E-survey results, the Checklist for Reporting Results of Internet E-surveys (CHERRIES), was considered. RESULTS: Survey information from 77 persons (69 female, 7 male, 1 diverse) were collected. A modified UTAUT appears as an appropriate model for measuring the constructs and integrating evidence-based approaches to advanced and safe sexual healthcare information. Results from the SEM showed perceived usefulness, attitude towards healthcare integrated evidence technology, and usability as well as having a significant positive impact on the acceptance, the intention to use as well as wellbeing. Having the resources and knowledge necessary for the usage of a digital tool turns out to have a significant negative impact. A SUS score of 67.3 indicates the usability of the tool for sexual health information, assessed as okay. CONCLUSIONS: The study adopts validated methods to assess the acceptability and usability of a digital sexual health education tool in Kenya. Emphasizing its potential effectiveness and highlighting the influence of cultural and contextual factors on technology adoption.

Creating an Intercultural User-Centric Design for a Digital Sexual Health Education App for Young Women in Resource-Poor Regions of Kenya: Qualitative Self-Extended Double Diamond Model for Requirements Engineering Analysis.

BACKGROUND: The stigmatization around sexual health due to culture, religious traditions, and norms leads to myths and a lack of available information and resources related to universal access to sexual and reproductive health services. Additional sexual health education not being part of the Kenya school curriculum leads to insufficient access to knowledge about safe contraception, menstruation, and female genital mutilation. A digital app could support and provide education and information for universal equal access, addressing United Nations Sustainable Development Goals 3, 4, and 5. OBJECTIVE: The study targeted the requirements for establishing a reusable framework to develop a successful accessible web-based sexual health education app and the behavioral intention to use it to obtain sexual health information by mainly young women in Kenya. METHODS: The double diamond model, with a problem room including the discover and define phases, enriched with cultural aspects and modeled to a self-expanded intercultural research model was used in a user-centered design thinking approach to develop a framework for requirements engineering analysis. For problem identification, semistructured pilot phase interviews based on Consolidated Criteria for Reporting Qualitative Research guidelines were conducted, followed by expert interviews for qualitative content analysis. A sample size of 12 pilot phase interviews and 5 expert interviews was determined using data saturation. The responses were coded and analyzed according to the affinity mapping method. RESULTS: The requirements engineering analysis showed potential enablers of and barriers to the use of a digital sexual health education app. Through this qualitative study, a conservative cultural background, classic text communication, and the influence of social affiliation within society were identified as barriers, which should be enhanced through visual and auditory channels as well as a fictional character in the app. CONCLUSIONS: The developed intercultural research model provides an impetus to providing digital sexual health education, integrating culture-specific aspects in the design process, while focusing on cultural and religious stigmata. The reusable framework enables identifying and overcoming hurdles in providing information about taboo and intimate topics. The overall use of online education tools focusing on intimate topics is correlated with accessibility and understanding specific cultural needs while delivering content on a basic and comprehensive level. It helps the target user from a social conservative background and in resource-poor circumstances to benefit from a digital educational solution.