Evidence-related requirements in Swedish public sector procurement of health and welfare technologies - a systematic review.

BACKGROUND: Health and welfare technologies (HWT) are becoming increasingly employed in the Nordic countries, and in Sweden in particular. The amount of HWT public procurement is likely increasing at a similar rate, but requirements for evidence for effectiveness placed on bidders during this process may be lacking. METHOD: This study investigated the use of evidence as a requirement in public sector tendering process of HWT, and how it affected bidder attributes and procurement outcomes. A novel type of systematic review and content analysis of requests for tenders for HWT announced prior to June 2021 was therefore conducted in Swedish public procurement databases. RESULT: Ninety requests for tenders for 11 types of HWT met the inclusion criteria for review, accounting for potential contracts worth 246 to 296 million EUR. Criteria requiring evidence for effectiveness were used in 16 requests for tenders, accounting for 183 million EUR in potential contracts. Eight of the requests referred to an established independent standard to confirm such evidence, such as CE standard of conformity, MDR and/or MDD. This prevalence appears to cut across all types of procuring organisations and all types of HWT. The use of any evidence criteria, or lack thereof, does not appear to affect the outcomes of the tendering process. CONCLUSION: Criteria requiring evidence for effectiveness are used in less than a fifth of all public procurements of health- and welfare technologies in Sweden, and less than 10% refer to some form of independent standard as confirmation of such evidence. The procurement process therefore risks creating a legacy of sub-optimal technologies in health- and social care services. More prevalent and specific requirements for evidence and its continual generation in the procurement process are highly recommended. Recommendations for decision makers, procurement managers, and developers are provided.

Gray Literature in Evaluating Effectiveness in Digital Health and Health and Welfare Technology: A Source Worth Considering.

BACKGROUND: The need to assess the effectiveness and value of interventions involving digital health and health and welfare technologies is becoming increasingly important due to the rapidly growing development of these technologies and their areas of application. Systematic reviews of scientific literature are a mainstay of such assessment, but publications outside the realm of traditional scientific bibliographic databases-known as gray literature-are often not included. This is a disadvantage, particularly apparent in the health and welfare technology (HWT) domain. OBJECTIVE: The aim of this article is to investigate the significance of gray literature in digital health and HWT when reviewing literature. As an example, the impact of including gray literature to the result of two systematic reviews in HWT is examined. METHODS: In this paper, we identify, discuss, and suggest methods for including gray literature sources when evaluating effectiveness and appropriateness for different review types related to HWT. The analysis also includes established sources, search strategies, documentation, and reporting of searches, as well as bias and credibility assessment. The differences in comparison to scientific bibliographic databases are elucidated. We describe the results, challenges, and benefits of including gray literature in 2 examples of systematic reviews of HWT. RESULTS: In the 2 systematic reviews described in this paper, most included studies came from context-specific gray literature sources. Gray literature contributed to the overall result of the reviews and corresponded well with the reviews' aims. The assessed risk of bias of the included studies derived from gray literature was similar to the included studies from other types of sources. However, because of less standardized publication formats, assessing and extracting data from gray literature studies were more time-consuming and compiling statistical results was not possible. The search process for gray literature required more time and the reproducibility of gray literature searches were less certain due to more unstable publication platforms. CONCLUSIONS: Gray literature is particularly relevant for digital health and HWT but searches need to be conducted systematically and reported transparently. This way gray literature can broaden the range of studies, highlight context specificity, and decrease the publication bias of reviews of effectiveness of HWT. Thus, researchers conducting systematic reviews related to HWT should consider including gray literature based on a systematic approach.

Nocturnal digital surveillance in aged populations and its effects on health, welfare and social care provision: a systematic review.

BACKGROUND: Nocturnal digital surveillance technologies are being widely implemented as interventions for remotely monitoring elderly populations, and often replace person-based surveillance. Such interventions are often placed in care institutions or in the home, and monitored by qualified personnel or relatives, enabling more rapid and/or frequent assessment of the individual's need for assistance than through on-location visits. This systematic review summarized the effects of these surveillance technologies on health, welfare and social care provision outcomes in populations ≥ 50 years, compared to standard care. METHOD: Primary studies published 2005-2020 that assessed these technologies were identified in 11 databases of peer-reviewed literature and numerous grey literature sources. Initial screening, full-text screening, and citation searching steps yielded the studies included in the review. The Risk of Bias and ROBINS-I tools were used for quality assessment of the included studies. RESULT: Five studies out of 744 identified records met inclusion criteria. Health-related outcomes (e.g. accidents, 2 studies) and social care outcomes (e.g. staff burden, 4 studies) did not differ between interventions and standard care. Quality of life and affect showed improvement (1 study each), as did economic outcomes (1 study). The quality of studies was low however, with all studies possessing a high to critical risk of bias. CONCLUSIONS: We found little evidence for the benefit of nocturnal digital surveillance interventions as compared to standard care in several key outcomes. Higher quality intervention studies should be prioritized in future research to provide more reliable evidence.

Mobile Safety Alarms Based on GPS Technology in the Care of Older Adults: Systematic Review of Evidence Based on a General Evidence Framework for Digital Health Technologies.

BACKGROUND: GPS alarms aim to support users in independent activities. Previous systematic reviews have reported a lack of clear evidence of the effectiveness of GPS alarms for the health and welfare of users and their families and for social care provision. As GPS devices are currently being implemented in social care, it is important to investigate whether the evidence of their clinical effectiveness remains insufficient. Standardized evidence frameworks have been developed to ensure that new technologies are clinically effective and offer economic value. The frameworks for analyzing existing evidence of the clinical effectiveness of GPS devices can be used to identify the risks associated with their implementation and demonstrate key aspects of successful piloting or implementation. OBJECTIVE: The principal aim of this study is to provide an up-to-date systematic review of evidence based on existing studies of the effects of GPS alarms on health, welfare, and social provision in the care of older adults compared with non-GPS-based standard care. In addition, the study findings were assessed by using the evidence standards framework for digital health technologies (DHTs) established by the National Institute for Health and Care Excellence (NICE) in the United Kingdom. METHODS: This review was conducted according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Primary studies published in peer-reviewed journals and gray literature from January 2005 to August 2020 were identified through searches in 13 databases and several sources of gray literature. Included studies had individuals (aged ≥50 years) who were receiving social care for older adults or for persons with dementia; used GPS devices as an intervention; were performed in Canada, the United States, European Union, Singapore, Australia, New Zealand, Hong Kong, South Korea, or Japan; and addressed quantitative outcomes related to health, welfare, and social care. The study findings were analyzed by using the NICE framework requirements for active monitoring DHTs. RESULTS: Of the screened records, 1.6% (16/986) were included. Following the standards of the NICE framework, practice evidence was identified for the tier 1 categories Relevance to current pathways in health/social care system and Acceptability with users, and minimum evidence was identified for the tier 1 category Credibility with health, social care professionals. However, several evidence categories for tiers 1 and 2 could not be assessed, and no clear evidence demonstrating effectiveness could be identified. Thus, the evidence required for using DHTs to track patient location according to the NICE framework was insufficient. CONCLUSIONS: Evidence of the beneficial effects of GPS alarms on the health and welfare of older adults and social care provision remains insufficient. This review illustrated the application of the NICE framework in analyses of evidence, demonstrated successful piloting and acceptability with users of GPS devices, and identified implications for future research.

Obstacles to Evidence-Based Procurement, Implementation, and Evaluation of Health and Welfare Technologies in Swedish Municipalities: Mixed Methods Study.

BACKGROUND: Health and welfare technologies (HWTs) are interventions that aim at maintaining or promoting health, well-being, quality of life, and increasing efficiency in the service delivery system of welfare, social, and health care services, while improving the working conditions of the staff. Health and social care must be evidence-based according to national policy, but there are indications that evidence for HWT effectiveness is lacking in related Swedish municipal work processes. OBJECTIVE: This study aimed to investigate whether the evidence is used when Swedish municipalities procure, implement, and evaluate HWT, and if so, the kinds of evidence and the manner of their use. The study also aimed to identify if municipalities currently receive adequate support in using evidence for HWT, and if not, what support is desired. METHODS: An explanatory sequential mixed methods design was used with quantitative surveys and subsequent semistructured interviews with officials in 5 nationally designated "model" municipalities regarding HWT implementation and use. RESULTS: In the past 12 months, 4 of 5 municipalities had required some form of evidence during procurement processes, but the frequency of this varied and often consisted of references from other municipalities instead of other objective sources. Formulating requirements or requests for evidence during procurement was viewed as difficult, and gathered evidence was often only assessed by procurement administration personnel. In total, 2 of 5 municipalities used an established process for the implementation of HWT, and 3 of 5 had a plan for structured follow-up, but the use and dissemination of evidence within these were varying and often weakly integrated. Standardized processes for follow-up and evaluation across municipalities did not exist, and those processes used by individual municipalities were described as inadequate and difficult to follow. Most municipalities desired support for using evidence when procuring, establishing evaluation frameworks for, and following up effectiveness of HWT, while all municipalities suggested tools or methods for this kind of support. CONCLUSIONS: Structured use of evidence in procurement, implementation, and evaluation of HWT is inconsistent among municipalities, and internal and external dissemination of evidence for effectiveness is rare. This may establish a legacy of ineffective HWT in municipal settings. The results suggest that existing national agency guidance is not sufficient to meet current needs. New, more effective types of support to increase the use of evidence in critical phases of municipal procurement and implementation of HWT are recommended.

Digital Microlearning for Training and Competency Development of Older Adult Care Personnel: Mixed Methods Intervention Study to Assess Needs, Effectiveness, and Areas of Application.

BACKGROUND: Older adult care organizations face challenges today due to high personnel turnover and pandemic-related obstacles in conducting training and competence development programs in a time-sensitive and fit-for-purpose manner. Digital microlearning is a method that attempts to meet these challenges by more quickly adapting to the educational needs of organizations and individual employees in terms of time, place, urgency, and retention capacity more than the traditional competency development methods. OBJECTIVE: This study aimed to determine if and how an app-based digital microlearning intervention can meet older adult care organizations' personnel competency development needs in terms of knowledge retention and work performance. METHODS: This study assessed the use of a digital microlearning app, which was at the testing stage in the design thinking model among managerial (n=4) and operational (n=22) employees within 3 older adult care organizations. The app was used to conduct predetermined competency development courses for the staff. Baseline measurements included participants' previous training and competency development methods and participation, as well as perceived needs in terms of time, design, and channel. They then were introduced to and used a digital microlearning app to conduct 2 courses on one or more digital devices, schedules, and locations of their own choice during a period of ~1 month. The digital app and course content, perceived knowledge retention, and work performance and satisfaction were individually assessed via survey upon completion. The survey was complemented with 4 semistructured focus group interviews, which allowed participants (in total 16 individuals: 6 managerial-administrative employees and 10 operational employees) to describe their experiences with the app and its potential usefulness within their organizations. RESULTS: The proposed advantages of the digital microlearning app were largely confirmed by the participants' perceptions, particularly regarding the ease of use and accessibility, and efficiency and timeliness of knowledge delivery. Assessments were more positive among younger or less experienced employees with more diverse backgrounds. Participants expressed a positive inclination toward using the app, and suggestions provided regarding its potential development and broader use suggested a positive view of digitalization in general. CONCLUSIONS: Our results show that app-based digital microlearning appears to be an appropriate new method for providing personnel competency development within the older adult care setting. Its implementation in a larger sample can potentially provide more detailed insights regarding its intended effects.

Artificial Intelligence and Precision Health Through Lenses of Ethics and Social Determinants of Health: Protocol for a State-of-the-Art Literature Review.

BACKGROUND:  Precision health is a rapidly developing field, largely driven by the development of artificial intelligence (AI)-related solutions. AI facilitates complex analysis of numerous health data risk assessment, early detection of disease, and initiation of timely preventative health interventions that can be highly tailored to the individual. Despite such promise, ethical concerns arising from the rapid development and use of AI-related technologies have led to development of national and international frameworks to address responsible use of AI. OBJECTIVE:  We aimed to address research gaps and provide new knowledge regarding (1) examples of existing AI applications and what role they play regarding precision health, (2) what salient features can be used to categorize them, (3) what evidence exists for their effects on precision health outcomes, (4) how do these AI applications comply with established ethical and responsible framework, and (5) how these AI applications address equity and social determinants of health (SDOH). METHODS:  This protocol delineates a state-of-the-art literature review of novel AI-based applications in precision health. Published and unpublished studies were retrieved from 6 electronic databases. Articles included in this study were from the inception of the databases to January 2023. The review will encompass applications that use AI as a primary or supporting system or method when primarily applied for precision health purposes in human populations. It includes any geographical location or setting, including the internet, community-based, and acute or clinical settings, reporting clinical, behavioral, and psychosocial outcomes, including detection-, diagnosis-, promotion-, prevention-, management-, and treatment-related outcomes. RESULTS:   This is step 1 toward a full state-of-the-art literature review with data analyses, results, and discussion of findings, which will also be published. The anticipated consequences on equity from the perspective of SDOH will be analyzed. Keyword cluster relationships and analyses will be visualized to indicate which research foci are leading the development of the field and where research gaps exist. Results will be presented based on the data analysis plan that includes primary analyses, visualization of sources, and secondary analyses. Implications for future research and person-centered public health will be discussed. CONCLUSIONS:  Results from the review will potentially guide the continued development of AI applications, future research in reducing the knowledge gaps, and improvement of practice related to precision health. New insights regarding examples of existing AI applications, their salient features, their role regarding precision health, and the existing evidence that exists for their effects on precision health outcomes will be demonstrated. Additionally, a demonstration of how existing AI applications address equity and SDOH and comply with established ethical and responsible frameworks will be provided. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): PRR1-10.2196/40565.