Mapping Theories, Models, and Frameworks to Evaluate Digital Health Interventions: Scoping Review.

BACKGROUND: Digital health interventions (DHIs) are a central focus of health care transformation efforts, yet their uptake in practice continues to fall short of their potential. In order to achieve their desired outcomes and impact, DHIs need to reach their target population and need to be used. Many factors can rapidly intersect between this dynamic of users and interventions. The application of theories, models, and frameworks (TMFs) can facilitate the systematic understanding and explanation of the complex interactions between users, practices, technology, and health system factors that underpin research questions. There remains a gap in our understanding of how TMFs have been applied to guide the evaluation of DHIs with real-world health system operations. OBJECTIVE: This study aims to map TMFs used in studies to guide the evaluation of DHIs. The objectives are to (1) describe the TMFs and the constructs they target, (2) identify how TMFs have been prospectively used (ie, their roles) in primary studies to evaluate DHIs, and (3) to reflect on the relevance and utility of our findings for knowledge users. METHODS: This scoping review was conducted in partnership with knowledge users using an integrated knowledge translation approach. We included papers (eg, reports; empirical quantitative, qualitative, and mixed methods studies; conference proceedings; and dissertations) if primary insights resulting from the application of TMFs were presented. Any type of DHI was eligible. Papers published from 2000 and onward were mainly identified from the following databases: MEDLINE (Ovid), CINAHL Complete (EBSCOhost), PsycINFO (Ovid), EBM Reviews (Ovid), and Embase (Ovid). RESULTS: A total of 156 studies published between 2000 and 2022 were included. A total of 68 distinct TMFs were identified across 85 individual studies. In more than half (85/156, 55%) of the included studies, 1 of following 6 prevailing TMFs were reported: Consolidated Framework for Implementation Research (n=39); the Reach, Effectiveness, Adoption, Implementation, and Maintenance Framework (n=17); the Technology of Acceptance Model (n=16); the Unified Theory on Acceptance and Use of Technology (n=12); the Diffusion of Innovation Theory (n=10); and Normalization Process Theory (n=9). The most common intended roles of the 6 TMFs were to inform data collection (n=86), to inform data analysis (n=69), and to identify key constructs that may serve as barriers and facilitators (n=52). CONCLUSIONS: As TMFs are most often reported to be applied to support data collection and analysis, researchers should consider more clearly synthesizing key insights as practical use cases to both increase the relevance and digestibility of their findings. There is also a need to adapt or develop guidelines for better reporting DHIs and the use of TMFs to guide evaluation. Hence, it would contribute to ensuring ongoing technology transformation efforts are evidence and theory informed rather than anecdotally driven.

What We Have Heard: Next Steps for Long-Term Care Pandemic Preparedness in Canada.

In this concluding article, Healthcare Excellence Canada and the Canadian Institutes of Health Research reflect upon and respond to the lessons learned from the contributing articles in the special issue and summarize key takeaways for the next steps in evidence-informed pandemic preparedness in long-term care in Canada. The implications of their cross-organizational partnership for achieving collective impact now and in the future are also discussed.

Opportunities for Action toward a Sustainable Health System.

The current pandemic is a stark reminder that crises bring to light society's vulnerabilities. In the lead paper of this issue of Healthcare Papers, Miller and Xie (2020) argue that the same is - and will be - true for climate change. They make a compelling and urgent case for its importance to health and healthcare in Canada and around the world. Opportunities to advance the multiple interrelated dimensions of sustainability in the health sector include understanding and mitigating the health implications of climate change; preparing the health sector for climate change; and accelerating the health sector's contribution to society-wide net-zero targets. High-performing, resilient health systems with their capacity to deeply engage with communities, and to respond dynamically to changing circumstances, will be key to proactively addressing climate change, just as they are proving to be in pandemic preparedness and response.

Identifying optimal frameworks to implement or evaluate digital health interventions: a scoping review protocol.

INTRODUCTION: Digital health interventions (DHIs) are defined as health services delivered electronically through formal or informal care. DHIs can range from electronic medical records used by providers to mobile health apps used by consumers. DHIs involve complex interactions between user, technology and the healthcare team, posing challenges for implementation and evaluation. Theoretical or interpretive frameworks are crucial in providing researchers guidance and clarity on implementation or evaluation approaches; however, there is a lack of standardisation on which frameworks to use in which contexts. Our goal is to conduct a scoping review to identify frameworks to guide the implementation or evaluation of DHIs. METHODS AND ANALYSIS: A scoping review will be conducted using methods outlined by the Joanna Briggs Institute reviewers' manual and will conform to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews. Studies will be included if they report on frameworks (ie, theoretical, interpretive, developmental) that are used to guide either implementation or evaluation of DHIs. Electronic databases, including MEDLINE, EMBASE, CINAHL and PsychINFO will be searched in addition to grey literature and reference lists of included studies. Citations and full text articles will be screened independently in Covidence after a reliability check among reviewers. We will use qualitative description to summarise findings and focus on how research objectives and type of DHIs are aligned with the frameworks used. ETHICS AND DISSEMINATION: We engaged an advisory panel of digital health knowledge users to provide input at strategic stages of the scoping review to enhance the relevance of findings and inform dissemination activities. Specifically, they will provide feedback on the eligibility criteria, data abstraction elements, interpretation of findings and assist in developing key messages for dissemination. This study does not require ethical review. Findings from review will support decision making when selecting appropriate frameworks to guide the implementation or evaluation of DHIs.

When Small Changes Add Up to System Transformation.

Just as sandy beaches are the result of uncountable individual waves crashing on a shore, fundamental shifts in the health sector often reflect the cumulative result of hundreds - perhaps thousands - of individual decisions. Each, on its own, may seem to have only local effect. But together, they represent a transformation in how care is delivered.

Nothing About Me Without Me.

Active and effective partnerships with patients are increasingly recognized as key to improving the design, delivery and organization of health services and policy. Co-designing for improvement yields new insights and tends to lead to better results than healthcare providers, researchers, or policy makers acting on their own. (For convenience, I use the term "patient" here since it has been identified as a term of choice in several Canadian focus groups. I intend it in an inclusive sense, not only referring to those who have a particular health problem or who use specific health services, but also family, friends, and other caregivers, as well as those who bring other important perspectives from their lived experience of interactions with the health and social sectors.).

Reflections on a Decade of Healthcare Policy/Politiques de Santé.

It has been a fantastic opportunity to be the editor for this journal over the last decade. I have very much enjoyed being exposed to the broad range of health services and policy research papers that have been submitted to the journal - more than 1,000 manuscripts that reflect a diverse set of questions, approaches, contributors and conclusions. The articles in this issue exemplify that diversity. Some themes continue to resonate strongly: health human resources, primary healthcare, and healthcare financing have been important topics since the journal was founded, for instance. But there have been changes too. Some topics - such as medical assistance in dying - have emerged as central questions in healthcare policy. We now see more research being undertaken with patients and communities. And the ratio of qualitative and mixed-methods research to quantitative analysis has risen.

An Assessment Framework for e-Mental Health Apps in Canada: Results of a Modified Delphi Process.

BACKGROUND: The number of e-mental health apps is increasing rapidly. Studies have shown that the use of some apps is beneficial, whereas others are ineffective or do not meet users' privacy expectations. Individuals and organizations that curate, recommend, host, use, or pay for apps have an interest in categorizing apps according to the consensus criteria of usability and effectiveness. Others have previously published recommendations for assessing health-related apps; however, the extent to which these recommendations can be generalized across different population groups (eg, culture, gender, and language) remains unclear. This study describes an attempt by Canadian stakeholders to develop an e-mental health assessment framework that responds to the unique needs of people living in Canada in an evidence-based manner. OBJECTIVE: The objective of our study was to achieve consensus from a broad group of Canadian stakeholders on guiding principles and criteria for a framework to assess e-mental health apps in Canada. METHODS: We developed an initial set of guiding principles and criteria from a rapid review and environmental scan of pre-existing app assessment frameworks. The initial list was refined through a two-round modified Delphi process. Participants (N=25) included app developers and users, health care providers, mental health advocates, people with lived experience of a mental health problem or mental illness, policy makers, and researchers. Consensus on each guideline or criterion was defined a priori as at least 70% agreement. The first round of voting was conducted electronically. Prior to Round 2 voting, in-person presentations from experts and a persona empathy mapping process were used to explore the perspectives of diverse stakeholders. RESULTS: Of all respondents, 68% (17/25) in Round 1 and 100% (13/13) in Round 2 agreed that a framework for evaluating health apps is needed to help Canadian consumers identify high-quality apps. Consensus was reached on 9 guiding principles: evidence based, gender responsive, culturally appropriate, user centered, risk based, internationally aligned, enabling innovation, transparent and fair, and based on ethical norms. In addition, 15 informative and evaluative criteria were defined to assess the effectiveness, functionality, clinical applicability, interoperability, usability, transparency regarding security and privacy, security or privacy standards, supported platforms, targeted users, developers' transparency, funding transparency, price, user desirability, user inclusion, and meaningful inclusion of a diverse range of communities. CONCLUSIONS: Canadian mental health stakeholders reached the consensus on a framework of 9 guiding principles and 15 criteria important in assessing e-mental health apps. What differentiates the Canadian framework from other scales is explicit attention to user inclusion at all stages of the development, gender responsiveness, and cultural appropriateness. Furthermore, an empathy mapping process markedly influenced the development of the framework. This framework may be used to inform future mental health policies and programs.

Illuminating the Consequences of Policy Change.

The ripple effects of major policy changes are difficult to predict in advance. That is why the ability to track consequences of decisions - intended and unintended - is an important role of health services and policy research. In this issue of the journal, Janine Brown, Lilian Thorpe and Donna Goodridge discuss our ability to track medically assisted death (MAiD), one of the most significant health policy changes in Canada in recent history. Whatever one's views on MAiD, there is consensus on the need to understand how often, and in what circumstances, such deaths take place. Being able to track deaths in a consistent way also matters. Comparisons between different population groups, across regions of the country, and with other nations can be helpful in illuminating the effects of this policy and its application.

International health IT benchmarking: learning from cross-country comparisons.

OBJECTIVE: To pilot benchmark measures of health information and communication technology (ICT) availability and use to facilitate cross-country learning. MATERIALS AND METHODS: A prior Organization for Economic Cooperation and Development-led effort involving 30 countries selected and defined functionality-based measures for availability and use of electronic health records, health information exchange, personal health records, and telehealth. In this pilot, an Organization for Economic Cooperation and Development Working Group compiled results for 38 countries for a subset of measures with broad coverage using new and/or adapted country-specific or multinational surveys and other sources from 2012 to 2015. We also synthesized country learnings to inform future benchmarking. RESULTS: While electronic records are widely used to store and manage patient information at the point of care-all but 2 pilot countries reported use by at least half of primary care physicians; many had rates above 75%-patient information exchange across organizations/settings is less common. Large variations in the availability and use of telehealth and personal health records also exist. DISCUSSION: Pilot participation demonstrated interest in cross-national benchmarking. Using the most comparable measures available to date, it showed substantial diversity in health ICT availability and use in all domains. The project also identified methodological considerations (e.g., structural and health systems issues that can affect measurement) important for future comparisons. CONCLUSION: While health policies and priorities differ, many nations aim to increase access, quality, and/or efficiency of care through effective ICT use. By identifying variations and describing key contextual factors, benchmarking offers the potential to facilitate cross-national learning and accelerate the progress of individual countries.