Mapping Theories, Models, and Frameworks to Evaluate Digital Health Interventions: Scoping Review.

BACKGROUND: Digital health interventions (DHIs) are a central focus of health care transformation efforts, yet their uptake in practice continues to fall short of their potential. In order to achieve their desired outcomes and impact, DHIs need to reach their target population and need to be used. Many factors can rapidly intersect between this dynamic of users and interventions. The application of theories, models, and frameworks (TMFs) can facilitate the systematic understanding and explanation of the complex interactions between users, practices, technology, and health system factors that underpin research questions. There remains a gap in our understanding of how TMFs have been applied to guide the evaluation of DHIs with real-world health system operations. OBJECTIVE: This study aims to map TMFs used in studies to guide the evaluation of DHIs. The objectives are to (1) describe the TMFs and the constructs they target, (2) identify how TMFs have been prospectively used (ie, their roles) in primary studies to evaluate DHIs, and (3) to reflect on the relevance and utility of our findings for knowledge users. METHODS: This scoping review was conducted in partnership with knowledge users using an integrated knowledge translation approach. We included papers (eg, reports; empirical quantitative, qualitative, and mixed methods studies; conference proceedings; and dissertations) if primary insights resulting from the application of TMFs were presented. Any type of DHI was eligible. Papers published from 2000 and onward were mainly identified from the following databases: MEDLINE (Ovid), CINAHL Complete (EBSCOhost), PsycINFO (Ovid), EBM Reviews (Ovid), and Embase (Ovid). RESULTS: A total of 156 studies published between 2000 and 2022 were included. A total of 68 distinct TMFs were identified across 85 individual studies. In more than half (85/156, 55%) of the included studies, 1 of following 6 prevailing TMFs were reported: Consolidated Framework for Implementation Research (n=39); the Reach, Effectiveness, Adoption, Implementation, and Maintenance Framework (n=17); the Technology of Acceptance Model (n=16); the Unified Theory on Acceptance and Use of Technology (n=12); the Diffusion of Innovation Theory (n=10); and Normalization Process Theory (n=9). The most common intended roles of the 6 TMFs were to inform data collection (n=86), to inform data analysis (n=69), and to identify key constructs that may serve as barriers and facilitators (n=52). CONCLUSIONS: As TMFs are most often reported to be applied to support data collection and analysis, researchers should consider more clearly synthesizing key insights as practical use cases to both increase the relevance and digestibility of their findings. There is also a need to adapt or develop guidelines for better reporting DHIs and the use of TMFs to guide evaluation. Hence, it would contribute to ensuring ongoing technology transformation efforts are evidence and theory informed rather than anecdotally driven.

Electronic communication between family physicians and patients: Findings from a multisite survey of academic family physicians in Ontario.

OBJECTIVE: To assess the proportion of academic family physicians using e-mail with patients and to explore related attitudes, barriers, and facilitators. DESIGN: A 47-item questionnaire was created after a literature review, discussions with study team members, pretesting, and pilot testing. The questionnaire was disseminated electronically from June to August 2017. SETTING: Ontario. PARTICIPANTS: All family physicians affiliated with the Department of Family and Community Medicine at the University of Toronto. MAIN OUTCOME MEASURES: Physician practices using e-mail (including barriers to and facilitators of e-mail use with patients), use of e-mail with other health care providers, use of communication technologies other than e-mail, and demographic and practice information. RESULTS: A total of 1553 surveys were disseminated and 865 responses received (56% response rate). Overall, 610 respondents met inclusion criteria. Of these respondents, 43% (265 of 610) personally sent e-mails to patients in a typical week. An additional 21% (126 of 610) reported that they did not personally e-mail patients, but their clinic staff did. Patient convenience and a decrease in the need for telephone communication were the most commonly noted reasons for e-mail use. Facilitators of e-mail use included integration with the electronic medical record, enhanced e-mail access control, security features, and financial compensation. Barriers to e-mail use included privacy and security concerns, concerns about inappropriate e-mail use by patients, and the creation of unrealistic expectations about physician availability. CONCLUSION: E-mail use between academic family physicians and patients was found to be much higher than shown in previous studies of Canadian physicians. This finding might have been owing to unique aspects of academic medicine, remuneration via capitation, or other factors. Efforts to increase physician use of e-mail with patients should address concerns related to privacy and security, electronic medical record integration, and financial compensation.

Alpine glacier resilience and Neoglacial fluctuations linked to Holocene snowfall trends in the western United States.

Geological evidence indicates that glaciers in the western United States fluctuated in response to Holocene changes in temperature and precipitation. However, because moraine chronologies are characteristically discontinuous, Holocene glacier fluctuations and their climatic drivers remain ambiguous, and future glacier changes are uncertain. Here, we construct a continuous 10-thousand-year (ka) record of glacier activity in the Teton Range, Wyoming, using glacial and environmental indicators in alpine lake sediments. We show that Teton glaciers persisted in some form through early Holocene warmth, likely as small debris-covered glaciers or rock glaciers. Subsequent Neoglacial ice expansion began ~6.3 ka, with two prominent glacier maxima at ~2.8 and 0.1 ka that were separated by a multicentennial phase of ice retreat. Comparison with regional paleoclimate records suggests that glacier activity was dominantly controlled by winter precipitation variability superposed on long-term Holocene temperature trends, offering key insights into western U.S. glacier resilience and vulnerability to future warming.

Identifying optimal frameworks to implement or evaluate digital health interventions: a scoping review protocol.

INTRODUCTION: Digital health interventions (DHIs) are defined as health services delivered electronically through formal or informal care. DHIs can range from electronic medical records used by providers to mobile health apps used by consumers. DHIs involve complex interactions between user, technology and the healthcare team, posing challenges for implementation and evaluation. Theoretical or interpretive frameworks are crucial in providing researchers guidance and clarity on implementation or evaluation approaches; however, there is a lack of standardisation on which frameworks to use in which contexts. Our goal is to conduct a scoping review to identify frameworks to guide the implementation or evaluation of DHIs. METHODS AND ANALYSIS: A scoping review will be conducted using methods outlined by the Joanna Briggs Institute reviewers' manual and will conform to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews. Studies will be included if they report on frameworks (ie, theoretical, interpretive, developmental) that are used to guide either implementation or evaluation of DHIs. Electronic databases, including MEDLINE, EMBASE, CINAHL and PsychINFO will be searched in addition to grey literature and reference lists of included studies. Citations and full text articles will be screened independently in Covidence after a reliability check among reviewers. We will use qualitative description to summarise findings and focus on how research objectives and type of DHIs are aligned with the frameworks used. ETHICS AND DISSEMINATION: We engaged an advisory panel of digital health knowledge users to provide input at strategic stages of the scoping review to enhance the relevance of findings and inform dissemination activities. Specifically, they will provide feedback on the eligibility criteria, data abstraction elements, interpretation of findings and assist in developing key messages for dissemination. This study does not require ethical review. Findings from review will support decision making when selecting appropriate frameworks to guide the implementation or evaluation of DHIs.

A model to measure self-assessed proficiency in electronic medical records: Validation using maturity survey data from Canadian community-based physicians.

OBJECTIVE: Adoption of electronic medical records (EMRs) does not necessarily translate to proficiency -referred to here as EMR maturity. To realize the full benefit of wide scale EMR adoption, the focus must shift from adoption to advancing mature use. This calls for validated assessment models so that researchers, health system planners and digital health developers can better understand what contributes to maturity among physicians. This research aims to validate a measurement model for self-assessed EMR maturity among community-based physicians. METHODS: As part of an Ontario government-funded EMR adoption program, the EMR Maturity Model for community-based practices was adapted from a hospital-based EMR maturity model. A survey instrument was developed on the foundation of the new model and revised by experts and stakeholders. Content validity, face validity and user acceptance were established before survey administration. Internal consistency and construct validity of the model were tested after survey data were collected. Finally, physicians' comments collected via the survey were qualitatively analyzed to provide additional insights that can be applied to refinement of the model and survey. RESULTS: As of August 1, 2019, 1588 physicians completed the survey. Ordinal alpha tests for reliability and content validity yielded an alpha value of 0.86 across all key measures specifically associated with maturity. Among most of these, there was a pattern of weak to moderate significant (p < .0001) positive Spearman inter-correlations. One factor was extracted for items measuring dimensions of maturity and all factor loadings of the key measures were greater than 0.40. The fit of the one-factor model was moderately adequate. This indicates the model is valid and reliable, with consistency across key measures for measuring one factor: maturity. CONCLUSIONS: This is the first known validated model published in English that measures EMR maturity among community-based physicians. While the model is shown to be valid and reliable statistically and qualitative analysis supports this, there is room for improvement. Both the statistical analysis and portions of the qualitative analysis suggest areas of exploration to strengthen the model and survey. Future efforts will include refining the survey to improve user interface and accrue further data, as the sample to date is insufficient for generalizability.

Progress in the Enhanced Use of Electronic Medical Records: Data From the Ontario Experience.

BACKGROUND: This paper describes a change management strategy, including a self-assessment survey tool and electronic medical record (EMR) maturity model (EMM), developed to support the adoption and implementation of EMRs among community-based physicians in the province of Ontario, Canada. OBJECTIVE: The aim of our study was to present an analysis of progress in EMR use in the province of Ontario based on data from surveys completed by over 4000 EMR users. METHODS: The EMM and the EMR progress report (EPR) survey tool clarify levels of capability and expected benefits of improved use. Maturity is assessed on a 6-point scale (0-5) for 25 functions, across 7 functional areas, ranging from basic to more advanced. A total of 4214 clinicians completed EPR surveys between April 2013 and March 2016. Univariate and multivariate descriptive statistics were calculated to describe the survey results. RESULTS: Physicians reported continual improvement over years of use, perceiving that the longer they used their EMR, the better patient care they provided. Those with at least two years of experience reported the greatest progress. CONCLUSIONS: From our analyses at this stage we identified: (1) a direct correlation between years of EMR use and EMR maturity as measured in our model, (2) a similar positive correlation between years of EMR use and the perception that these systems improve clinical care in at least four patient-centered areas, and (3) evidence of ongoing improvement even in advanced years of use. Future analyses will be supplemented by qualitative and quantitative data collected from field staff engagements as part of the new EMR practice enhancement program (EPEP).

A pragmatic randomized control trial and realist evaluation on the implementation and effectiveness of an internet application to support self-management among individuals seeking specialized mental health care: a study protocol.

BACKGROUND: Mental illness is a substantial and rising contributor to the global burden of disease. Access to and utilization of mental health care, however, is limited by structural barriers such as specialist availability, time, out-of-pocket costs, and attitudinal barriers including stigma. Innovative solutions like virtual care are rapidly entering the health care domain. The advancement and adoption of virtual care for mental health, however, often occurs in the absence of rigorous evaluation and adequate planning for sustainability and spread. METHODS: A pragmatic randomized controlled trial with a nested comparative effectiveness arm, and concurrent realist process evaluation to examine acceptability, effectiveness, and cost-effectiveness of the Big White Wall (BWW) online platform for mental health self-management and peer support among individuals aged 16 and older who are accessing mental health services in Ontario, Canada. Participants will be randomized to 3 months of BWW or treatment as usual. At the end of the 3 months, participants in the intervention group will have the opportunity to opt-in to an intervention extension arm. Those who opt-in will be randomized to receive an additional 3 months of BWW or no additional intervention. The primary outcome is recovery at 3 months as measured by the Recovery Assessment Scale-revised (RAS-r). Secondary outcomes include symptoms of depression and anxiety measured with the Personal Health Questionnaire-9 item (PHQ-9) and the Generalized Anxiety Disorder Questionnaire-7 item (GAD-7) respectively, quality of life measured with the EQ-5D-5L, and community integration assessed with the Community Integration Questionnaire. Cost-effectiveness evaluations will account for the cost of the intervention and direct health care costs. Qualitative interviews with participants and stakeholders will be conducted throughout. DISCUSSION: Understanding the impact of virtual strategies, such as BWW, on patient outcomes and experience, and health system costs is essential for informing whether and how health system decision-makers can support these strategies system-wide. This requires clear evidence of effectiveness and an understanding of how the intervention works, for whom, and under what circumstances. This study will produce such effectiveness data for BWW, while simultaneously exploring the characteristics and experiences of users for whom this and similar online interventions could be helpful. TRIAL REGISTRATION: Clinicaltrials.gov NCT02896894 . Registered on 31 August 2016 (retrospectively registered).

Outcome in patients previously on antithrombotic therapy in the SAMMPRIS trial: subgroup analysis.

BACKGROUND AND PURPOSE: Stenting has been used as a rescue therapy in patients with intracranial arterial stenosis and a transient ischemic attack or stroke when on antithrombotic therapy (AT). We determined whether the stenting versus aggressive medical therapy for intracranial arterial stenosis (SAMMPRIS) trial supported this approach by comparing the treatments within subgroups of patients whose qualifying event (QE) occurred on versus off of AT. METHODS: The primary outcome, 30-day stroke and death and later strokes in the territory of the qualifying artery, was compared between (1) percutaneous transluminal angioplasty and stenting plus aggressive medical therapy (PTAS) versus aggressive medical management therapy alone (AMM) for patients whose QE occurred on versus off AT and between (2) patients whose QE occurred on versus off AT separately for the treatment groups. RESULTS: Among the 284/451 (63%) patients who had their QE on AT, the 2-year primary end point rates were 15.6% for those randomized to AMM (n=140) and 21.6% for PTAS (n=144; P=0.043, log-rank test). In the 167 patients not on AT, the 2-year primary end point rates were 11.6% for AMM (n=87) and 18.8% for PTAS (n=80; P=0.31, log-rank test). Within both treatment groups, there was no difference in the time to the primary end point between patients who were on or off AT (AMM, P=0.96; PTAS, P=0.52; log-rank test). CONCLUSIONS: SAMMPRIS results indicate that the benefit of AMM over PTAS is similar in patients on versus off AT at the QE and that failure of AT is not a predictor of increased risk of a primary end point. CLINICAL TRIAL REGISTRATION URL: http://www.clinicaltrials.gov. Unique identifier: NCT00576693.