Using "Big Data" to Provide Insights into Early Adopters of Continuing Professional Development: An Example from Project ECHO.

INTRODUCTION: Mental health care is often managed in primary care with limited specialist support, particularly in rural and remote communities. Continuing professional development programs (CPD) can offer a potential solution to further mental health training; however, engaging primary care organizations (PCOs) can be challenging. The use of "big data" to identify factors influencing engagement in CPD programs has not been well studied. Therefore, the aim of this project was to use administrative health data from Ontario, Canada to identify characteristics of PCOs associated with early engagement in a virtual CPD program, Project Extension for Community Healthcare Outcomes (ECHO) Ontario Mental Health (ECHO ONMH). METHODS: Ontario health administrative data for fiscal year 2014 was used to compare the characteristics of ECHO ONMH-adopting PCOs, and their patients, to nonadopter organizations (N = 280 vs. N = 273 physicians). RESULTS: ECHO-adopting PCOs did not differ with respect to physician age or years of practice, although PCOs with more female physicians were somewhat more likely to participate. ECHO ONMH adoption was more likely in regions with lower psychiatrist supply, among PCOs using partial salary payment models, and those with a greater interprofessional complement. Patients of ECHO-adopters did not differ on the basis of gender or health care utilization (physical or mental health); however, ECHO-adopting PCOs tended to have patients with less psychiatric comorbidity. CONCLUSION: Models such as Project ECHO, which deliver CPD to primary care, are advanced to address lack of access to specialist health care. These findings support the use of administrative health data to assess the implementation, spread, and impact of CPD.

Technology-Enabled Collaborative Care for Concurrent Diabetes and Distress Management During the COVID-19 Pandemic: Protocol for a Mixed Methods Feasibility Study.

BACKGROUND: The COVID-19 pandemic disrupted the delivery of diabetes care and worsened mental health among many patients with type 2 diabetes (T2D). This disruption puts patients with T2D at risk for poor diabetes outcomes, especially those who experience social disadvantage due to socioeconomic class, rurality, or ethnicity. The appropriate use of communication technology could reduce these gaps in diabetes care created by the pandemic and also provide support for psychological distress. OBJECTIVE: The purpose of this study is to test the feasibility of an innovative co-designed Technology-Enabled Collaborative Care (TECC) model for diabetes management and mental health support among adults with T2D. METHODS: We will recruit 30 adults with T2D residing in Ontario, Canada, to participate in our sequential explanatory mixed methods study. They will participate in 8 weekly web-based health coaching sessions with a registered nurse, who is a certified diabetes educator, who will be supported by a digital care team (ie, a peer mentor, an addictions specialist, a dietitian, a psychiatrist, and a psychotherapist). Assessments will be completed at baseline, 4 weeks, and 8 weeks, with a 12-week follow-up. Our primary outcome is the feasibility and acceptability of the intervention, as evident by the participant recruitment and retention rates. Key secondary outcomes include assessment completion and delivery of the intervention. Exploratory outcomes consist of changes in mental health, substance use, and physical health behaviors. Stakeholder experience and satisfaction will be explored through a qualitative descriptive study using one-on-one interviews. RESULTS: This paper describes the protocol of the study. The recruitment commenced in June 2021. This study was registered on October 29, 2020, on ClinicalTrials.gov (Registry ID: NCT04607915). As of June 2022, all participants have been recruited. It is anticipated that data analysis will be complete by the end of 2022, with study findings available by the end of 2023. CONCLUSIONS: The development of an innovative, technology-enabled model will provide necessary support for individuals living with T2D and mental health challenges. This TECC program will determine the feasibility of TECC for patients with T2D and mental health issues. TRIAL REGISTRATION: ClinicalTrials.gov NCT04607915; https://clinicaltrials.gov/ct2/show/NCT04607915. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/39724.

A Telehealth and Telepsychiatry Economic Cost Analysis Framework: Scoping Review.

Introduction: Despite a good evidence base for telepsychiatry (TP), economic cost analyses are infrequent and vary in quality. Methods: A scoping review was conducted based on the research question, "From the perspective of an economic cost analysis for telehealth and telepsychiatry, what are the most meaningful ways to ensure a study/intervention improved clinical care, provided value to participants, had population level impact, and is sustainable?" The search in seven databases focused on keywords in four concept areas: (1) economic cost analysis, (2) evaluation, (3) telehealth and telepsychiatry, and (4) quantifiable health status outcomes. The authors reviewed the full-text articles based on the inclusion (Medical Subject Headings [MeSH] of the keywords) and exclusion criteria. Results: Of a total of 2,585 potential references, a total of 99 articles met the inclusion criteria. The evaluation of telehealth and TP has focused on access, quality, patient outcomes, feasibility, effectiveness, outcomes, and cost. Cost-effectiveness, cost-benefit, and other analytic models are more common with telehealth than TP studies, and these studies show favorable clinical, quality of life, and economic impact. A standard framework for economic cost analysis should include: an economist for planning, implementation, and evaluation; a tool kit or guideline; comprehensive analysis (e.g., cost-effectiveness or cost-benefit) with an incremental cost-effectiveness ratio; measures for health, quality of life, and utility outcomes for populations; methods to convert outcomes into economic benefits (e.g., monetary, quality of adjusted life year); broad perspective (e.g., societal perspective); sensitivity analysis for uncertainty in modeling; and adjustments for differential timing (e.g., discounting and future costs). Conclusions: Technology assessment and economic cost analysis-such as effectiveness and implementation science approaches-contribute to clinical, training, research, and other organizational missions. More research is needed with a framework that enables comparisons across studies and meta-analyses.

Interprofessional continuing professional development programs can foster lifelong learning in healthcare professionals: experiences from the Project ECHO model.

BACKGROUND: The success of continuing professional development (CPD) programs that foster skills in lifelong learning (LLL) has been well established. However, healthcare professionals often report barriers such as access to CPD and cost which limit uptake. Further research is required to assess how accessible CPD programs, such as those delivered virtually, impact orientation to LLL. Project Extension for Community Healthcare Outcomes (Project ECHO®) is a CPD model that has a growing body of evidence demonstrating improvements in knowledge and skills. Central to this model is the use of a virtual platform, varied teaching approaches, the promotion of multi-directional learning and provider support through a community of practice. This study aimed to explore whether participation in a provincial mental health ECHO program had an effect on interprofessional healthcare providers' orientation to LLL. METHODS: Using a pre-post design, orientation to LLL was measured using the Jefferson Scale of Lifelong Learning. Eligible participants were healthcare professionals enrolled in a cycle of ECHO Ontario Mental Health from 2017 to 2020. Participants were classified as 'high' or 'low' users using median session attendance as a cut-point. RESULTS: The results demonstrate an increase in orientation to LLL following program participation (Pre: 44.64 ± 5.57 vs. Post: 45.94 ± 5.70, t (66) = - 3.023, p < .01, Cohen's d = 0.37), with high ECHO users demonstrating greater orientation to LLL post-ECHO. CONCLUSION: Findings are discussed in the context of self-determination theory and suggest there may be components of CPD programs that more readily support increased motivation for LLL for interprofessional healthcare professionals.

Describing implementation outcomes for a virtual community of practice: The ECHO Ontario Mental Health experience.

BACKGROUND: Project ECHO is a virtual education model aimed at building capacity among healthcare providers to support optimal management for a range of health conditions. The expansion of the ECHO model, further amplified by the pandemic, has demonstrated an increased need to evaluate implementation success to ensure that interventions are implemented as planned. This study describes how Proctor et al.'s implementation outcomes (acceptability, adoption, appropriateness, costs, feasibility, fidelity, penetration, and sustainability) were adapted and used to assess the implementation of ECHO Ontario Mental Health (ECHO-ONMH), a mental health-focused capacity-building programme. METHODS: Using Proctor et al.'s implementation outcomes, the authors developed an implementation outcomes framework for ECHO-ONMH more generally. Using this, outcome measures and success thresholds were identified for each outcome for the ECHO-ONMH context, and then applied to evaluate the implementation of ECHO-ONMH using data from the first 4 years of the programme. RESULTS: An ECHO-ONMH implementation outcomes framework was developed using Proctor's implementation outcomes. ECHO-ONMH adapted implementation outcomes suggest that ECHO-ONMH was implemented successfully in all domains except for penetration, which only had participation from 13/14 regions. Acceptability, appropriateness and adoption success thresholds were surpassed for all 4 years, showing strong signs of sustainability. The programme was deemed feasible all 4 years and was found to be more cost-effective. ECHO-ONMH also showed high rates of fidelity to the ECHO model, and high rates of penetration. CONCLUSIONS: This is the first study to use Proctor et al.'s implementation outcomes to describe implementation success for a virtual capacity-building model. The proposed ECHO implementation outcomes framework provides a base for similar interventions to evaluate implementation success, which is an important precursor to understanding learning, service or health outcomes related to the model. Additionally, these findings can act as a benchmark for other international ECHOs and educational programmes.

The Role of Tele-Education in Advancing Mental Health Quality of Care: A Content Analysis of Project ECHO Recommendations.

Introduction: Project Extension for Community Healthcare Outcomes (Project ECHO®) is a global-guided practice initiative aimed at building primary care capacity and improving health care quality for underserved populations. This tele-education model brings together primary care providers and subject-matter specialists in online communities of practice to share knowledge, discuss complexities in patient care, and collaborate to reduce health disparities. Methods: Using co-generated clinical care recommendations from ECHO Ontario Mental Health, a mental health focused ECHO program, we explored alignment of recommendations across the Institute of Medicine's (IOM) six domains of health care quality to characterize its impact. A total of 417 recommendations, made for 32 patient cases, were analyzed using a modified directed content analysis method. Each recommendation was coded with one or multiple codes, representing each of the six IOM domains. Key examples of recommendations within each domain are described. Results: An average of 13 recommendations were generated per patient case. The effective domain occurred at least once in each complete set of patient care recommendations. The next highest occurring domain was safe (71.9%), followed by patient-centered (68.8%), efficient (40.6%), equitable (18.8%), and timely (12.5%). Recommendation distribution across the entire data set was effective (97.8%), safe (15.6%), patient-centered (12.0%), efficient (3.6%), equitable (1.9%), and timely (1.4%). Discussion: As the first study to characterize ECHO's impact using health care quality domains, the study highlights ECHO's significant focus on effective, safe, and patient-centered care. These findings can inform ways for ECHO to target quality improvement and measure impact in additional health care quality domains, such as efficient, equitable, and timely.

Building Mental Health Capacity: Exploring the Role of Adaptive Expertise in the ECHO Virtual Learning Model.

INTRODUCTION: With the proliferation of virtual learning programs during the COVID-19 pandemic, there is increased need to understand learner experiences and impact on developing expertise. Project Extension for Community Healthcare Outcomes (Project ECHO®) is an established hub-and-spoke tele-education model aimed at building capacity and expertise in primary care providers. Our qualitative study explored how learning experiences within an ECHO mental health care program supported provider learning and ability to solve complex clinical problems. METHODS: We sampled ECHO sessions across a 34-week cycle and analyzed audio transcribed data. Two individuals coded participant interactions during 2-hour recorded sessions using an iterative, constant comparative methodology. RESULTS: The authors identified four key mechanisms of learning in ECHO: (1) fostering participants' productive struggle with cases, (2) development of an integrated understanding, (3) collaborative reformulation of cases, and (4) generation of conceptual solutions based on a new understanding. Throughout the ECHO sessions, learning was observed to be multidirectional from both the hub-to-spoke and between spoke sites. DISCUSSION: Despite the widespread implementation of Project ECHO and other virtual learning models, a paucity of research has focused on mechanisms of virtual learning within these models. Our study demonstrated a bidirectional exchange of knowledge between hub specialist teams and primary care provider spokes that aligned with the development of adaptive expertise through specific learning experiences in Project ECHO. Moreover, the ECHO structure may further support the development of adaptive expertise to better prepare participants to address patients' complex mental health needs.

Few guidelines offer recommendations on how to assess and manage anxiety and distress in children with cancer: a content analysis.

PURPOSE: To (1) describe and compare, across all eligible guidelines, recommendations that address any aspect of clinical assessment or management of anxiety and distress experienced by children and adolescents undergoing cancer treatment or hematopoietic stem cell transplantation (HSCT), and (2) assess guideline characteristics that influence identified recommendations. METHODS: We searched five databases for relevant guidelines and conducted a grey literature search. Guidelines had to refer to children 0-18 years old who were undergoing cancer treatment or HSCT, describe any aspect of clinical assessment or management of symptoms of anxiety and distress, and be publicly accessible and published in English on or after 2000. RESULTS: We identified 118 guidelines on pediatric cancer of which 13 mentioned clinical assessment or management of anxiety and distress. Six contained ≥ 1 recommendation addressing assessments of symptoms of which only two recommended specific screening instruments. Ten contained ≥ 1 recommendation addressing interventions for symptoms, of which six described specific interventions such as distraction and medication. Psychologists and nurses were the most common panel members and three guideline panels included a patient advocate. Only two guidelines received overall quality ratings > 80.0%. CONCLUSION: We identified no guidelines that were specific to clinical assessment or management of anxiety and distress among children and adolescents undergoing cancer treatment or HSCT, and thus, clinicians lack evidence-informed guidance on how to manage these specific symptoms. Future research should establish high-quality guidelines that offer recommendations specific to clinical assessment and management of anxiety and distress in pediatric oncology and HSCT.

Characterizing Family Physicians Who Refer to Telepsychiatry in Ontario.

INTRODUCTION: Telepsychiatry can improve access to psychiatric services for those who otherwise cannot easily access care. Family physicians are gatekeepers to specialized care in Ontario, so it is essential to understand predictors relating to referrals to telepsychiatry to better plan services and increase telepsychiatry adoption. METHODS: This study used an annual retrospective cross-sectional study design to compare physicians who referred their patients to telepsychiatry each year from fiscal year (FY) 2008 to FY 2016. A 1-year (FY 2016) comparison of family physicians who referred to telepsychiatry (FPTs) compared to family physicians who did not refer to telepsychiatry (FPNTs) matched (1:2) by region was also conducted. Finally, we used statistical modeling to understand the predictors of referring to telepsychiatry among physicians. RESULTS: Between FY 2008 and FY 2016, the number of patients receiving telepsychiatry increased from 925 visits to 13,825 visits. Thirty-two percent of Ontario primary care physicians referred to telepsychiatry in 2016. Several characteristics were notably different between FPTs and FPNTs: FPTs were more likely to be from a residence with less than 10,000 people, to have more nurse practitioners in the practice, and to be from a family health team than FPNTs. Rostered patients of FPTs were more likely to reside in rural areas, have more clinical complexity, and to utilize more mental health services compared to FPNTs. CONCLUSIONS: There has been an increase in the use of telepsychiatry by patients and family physicians over the study period, although there remains opportunity for significant growth. Family physicians who live in rural areas, are part of an FHT, have more NPs, with more rural and complex patients were more likely to refer to telepsychiatry. As recent pro-telemedicine policies support the growth of telepsychiatry, this study will serve as an important baseline.

Client Satisfaction and Experience With Telepsychiatry: Development and Validation of a Survey Using Clinical Quality Domains.

BACKGROUND: Telepsychiatry is an increasingly used model of mental health care that connects patients with psychiatrists at a distance via videoconference. Telepsychiatry is an effective clinical intervention that improves access to quality care in regions with limited resources or in clinical situations where in-person care is unavailable. OBJECTIVE: This study aims to develop a validated survey tool to measure patient experience and satisfaction with telepsychiatry based on the quality of care domains. This study also seeks to understand which health service outcomes were most strongly correlated with overall satisfaction in the context of telepsychiatry. METHODS: The survey created in this study was developed and validated with a panel of subject matter and process experts and was piloted with 274 patients who received clinical consultations through the TeleMental Health Program at the Centre for Addiction and Mental Health. Factor analysis was used to determine correlations between questions and quality of care domains and was also used to assess model fit. RESULTS: The study provides a validated survey to measure patient satisfaction and experience with telepsychiatry across 4 domains: access and timeliness, appropriateness, effectiveness, and safety. Both safety and access and timeliness were found to be statistically significant predictors of satisfaction in our sample. CONCLUSIONS: By situating patient satisfaction and experience within this framework, the survey facilitates patient data collection and interpretation through a clinical quality lens.

Responding to Health Care Professionals' Mental Health Needs During COVID-19 Through the Rapid Implementation of Project ECHO.

INTRODUCTION: The COVID-19 pandemic can cause significant mental health distress among health care professionals (HCPs). We describe the psychological needs of HCPs during COVID-19 and the implementation of Project Extension for Community Healthcare Outcomes (ECHO) Coping with COVID (ECHO-CWC) to help HCPs manage COVID-19 distress. METHODS: We used an established rapid implementation approach to accelerate the development and delivery of ECHO-CWC to address the emerging needs of HCPs. Participants' needs were identified using a 10-question survey of participants' perceived risk of COVID-19 and a five-item self-efficacy measure. Implementation outcomes consisted of participant engagement and session satisfaction scores using a five-point Likert scale. RESULTS: A total of 426 participants registered for ECHO-CWC. Needs assessment data (n = 129) showed that most participants reported feeling increased stress at work (84.5%), fear of infecting others (75.2%), and fear of falling ill (70.5%) from COVID-19, yet most participants accepted the risk associated with work during this time (59.7%). Participants were highly satisfied with the initial five sessions (mean = 4.26). DISCUSSION: HCPs reported the greatest concern with fears of infection and infecting others during the acute phase of the pandemic. Using an iterative curriculum design approach and existing implementation frameworks, the ECHO tele-education model can be rapidly mobilized to address HCPs' mental health needs during the COVID-19 pandemic.

Early Psychosis Intervention-Spreading Evidence-based Treatment (EPI-SET): protocol for an effectiveness-implementation study of a structured model of care for psychosis in youth and emerging adults.

INTRODUCTION: While early psychosis intervention (EPI) has proliferated in recent years amid evidence of its effectiveness, programmes often struggle to deliver consistent, recovery-based care. NAVIGATE is a manualised model of EPI with demonstrated effectiveness consisting of four components: individualised medication management, individual resiliency training, supported employment and education and family education. We aim to implement NAVIGATE in geographically diverse EPI programmes in Ontario, Canada, evaluating implementation and its effect on fidelity to the EPI model, as well as individual-level outcomes (patient/family member-reported and interviewer-rated), system-level outcomes (captured in provincial administrative databases) and engagement of participants with lived experience. METHODS AND ANALYSIS: This is a multisite, non-randomised pragmatic hybrid effectiveness-implementation type III mixed methods study coordinated at the Centre for Addiction and Mental Health (CAMH) in Toronto. Implementation is supported by the Provincial System Support Program, a CAMH-based programme with provincial offices across Ontario, and Extension of Community Healthcare Outcomes Ontario Mental Health at CAMH and the University of Toronto. The primary outcome is fidelity to the EPI model as measured using the First Episode Psychosis Services-Fidelity Scale. Four hundred participants in the EPI programmes will be recruited and followed using both individual-level assessments and health administrative data for 2 years following NAVIGATE initiation. People with lived experience will be engaged in all aspects of the project, including through youth and family advisory committees. ETHICS AND DISSEMINATION: Research ethics board approval has been obtained from CAMH and institutions overseeing the local EPI programmes. Study findings will be reported in scientific journal articles and shared with key stakeholders including youth, family members, programme staff and policymakers. TRIAL REGISTRATION NUMBER: NCT03919760; Pre-results.

Digital Health Equity and COVID-19: The Innovation Curve Cannot Reinforce the Social Gradient of Health.

Digital health innovations have been rapidly implemented and scaled to provide solutions to health delivery challenges posed by the coronavirus disease (COVID-19) pandemic. This has provided people with ongoing access to vital health services while minimizing their potential exposure to infection and allowing them to maintain social distancing. However, these solutions may have unintended consequences for health equity. Poverty, lack of access to digital health, poor engagement with digital health for some communities, and barriers to digital health literacy are some factors that can contribute to poor health outcomes. We present the Digital Health Equity Framework, which can be used to consider health equity factors. Along with person-centered care, digital health equity should be incorporated into health provider training and should be championed at the individual, institutional, and social levels. Important future directions will be to develop measurement-based approaches to digital health equity and to use these findings to further validate and refine this model.

A cost analysis comparing telepsychiatry to in-person psychiatric outreach and patient travel reimbursement in Northern Ontario communities.

INTRODUCTION: Residents of Northern Ontario have limited access to local psychiatric care. To address this, three program models exist: (1) telepsychiatry; (2) psychiatrists traveling to underserved areas; and (3) reimbursing patients for travel to a psychiatrist. Evidence shows that telepsychiatry has comparable outcomes to in-person consultations. The objective of this study was to determine the cost difference between programs. METHODS: A cost-minimization analysis estimating cost per visit from a public healthcare payer economic costing perspective was conducted. Data on fixed and variable costs were obtained. Evidence-based assumptions were made where relevant. Base-case scenarios and a break-even analysis were completed, as well as deterministic and probabilistic sensitivity analyses, to explore the effects of parameter variability on program costs. RESULTS: Costs per visit were lowest in telepsychiatry (CAD$360) followed by traveling physicians (CAD$558) and patient reimbursement (CAD$620). Among the 100,000 Monte Carlo simulations, results showed telepsychiatry was the least costly program in 71.2% of the simulations, while the reimbursement and outreach programs were least costly in 15.1% and 13.7% of simulations, respectively. The break-even analysis found telepsychiatry was the least costly program after an annual patient visit threshold of approximately 76 visits (compared to traveling psychiatrists) and 126 visits (compared to reimbursed patients). DISCUSSION: Our analyses support telepsychiatry as the least costly program. These results have important implications for program planning, including the prioritization of telepsychiatry, increased integration of telepsychiatry with other modalities of outreach psychiatry, and limiting use of the patient remuneration program to where medically necessary, to reduce overall cost.

Implementation of Off-Site Integrated Care for Children: A Scoping Review.

BACKGROUND: As an alternative to co-located integrated care, off-site integration (partnerships between primary care and non-embedded specialty mental health providers) can address the growing need for pediatric mental health services. Our goal is to review the existing literature on implementing off-site pediatric integrated care. METHODS: We systematically searched the literature for peer-reviewed publications on off-site pediatric integrated care interventions. We included studies that involved systematic data collection and analysis, both qualitative and quantitative, of implementation outcomes (acceptability, adoption, appropriateness, feasibility, fidelity, implementation cost, penetration, and sustainability). RESULTS: We found 39 original articles from 24 off-site programs with a variety of study designs, most with secondary implementation outcomes. Models of off-site integration varied primarily along two dimensions: direct vs. indirect, and in-person vs. remote. Overall, off-site models were acceptable to providers, particularly when the following were present: strong interdisciplinary communication, timely availability and reliability of services, additional support beyond one-time consultation, and standardized care algorithms. Adoption and penetration were facilitated by enhanced program visibility, including on-site champions. Certain clinical populations (e.g., school-age, less complicated ADHD) seemed more amenable to off-site integrated models than others (e.g., preschool-age, conduct disorders). Lack of funding and inadequate reimbursement limited sustainability in all models. CONCLUSIONS: Off-site interventions are feasible, acceptable, and often adopted widely with adequate planning, administrative support, and interprofessional communication. Studies that focus primarily on implementation and that consider the perspectives of specialty providers and patients are needed.

Adapting the Consolidated Framework for Implementation Research to Create Organizational Readiness and Implementation Tools for Project ECHO.

The Project Extension for Community Healthcare Outcomes (ECHO) model expands primary care provider (PCP) capacity to manage complex diseases by sharing knowledge, disseminating best practices, and building a community of practice. The model has expanded rapidly, with over 140 ECHO projects currently established globally. We have used validated implementation frameworks, such as Damschroder's (2009) Consolidated Framework for Implementation Research (CFIR) and Proctor's (2011) taxonomy of implementation outcomes, combined with implementation experience to (1) create a set of questions to assess organizational readiness and suitability of the ECHO model and (2) provide those who have determined ECHO is the correct model with a checklist to support successful implementation. A set of considerations was created, which adapted and consolidated CFIR constructs to create ECHO-specific organizational readiness questions, as well as a process guide for implementation. Each consideration was mapped onto Proctor's (2011) implementation outcomes, and questions relating to the constructs were developed and reviewed for clarity. The Preimplementation list included 20 questions; most questions fall within Proctor's (2001) implementation outcome domains of "Appropriateness" and "Acceptability." The Process Checklist is a 26-item checklist to help launch an ECHO project; items map onto the constructs of Planning, Engaging, Executing, Reflecting, and Evaluating. Given that fidelity to the ECHO model is associated with robust outcomes, effective implementation is critical. These tools will enable programs to work through key considerations to implement a successful Project ECHO. Next steps will include validation with a diverse sample of ECHO projects.

Building Provincial Mental Health Capacity in Primary Care: An Evaluation of a Project ECHO Mental Health Program.

OBJECTIVE: Project Extension for Community Healthcare Outcomes (Project ECHO©) addresses urban-rural disparities in access to specialist care by building primary care provider (PCP) capacity through tele-education. Evidence supporting the use of this model for mental health care is limited. Therefore, this study evaluated a mental health and addictions-focused ECHO program. Primary outcome measures were PCP knowledge and perceived self-efficacy. Secondary objectives included: satisfaction, engagement, and sense of professional isolation. PCP knowledge and self-efficacy were hypothesized to improve with participation. METHODS: Using Moore's evaluation framework, we evaluated the ECHO program on participant engagement, satisfaction, learning, and competence. A pre-post design and weekly questionnaires measured primary and secondary outcomes, respectively. RESULTS: Knowledge test performance and self-efficacy ratings improved post-ECHO (knowledge change was significant, p < 0.001, d = 1.13; self-efficacy approached significance; p = 0.056, d = 0.57). Attrition rate was low (7.7%) and satisfaction ratings were high across all domains, with spokes reporting reduced feelings of isolation. DISCUSSION: This is the first study to report objective mental health outcomes related to Project ECHO. The results indicate high-participant retention is achievable, and provide preliminary evidence for increased knowledge and self-efficacy. These findings suggest this intervention may improve mental health management in primary care.

Implementation and Utilisation of Telepsychiatry in Ontario: A Population-Based Study.

OBJECTIVE: Rural areas in Ontario have fewer psychiatrists, making access to specialist mental health care challenging. Our objective was to characterise psychiatrists delivering and patients receiving telepsychiatry in Ontario and to determine the number of patients who accessed a psychiatrist via telepsychiatry following discharge from psychiatric hospitalisation. METHOD: We conducted a serial panel study to evaluate the characteristics of psychiatrists providing telepsychiatry from April 2007 to March 2013. In addition, we conducted a cross-sectional study for fiscal year 2012-2013 to examine telepsychiatry patient characteristics and create an in-need patient cohort of individuals with a recent psychiatric hospitalisation that assessed if they had follow-up with a psychiatrist in person or through telepsychiatry within 1 year of discharge. RESULTS: In fiscal year 2012-2013, a total of 3801 people had 5635 telepsychiatry visits, and 7% ( n = 138) of Ontario psychiatrists provided telepsychiatry. Of the 48,381 people identified as in need of psychiatric care, 60% saw a local psychiatrist, 39% saw no psychiatrist, and less than 1% saw a psychiatrist through telepsychiatry only or telepsychiatry in addition to local psychiatry within a year. Three northern regions had more than 50% of in-need patients fail to access psychiatry within 1 year. CONCLUSIONS: Currently, relatively few patients and psychiatrists use telepsychiatry. In addition, patients scarcely access telepsychiatry for posthospitalisation follow-up. This study, which serves as a preliminary baseline for telepsychiatry in Ontario, demonstrates that telepsychiatry has not evolved systematically to address need and highlights the importance of system-level planning when implementing telepsychiatry to optimise access to care.

The Impact of Project ECHO on Participant and Patient Outcomes: A Systematic Review.

PURPOSE: Project Extension for Community Healthcare Outcomes (ECHO) uses tele-education to bridge knowledge gaps between specialists at academic health centers and primary care providers from remote areas. It has been implemented to address multiple medical conditions. The authors examined evidence of the impact of all Project ECHO programs on participant and patient outcomes. METHOD: The authors searched PubMed, MEDLINE, EMBASE, PsycINFO, and ProQuest from January 2000 to August 2015 and the reference lists of identified reviews. Included studies were limited to those published in English, peer-reviewed articles or indexed abstracts, and those that primarily focused on Project ECHO. Editorials, commentaries, gray literature, and non-peer-reviewed articles were excluded. The authors used Moore's evaluation framework to organize study outcomes for quality assessment. RESULTS: The authors identified 39 studies describing Project ECHO's involvement in addressing 17 medical conditions. Evaluations of Project ECHO programs generally were limited to outcomes from Levels 1 (number of participants) to 4 (providers' competence) of Moore's framework (n = 22 studies, with some containing data from multiple levels). Studies also suggested that Project ECHO changed provider behavior (n = 1), changed patient outcomes (n = 6), and can be cost-effective (n = 2). CONCLUSIONS: Project ECHO is an effective and potentially cost-saving model that increases participant knowledge and patient access to health care in remote locations, but further research examining its efficacy is needed. Identifying and addressing potential barriers to Project ECHO's implementation will support the dissemination of this model as an education and practice improvement initiative.