Performance Change in Treating Tobacco Addiction: An Online, Interprofessional, Facilitated Continuing Education Course (TEACH) Evaluation at Moore's Level 5.

INTRODUCTION: Continuing education is essential to build capacity among health care providers (HCPs) to treat people with tobacco addiction. Online, interprofessional training programs are valuable; however, interpretation and comparison of outcomes remain challenging because of inconsistent use of evaluation frameworks. In this study, we used level 5 of Moore's evaluation framework to examine whether an online training program in intensive tobacco cessation counseling achieved sustained performance change among HCPs across multiple health disciplines. METHODS: The evaluation sample included 62 HCPs with direct clinical duties, who completed the online Training Enhancement in Applied Counseling and Health (TEACH) Core Course in 2015 and 2016. We compared self-reported changes in cessation counseling and clinical practices across eight core competencies from baseline to 6-month follow-up using McNemar's tests and descriptive analyses. RESULTS: Compared with baseline, significantly more HCPs reported providing cessation counseling at 6-month follow-up (44% versus 81%, P < .001). HCPs also reported significant increases in engagement in six of the eight core competencies. DISCUSSION: Online training in intensive tobacco cessation treatment can result in sustained performance improvement at 6 months. However, availability of resources and clinical context may influence the extent to which HCPs are able to implement their learned skills. Furthermore, continuing education programs should consider the use of consistent evaluation frameworks to promote cross program comparisons.

A Qualitative Evaluation of an Online Expert-Facilitated Course on Tobacco Dependence Treatment.

Qualitative evaluations of courses prove difficult due to low response rates. Online courses may permit the analysis of qualitative feedback provided by health care providers (HCPs) during and after the course is completed. This study describes the use of qualitative methods for an online continuing medical education (CME) course through the analysis of HCP feedback for the purpose of quality improvement. We used formative and summative feedback from HCPs about their self-reported experiences of completing an online expert-facilitated course on tobacco dependence treatment (the Training Enhancement in Applied Cessation Counselling and Health [TEACH] Project). Phenomenological, inductive, and deductive approaches were applied to develop themes. QSR NVivo 11 was used to analyze the themes derived from free-text comments and responses to open-ended questions. A total of 277 out of 287 participants (96.5%) completed the course evaluations and provided 690 comments focused on how to improve the program. Five themes emerged from the formative evaluations: overall quality, content, delivery method, support, and time. The majority of comments (22.6%) in the formative evaluation expressed satisfaction with overall course quality. Suggestions for improvement were mostly for course content and delivery method (20.4% and 17.8%, respectively). Five themes emerged from the summative evaluation: feedback related to learning objectives, interprofessional collaboration, future topics of relevance, overall modifications, and overall satisfaction. Comments on course content, website function, timing, and support were the identified areas for improvement. This study provides a model to evaluate the effectiveness of online educational interventions. Significantly, this constructive approach to evaluation allows CME providers to take rapid corrective action.

Evaluation of Web-Based Continuing Professional Development Courses: Aggregate Mixed-Methods Model.

BACKGROUND: Many continuing professional development (CPD) Web-based programs are not explicit about underlying theory and fail to demonstrate impact. OBJECTIVE: The aim of this study was to develop and apply an aggregate mixed-methods evaluation model to describe the paradigm, theoretical framework, and methodological approaches used to evaluate a CPD course in tobacco dependence treatment, the Training Enhancement in Applied Cessation Counseling and Health (TEACH) project. METHODS: We evaluated the effectiveness of the 5-week TEACH Web-based Core Course in October 2015. The model of evaluation was derived using a critical realist lens to incorporate a dimension of utilitarian to intuitionist approaches. In addition, we mapped our findings to models described by Fitzpatrick et al, Moore et al, and Kirkpatrick. We used inductive and deductive approaches for thematic analysis of qualitative feedback and dependent samples t tests for quantitative analysis. RESULTS: A total of 59 participants registered for the course, and 48/59 participants (81%) completed all course requirements. Quantitative analysis indicated that TEACH participants reported (1) high ratings (4.55/5, where 5=best/excellent) for instructional content and overall satisfaction of the course (expertise and consumer-oriented approach), (2) a significant increase (P ˂.001) in knowledge and skills (objective-oriented approach), and (3) high motivation (78.90% of participants) to change and sustain practice change (management-oriented approach). Through the intuitionist lens, inductive and deductive qualitative thematic analysis highlighted three central themes focused on (1) knowledge acquisition, (2) recommendations to enhance learning for future participants, and (3) plans for practice change in the formative assessment, and five major themes emerged from the summative assessment: (1) learning objectives, (2) interprofessional collaboration, (3) future topics of relevance, (4) overall modification, and (5) overall satisfaction. CONCLUSIONS: In the current aggregate model to evaluate CPD Web-based training, evaluators have been influenced by different paradigms, theoretical lenses, methodological approaches, and data collection methods to address and respond to different needs of stakeholders impacted by the training outcomes.

Review and evaluation of online tobacco dependence treatment training programs for health care practitioners.

BACKGROUND: Training health care professionals is associated with increased capacity to deliver evidence-based smoking cessation interventions and increased quit rates among their patients. Online training programs hold promise to provide training but questions remain regarding the quality and usability of available programs. OBJECTIVE: The aim was to assess the quality of English-language online courses in tobacco dependence treatment using a validated instrument. METHODS: An environmental scan was conducted using the Google search engine to identify available online tobacco dependence treatment courses. The identified courses were then evaluated using the Peer Review Rubric for Online Learning, which was selected based on its ability to evaluate instructional design. It also has clear and concise criteria descriptions to ensure uniformity of evaluations by trained experts. RESULTS: A total of 39 courses were identified, of which 24 unique courses were assessed based on their accessibility and functionality during the period of evaluation. Overall, the course ratings indicated that 17 of 24 courses evaluated failed to meet minimal quality standards and none of the courses evaluated could be ranked as superior. However, many excelled in providing effective navigation, course rationale, and content. Many were weak in the use of instructional design elements, such as teaching effectiveness, learning strategies, instructor's role, and assessment and evaluation. Evaluation results and suggestions for improvement were shared with course administrators. CONCLUSIONS: Based on the courses evaluated in this review, course developers are encouraged to employ best practices in instructional design, such as cohesiveness of material, linearity of design, practice exercises, problem solving, and ongoing evaluation to improve existing courses and in the design of new online learning opportunities.