Twelve tips for Natural Language Processing in medical education program evaluation.

With the increasing application of Natural Language Processing (NLP) in Medicine at large, medical educators are urged to gain an understanding and implement NLP techniques within their own education programs to improve the workflow and make significant and rapid improvements in their programs. This paper aims to provide twelve essential tips inclusive of both conceptual and technical factors to facilitate the successful integration of NLP in medical education program evaluation. These twelve tips range from advising on various stages of planning the evaluation process, considerations for data collection, and reflections on preprocessing of data in preparation for analysis and interpretation of results. Using these twelve tips as a framework, medical researchers, educators, and administrators will have an understanding and reference to navigating applications of NLP and be able to unlock its potential for enhancing the evaluation of their own medical education programs.

The Power of Partnerships to Identify and Address Mental Health Concerns Experienced by Adults With Intellectual and Developmental Disabilities.

There are significant research gaps with regard to understanding and addressing the mental health concerns of adults with intellectual and developmental disabilities (IDD) and their families. In this article, we reflect on research we have carried out about mental health and IDD prior to and during the pandemic in Ontario, Canada. We aim to address how partnering with people with IDD, family caregivers, service providers, and policy makers can help accelerate needed progress in this area. We conclude with some lessons learned during the pandemic about what to emphasize in building and maintaining such partnerships.

Implementation of structured feedback in a psychiatry residency program in Canada: a qualitative analysis study.

Introduction: Structured feedback is important to support learner progression in competency-based medical education (CBME). R2C2 is an evidence-based four-phased feedback model that has been studied in a range of learner contexts; however, data on factors influencing implementation of this model are lacking. This pilot study describes implementation of the R2C2 model in a psychiatry CBME residency program, using the Consolidated Framework for Implementation Research (CFIR). Methods: The study was carried out in three phases: planning, implementation and evaluation. After receiving training, 15 supervisors used the R2C2 feedback model with residents. Semi-structured interviews explored (n = 10) supervisors' experience of the model. CFIR was used to identify factors that influence implementation of the R2C2 model when providing feedback to residents. Results: Qualitative data analysis revealed four key themes: Perceptions about the R2C2 model, Facilitators and barriers to its implementation, Fidelity to R2C2 model and Intersectionality related to the feedback. The CFIR implementation domains provided structure to the themes and subthemes. Conclusion: The R2C2 model is a helpful tool to provide structured feedback. Structure of the model, self-efficacy, in-house educational expertise, learning culture, organizational readiness, and training support are important facilitators of implementation. Further studies are needed to explore the learner's perspective and fidelity of this model.

A systematic review of intellectual and developmental disability curriculum in international pre-graduate health professional education.

BACKGROUND: Despite the increasing global population of individuals with intellectual and developmental disabilities (IDD), this population remains especially vulnerable to health disparities through several factors such as a lack of access to sufficient medical care and poor determinants of health. To add, numerous studies have shown that healthcare professionals are still insufficiently prepared to support this population of patients. This review synthesizes the literature on current pre-graduate IDD training programs across healthcare professions with the goal of informing the creation of evidence-based curricula. METHODS: Four major databases were searched for current pre-graduate IDD training interventions for healthcare professionals. The Preferred Reporting Items for Systematic Reviews and Meta-Analysis flow diagram and the Best Evidence Medical Educations systematic review guide were used to frame our collection and analysis. RESULTS: Of the 8601 studies screened, 32 studies were identified, with most studies involving medical students (50%). Of note, 35% of studies were interprofessional. Most interventions utilized multiple pedagogical methods with a majority including clinical experiences (63%) followed by theoretical teaching (59%). Kirkpatrick levels showed 9% were level 0, 6% were level 1, 31% were level 2A, 31% were level 2B, 19% were level 3, 3% were level 4A, and none were level 4B. CONCLUSIONS: There is a paucity of formally evaluated studies in pre-graduate health professional IDD education. As well, there are a lack of longitudinal learning opportunities and integration into formal curriculum. Strengths identified were the use of multimodal approaches to teaching, including interprofessional approaches to optimize team competencies.

Evolution of a physician wellness, engagement and excellence strategy: lessons learnt in a mental health setting.

OBJECTIVE: This study aims to evaluate the impact of several organisational initiatives implemented as part of a physician engagement, wellness and excellence strategy at a large mental health hospital. Interventions that were examined include: communities of practice, peer support programme, mentorship programme and leadership and management programme for physicians. METHODS: A cross-sectional study, guided by the Reach, Effectiveness/Efficacy, Adoption, Implementation and Maintenance evaluation framework, was conducted with physicians at a large academic mental health hospital in Toronto, Canada. Physicians were invited to complete an online survey in April 2021, which composed of questions on the awareness, use and perceived impact of the organisational wellness initiatives and the two-item Maslach Burnout Inventory tool. The survey was analysed using descriptive statistics and a thematic analysis. RESULTS: 103 survey responses (40.9% response rate) were gathered from physicians, with 39.8% of respondents reporting experiences of burn-out. Overall, there was variable reach and suboptimal use of the organisational interventions reported by physicians. Themes emerging from open-ended questions included the importance of addressing: workload and resource related factors; leadership and culture related factors; and factors related to the electronic medical record and virtual care. CONCLUSIONS: Organisational strategies to address physician burn-out and support physician wellness require repeated evaluation of the impact and relevance of initiatives with physicians, taking into account organisational culture, external variables, emerging barriers to access and participation, and physician needs and interest over time. These findings will be embedded as part of ongoing review of our organisational framework to guide changes to our physician engagement, wellness and excellence strategy.

'More together than apart': The evaluation of a virtual course to improve mental health and well-being of adults with intellectual disabilities during the COVID-19 pandemic.

BACKGROUND: To address the growing concerns over poor mental health experienced by adults with intellectual disabilities due to the COVID-19 pandemic, a national virtual mental health course was delivered and evaluated. METHODS: This mixed methods study utilized both qualitative and quantitative assessments. Participants were 27 adults with intellectual disabilities who participated in the 6-week course. Participants completed measures of self-efficacy and well-being at three time points and qualitative satisfaction measures at post and follow-up. RESULTS: Attendance was high and the course was feasible and acceptable to participants. Positive changes related to mental health self-efficacy were detected (p = .01), though mental well-being did not improve. CONCLUSION: The study provided evidence for the feasibility and value of the course for this population. Future research should examine how virtual courses could support the population in terms of pandemic recovery and how courses may work for individuals who are less independent.

International clinician perspectives on pandemic-associated stress in supporting people with intellectual and developmental disabilities.

BACKGROUND: People living with intellectual and developmental disabilities (IDD) have suffered disproportionately in health outcomes and general well-being during the COVID-19 pandemic. There is emerging evidence of increased psychological distress. Increased strain has also fallen on clinicians managing the psychological needs of people with IDD, in the context of learning new technologies, staff shortages, reduced services and paused training opportunities. AIMS: To examine clinicians' experiences of patient care, clinical management and the impact of care delivery. METHOD: A mixed fixed-response and free-text survey comprising 28 questions covering four areas (responder demographics, clinical practice, changes to local services and clinician experiences) was developed, using the STROBE guidance. It was disseminated through an exponential snowballing technique to clinicians in seven high-income countries. Quantitative data were analysed and presented with Microsoft Excel. Qualitative data were coded and thematically analysed, and presented with in-text quotations. RESULTS: There were 139 respondents, mostly senior physicians (71%). Two-thirds reported over 10 years working in the field. Quantitative findings include increased clinician stress (77%), referrals (53%), patient distress presentations (>70%), patient isolation (73%) and carer burden (89%), and reduced patient participation in daily activities (86%). A third reported increased psychotropic prescribing. Qualitative analysis outlined changes to clinical practice, particularly the emergence and impact of telehealth. CONCLUSIONS: In the countries surveyed, the pandemic has not only had a significant impact on people with IDD, but also their carers and clinicians. A proactive, holistic international response is needed in preparedness for future public health emergencies.

Virtual Education Program to Support Providers Caring for People With Intellectual and Developmental Disabilities During the COVID-19 Pandemic: Rapid Development and Evaluation Study.

BACKGROUND: People with intellectual and developmental disabilities are at increased health-related risk due to the COVID-19 pandemic. Virtual training programs that support providers in caring for the physical and mental health needs of this population, as well provide psychological support to the providers themselves, are needed during the pandemic. OBJECTIVE: This paper describes the design, delivery, and evaluation of a virtual educational COVID-19-focused Extension for Community Healthcare Outcomes program to support providers during the COVID-19 pandemic in caring for the mental health of people with intellectual and developmental disabilities. METHODS: A rapid design thinking approach was used to develop a 6-session program that incorporates mindfulness practice, a wellness check, COVID-19-related research and policy updates, a didactic presentation on a combination mental health and COVID-19 related topic, and a case-based discussion to encourage practical learning. We used the first 5 outcome levels of Moore's evaluation framework-focusing on participation, satisfaction, learning, self-efficacy, and change in practice-which were rated (out of 5) by care providers from health and disability service sectors, as well as additional reflection measures about innovations to the program. Qualitative feedback from open-text responses from participants were analyzed using modified manifest content analysis. RESULTS: A total of 104 care providers from health and disability service sectors participated in the program. High levels of engagement (81 participants per session on average) and satisfaction (overall satisfaction score: mean 4.31, SD 0.17) were observed. Self-efficacy (score improvement: 19.8%), support, and coping improved. Participants also rated the newly developed COVID-19 program and its innovative components highly. Open text feedback showed participants felt that the Extension for Community Healthcare Outcomes program expanded their knowledge and competency and created a sense of being part of a community of practice; provided value for the COVID-19 innovations; supported resource-sharing within and beyond program participants; and facilitated changes to participants' approaches to client care in practice and increased participants' confidence in supporting clients and families. CONCLUSIONS: The Extension for Community Healthcare Outcomes program is an effective model for capacity-building programs with a shared-learning approach. Future iterations should include targeted evaluation of long-term outcomes such as staff burnout.

Post-graduate Medical Training in Intellectual and Developmental Disabilities: a Systematic Review.

OBJECTIVE: Despite the increasing number of people with autism-spectrum disorder (ASD), intellectual disabilities (ID), and developmental disabilities (DDs), individuals with these conditions continue to have high levels of unmet physical and mental health needs. Robust training of health professionals can help bridge this gap. A systematic review was conducted to describe the features and educational outcomes of existing postgraduate medical education curricula to inform the development of future training to address the growing unmet care needs of people with intellectual and developmental disabilities (IDD) such as ASD and ID. METHODS: Four major databases were searched for peer-reviewed, English-language research focusing on post-graduate training in IDD education. Educational curricula and outcomes were summarized including Best Evidence in Medical Education (BEME) Quality of Evidence and Kirkpatrick training evaluation model. RESULTS: Sixteen studies were identified with a majority published after 2000 (69%). Pediatric departments were involved in 69%, Psychiatry 19%, Medicine-Pediatrics 19%, and Family Medicine 6.3%. Analysis of Kirkpatrick outcomes showed 31% were level 1 (satisfaction or comfort); 38% level 2 (change in objective knowledge or skills); 13% level 3 (change in behavior); and none at level 4. BEME analysis showed 19% of studies were grade 1 (no clear conclusions), 31% grade 2 (ambiguous results), and half (50%) grade 3 (conclusions can probably be based on findings), with none scoring four or higher. CONCLUSIONS: There is a paucity of objectively evaluated research in the area. Studies reviewed show clear promise for specialized, interdisciplinary, competency-based education which may be foundational for future curriculum development.

Using Rapid Design Thinking to Overcome COVID-19 Challenges in Medical Education.

The rapid rise of cases of coronavirus disease 2019 (COVID-19) has led to the implementation of public health measures on an unprecedented scale. These measures have significantly affected the training environment and the mental health of health care providers and learners. Design thinking offers creative and innovative solutions to emergent complex problems, including those related to training and patient care that have arisen as a result of the COVID-19 pandemic. Design thinking can accelerate the development and implementation of solution prototypes through a process of inspiration, ideation, and implementation. Digital technology can be leveraged as part of this process to provide care and education in new or enhanced ways. Online knowledge hubs, videoconference-based interactive sessions, virtual simulations, and technology-enhanced coaching for health care providers are potential solutions to address identified issues. Limitations of this model include inherent bias toward utilitarian instead of egalitarian principles and the subsequent threat to diversity, equity, and inclusion in solutions. Although medical educators have embraced digital transformation during the COVID-19 pandemic, there is a need to ensure that these changes are sustained.

"You are in Some Sort of Occupational Adolescence": An Exploratory Study to Understand Knowledge, Skills, and Experiences of CanMEDS Leader Role in Psychiatry.

OBJECTIVE: The study explored knowledge, attitude, and practices of Canadian Medical Education Directions for Specialists (CanMEDS) "'Leader" role in faculty psychiatrists in a university setting. METHODS: This is a qualitative study using a thematic analysis approach. Participants were identified by purposive, convenience, and snowball sampling. Telephone interviews were conducted to explore participants' perceptions of leadership. Qualitative analysis was carried out using a constant comparative analysis approach to identify themes across the interview data. Data was transcribed and coded into themes and categories to form an analysis of physicians' knowledge, attitude, and practices of the CanMEDS "Leader" role. RESULTS: Twenty-eight faculty psychiatrists participated in the study. The following themes and subthemes emerged from analysis: (a) characteristics of leadership and lack of clarity regarding CanMEDS "Leader" role (subthemes: differences between manager and leader, lack of specificity and practicality, relevance to practice) and (b) the meaning of leadership (subthemes: "positional leadership" and "everyday leadership," physician as team leader, developing leadership skills, getting beyond "occupational adolescence"). CONCLUSION: Participants perceived CanMEDS "Leader" role description as a high-level vision, which needs practice-oriented guidance. Participants conceptualized the dual nature of physician leadership at an individual level and at an organizational level. Leadership training is important both in residency as well as lifelong learning.

Clinical, Demographic, and Criminal Behavior Characteristics of Patients With Intellectual Disabilities in a Canadian Forensic Program.

Background: People with intellectual disability (ID) and forensic issues constitute a challenging clinical group that has been understudied in forensic settings. Methods: We assessed the characteristics of patients with ID under the authority of the Ontario Review Board (ORB) in a large forensic program of a tertiary psychiatric hospital (excluding those with a cognitive disorder) and compared their characteristics with those of a non-ID control group. Results: Among 510 adult ORB patients, 47 had an ID diagnosis. ID patients were of younger age at index offense, with a lower level of education, and were less likely to have been married or employed, more likely to have committed a sexual offense, more likely to have a diagnosis of paraphilia, less likely to be "not criminally responsible," and more likely to be "unfit to stand trial." They were also more likely to have committed their index offenses against care professionals and be treated in a secure unit. Conclusion: Our findings have major implications for clinicians, clinical leaders, and policymakers about the specific needs of patients with ID presenting with forensic issues and differing needs in terms of treatment and risk management.

HELP for behaviours that challenge in adults with intellectual and developmental disabilities.

OBJECTIVE: To provide primary care physicians with an understanding of the causes of behaviours that challenge (BTC) in adults with intellectual and developmental disabilities (IDD), as presented in the 2018 Canadian consensus guidelines for primary care of adults with IDD; to offer a systematic approach to the assessment and treatment of such behaviours; and to link to tools to support these assessments. SOURCES OF INFORMATION: This review elaborates upon guidelines 26 to 29 in the mental health section of the 2018 Canadian consensus guidelines. Several of the authors participated in the development of these guidelines, which were based on literature searches and interdisciplinary input. MAIN MESSAGE: Most adults with IDD are followed by primary care providers but they comprise a small proportion of primary care practices. Unique ways of communicating needs, diagnostic queries, and BTC are common in this population. This complexity can lead to missed diagnoses and inappropriate antipsychotic medication use with attendant risks. This article presents a systematic approach, HELP, to the assessment and treatment of factors of Health, Environment, Lived experience, and Psychiatric conditions that can lead to BTC and includes tools to support these assessments. CONCLUSION: A structured approach to the assessment and treatment of BTC in adults with IDD helps family physicians provide guideline-directed, individualized care to this population. This includes a systematic evaluation using the HELP framework that takes place over multiple visits. A team of health professionals might be needed for optimal care, but these resources are not routinely available across Canada.

Intravenous valproate in acute adolescent mania: a preliminary report.

This study evaluated the efficacy and tolerability of intravenous valproate in acute mania in adolescents. Five consecutive patients, meeting the study criteria, were treated with intravenous valproate. Of them, three significantly improved and one had no appreciable improvement. Two out of five patients experienced gastrointestinal and neurological side effects. This preliminary study suggests intravenous valproate is useful and safe to treat acute juvenile mania.