Immersive Leadership Training for Physicians: Comparing Retrospective Pre- and Post-Test Virtual vs in-Person 6-Month Follow-Up of Learning.

Purpose: Little is known about the long-term efficacy of virtual leadership training for physicians. This study compares two highly similar groups of Obstetricians-Gynecologists' (OB-GYN) 6-month post-program changes in competency and skills after experiencing equity-centered leadership training in a virtual or in-person format. Participants and Methods: Using a retrospective pre- and post-test method, we collected 6-month post-program data on 14 competencies for knowledge gains and skills use, comparing the virtual cohort (2021, n = 22) to the in-person cohort (2022, n = 33) in 55 total participants. Qualitative data from open-ended feedback questions informed on skills relevancy and professional impact since program participation. Results: Data indicate strong, statistically significant knowledge and skills retention in both cohorts, with 63% of the virtual and 85% of the in-person participants responding. Data indicate participants report the course having a positive impact on their healthcare provision and nearly all report they made changes to their communication and leadership approaches in the 6-months after the program. 59% of the virtual and 55% of the in-person cohorts report new leadership opportunities since their participation and that the course helped prepare them for those roles. Qualitative data support the need for the training, specific elements of the training these physicians found particularly helpful, and that the learning was "sticky", in that it stayed with them in the months post-program. There was a clear stated preference for in-person experiences. Conclusion: Either virtual or in-person leadership training can result in long-term (6-month) significant retention and application of knowledge and skills in physicians. While limited in size, this study suggests that in-person experiences seem to foster more effective bonds and also greater willingness to participate in post-program follow-up. Physicians find equity-centered leadership training to impact their subsequent communication and leadership practices and they report career benefits even in 6-month follow-up.

While physicians serve in many leadership roles in healthcare, leadership training is generally not part of their medical training. The American College of Obstetricians and Gynecologists (ACOG) Robert C. Cefalo Leadership Institute has provided an annual leadership training for obstetrician and gynecologist leaders since 2006. Our previous research has repeatedly shown the program is effective, with participants experiencing significant and impressive gains in leadership learning and skills development. The COVID-19 pandemic led to the 2021 program being held virtually with a return to an in-person format possible with the 2022 program. As such, the opportunity arose to compare the 6-month post-program learning and impact of these two formats, virtual versus in-person training, in two highly similar groups experiencing nearly identical program content. Both virtual and in-person participants rated their six-month post-program skill level/ability and skills use/implementation as significantly higher than pre-program and both groups noted the learning helped them be better physicians, communicators, and leaders. Additionally, many experienced new leadership opportunities in the 6-months post-program and most of those agreed that the program prepared them to take on those new roles. This study shows that our approach to physician leadership development is highly effective and that the learning demonstrated "stickiness" in that it persisted over time. While both virtual and in-person programs were highly effective, overwhelmingly the participants prefer in-person training to virtual training.

Evaluating Leadership Development Competencies of Clinicians to Build Health Equity in America.

INTRODUCTION: To achieve more equitable health, health care must be grounded in an understanding of social determinants of health. Clinicians need hands-on, equity-centered training in interdisciplinary settings where they can further develop leadership skills and apply learnings in real-time. The Clinical Scholars program trained five cohorts of health care professionals in 25 leadership development competencies to contribute toward advancing health equity within the organizations and communities where they work. This study describes the self-reported ratings of three dimensions of competencies within four domains. METHODS: Data from 169 Fellows were collected at three time-points during the three-year training program using Qualtrics and Research Electronic Data Captrue software. Analysis was conducted in R and included descriptive statistics, fitting a linear mixed-effects model using random intercepts, and paired-sample t tests to assess significance between baseline and endpoint ratings. RESULTS: We found improved ratings over time for each of the three competency dimensions (knowledge, self-efficacy, use) and significant differences in ratings from baseline to endpoint, by domain (personal, interpersonal, organizational, community, and systems). DISCUSSION: These findings support the effectiveness of an equity-centered leadership development curriculum in training health care professionals to address health challenges in their communities and organizations, thereby furthering the broader goal of achieving more equitable health for all.

Comparing Virtual vs In-Person Immersive Leadership Training for Physicians.

Purpose: The COVID-19 pandemic caused a disruption of in-person workforce development programs. Our immersive physician-oriented leadership institute suspended in 2020, resumed in 2021 with a virtual program, and in 2022 reconvened in-person training. We used this opportunity to compare the participant experience, including reported knowledge acquisition and ability gains, between these nearly identical curricula delivered in vastly different circumstances and formats. Participants and Methods: We describe the differences in immersive leadership training implementation and adaptations made for virtual vs in-person engagement of two cohorts of OB-GYN physicians. Data were collected from virtual (n=32) and in-person (n=39) participants via post-session surveys. Quantitative data reported includes participant ratings for knowledge gain and ability gain. Qualitative data were obtained via open-ended feedback questions per session and the overall experience. Results: Knowledge and ability scores indicated strong, statistically significant gains in both formats, with some reported learning gains higher in the virtual training. Qualitative data of participant feedback identified a number of positive themes similar across the in-person and virtual settings, with virtual participants noting how construction of the virtual program produced highly effective experiences and engagement. Constructive or negative feedback of the virtual setting included time constraint issues (eg, a desire for more sessions overall or more time per session) and technical difficulties. Positive comments focused on the effectiveness of the experience in both formats and the surprising ability to connect meaningfully with others, even in a virtual environment. However, there were also many comments clearly supporting the preference for in-person over virtual experiences. Conclusion: Immersive physician leadership training can be effectively delivered via virtual or in-person methods, resulting in significant reported gains of knowledge and skills. These programs provide valuable interpersonal connections and skills to support physician leadership. While both formats are effective, participants clearly prefer in-person leadership development experiences and interpersonal learning.

Accelerating diversity, equity, and inclusion goals: a qualitative assessment from the lens of scientists at the 2020 Clinical Translational Science Awards Annual Meeting.

Participants in the leadership breakout session at the Clinical Translational Science Awards (CTSA) virtual 2020 conference discussed and ranked six recommendations in terms of feasibility, impact, and priority for advancing Diversity, Equity, and Inclusion (DEI) efforts to elevate underrepresented populations to leadership positions in CTSAs and their broader institutions. A thematic analysis of chat and polling data identified challenges and opportunities to achieve DEI goals, with the three most promising recommendations as: cross-institutional Principal Investigator (P.I.) action-learning workgroups, transparent policies for recruiting and promoting underrepresented minorities (URM) leadership, and a clear succession plan to nurture and elevate URM leaders. Suggestions are made to improve DEI in CTSA leadership and allow for greater representation in the translational science field.

Leadership Development to Advance Health Equity: An Equity-Centered Leadership Framework.

Enduring questions about equity are front and center at this watershed moment in health care and public health. Inequities that became evident in the COVID-19 pandemic in 2020 have highlighted long-standing disparities in health by race and ethnicity. Current crises require examining and reorienting the systems that have, for decades, produced these health inequities; yet, public health and health care leaders are inadequately prepared to respond. The authors offer an equity-centered leadership framework to support the development of visionary leaders for tomorrow. This framework for leadership development programs interweaves traditional leadership and equity, diversity, and inclusion domains in both conceptual knowledge and skills-based teaching for health care and public health leaders, grouping competencies and skills into 4 domains: personal, interpersonal, organizational, and community and systems. A framework such as this will equip leaders with the mindset and skill set to challenge the paradigms that lead to inequity and health disparities.

Using a most significant change approach to evaluate learner-centric outcomes of clinical scholars leadership training program.

BACKGROUND: Most significant change (MSC) analysis is an evaluative method used to supplement outcome and impact program evaluations. MSC stories provide mini-narratives contextualizing the effect a program had on an individual. The Clinical Scholars program evaluation team used MSC to explore learner-centric leadership outcomes of the Clinical Scholars leadership training program. PURPOSE OF THE RESEARCH: To identify thematic outcomes of trainees during the three years of their leadership training in the Clinical Scholars program. RESULTS: We identified three central thematic areas of most significant change: (1) leadership in practice, (2) self-awareness, and (3) equity, diversity, and inclusion. Other present themes included stakeholder engagement, effective leadership, collaboration, impact, expanded influence, innovation, networking, community engagement, evidence-based, implementation science, organizational development, hope, and advocating within organizations, teams, and the community. A total of 25 mid- and advanced-career level participants represented a diverse array of disciplines within the healthcare sector. CONCLUSIONS: MSC stories represented leaders' individual and team growth due to participating in the Clinical Scholars program. Results confirmed fidelity of program implementation to the original program goals, as outlined in the grant proposal. They highlighted three critical areas of development for this cohort of Fellows.

Advancing Health Equity Through Equity-Centered Leadership Development with Interprofessional Healthcare Teams.

INTRODUCTION: Events of spring 2020-the COVID19 pandemic and re-birth of a social justice movement-have thrown disparities in disease risk, morbidity, and mortality in sharp relief. In response, healthcare organizations have shifted attentions and resources towards equity, diversity, and inclusion (EDI) issues and initiatives like never before. Focused, proven equity-centered skill and mindset development is needed for healthcare professionals to operationalize these pledges and stated aims. AIM: This article highlights program evaluation results for this Clinical Scholars National Leadership Institute (CSNLI) specific to EDI. We will show that CSNLI imparts the valuable and essential skills to health professionals that are needed to realize health equity through organizational and system change. SETTING: Initial cohort of 29 participants in CSNLI, engaging in the program over 3 years through in-person and distance-based learning offerings and activities. PROGRAM DESCRIPTION: The CSNLI is a 3-year, intensive leadership program that centers EDI skill development across personal, interpersonal, organizational, and systems domains through its design, competencies, and curriculum. PROGRAM EVALUATION: A robust evaluation following the Kirkpatrick Model offers analysis of four data collecting activities related to program participants' EDI learning, behavioral change, and results. DISCUSSION: Over the course of the program, participants made significant gains in competencies related to equity, diversity, and inclusion. Furthermore, participants demonstrated growth in behavior change and leadership activities in the areas of organizational and system change. Results demonstrate the need to center both leader and leadership development on equity, diversity, and inclusion curriculum to make real change in the US Healthcare System.

Menstrual Cycle Tracking Applications and the Potential for Epidemiological Research: A Comprehensive Review of the Literature.

PURPOSE OF REVIEW: We reviewed published studies on menstrual cycle tracking applications (MCTAs) in order to describe the potential of MCTAs for epidemiologic research. RECENT FINDINGS: A search of PubMed, Web of Science, and Scopus for MCTA literature yielded 150 articles. After exclusions, there were 49 articles that addressed the primary interest areas: 1) characteristics of MCTA users in research, 2) reasons women use or continue using MCTAs, 3) accuracy of identifying ovulation and utility at promoting and preventing pregnancy, and 4) quality assessments of MCTAs across several domains. SUMMARY: MCTAs are an important tool for the advancement of epidemiologic research on menstruation. MCTA studies should describe the characteristics of their user-base and missing data patterns. Describing the motivation for using MCTAs throughout a user's life and validating the data collected should be prioritized in future research.

A Qualitative Analysis of Maternal and Child Health Public Health Leadership Institute (MCH PHLI) Leaders: Assessing the Application of Leadership Skills at the "Others" and "Wider Community" Levels of the MCH Leadership Competencies 4.0.

OBJECTIVES: To gain insight into how participants in the Maternal and Child Health Public Health Leadership Institute (MCH PHLI) report applying the leadership skills gained through the program at the "Others" and "Wider Community" levels of the MCH Leadership Competencies 4.0. METHODS: 111 mid- to senior-level MCH leaders participating in the MCH PHLI gave < 5 min oral presentations detailing the impacts resulting from implementation of the skills gained through the leadership development training. Presentations were recorded and transcribed then qualitatively analyzed in reference to the MCH Leadership Competencies 4.0. Impacts were stratified by the "Others" and "Wider Community" levels. RESULTS: Analysis resulted in 1510 separate coded examples, 948 of which were coded as aligning with the MCH Leadership Competency 4.0 areas of "Others", "Wider Community" and with an additional emerging competency. In many examples Participants estimated the numbers of people affected by these leadership activities, which totaled more than 80,773 people across the US. CONCLUSIONS FOR PRACTICE: This analysis suggests that mid-to-senior level intensive leadership development strategies benefit organizations, communities, and systems quite broadly through a virtual "ripple effect" of training. Capturing qualitative data can help elucidate the return on investment for leader development programs in terms of impacts on communities and systems.

Training "Pivots" from the Pandemic: Lessons Learned Transitioning from In-Person to Virtual Synchronous Training in the Clinical Scholars Leadership Program.

INTRODUCTION: Since the inception of distance-based teaching modalities, a debate has ensued over the quality of online versus in-person instruction. Due to the COVID-19 pandemic, a number of teaching environments-including leadership development trainings for post-graduate learners-have been thrust into exploring the virtual learning environment more thoroughly. One three-year leadership development program for interdisciplinary healthcare professionals transitioned three simultaneous leadership intensives from in-person to online in the spring of 2020. METHODS: Documented changes in overall training length, session length, and session format are described. Further, evaluative data were collected from participants at both retreats via post-session surveys. Ninety-three participants attended the 2019 retreat, and 92 participants attended the 2020 virtual retreat. Quantitative data of three rating questions per session are reported: 1) overall session satisfaction, 2) participants' reported knowledge gain, and 3) participants' reported ability gain. Qualitative data were obtained via two open-ended feedback questions per session. RESULTS: In comparing pre/post scores for knowledge and ability, participants had meaningful (and in some cases higher) self-reported gains in knowledge and ability measures in the online environment, as compared to the in-person environment. Participants reported statistically significant gains in all sessions for both knowledge and ability. Qualitative data of participant feedback identified a number of positive themes similar across the in-person and virtual settings. Negative or constructive feedback of the virtual setting included time constraint issues (eg too much content in one session, a desire for more sessions overall), technical difficulties, and the loss of social connection and networking with fellow participants as compared to in-person trainings. DISCUSSION: While meaningful shifts in knowledge and ability ratings indicate that the transition to successful online learning is possible, several disadvantages remain. The preparation time for both faculty and participants was considerable, there is a need to reduce overall content in each session due to time restraints, and participants indicated feeling the loss of one-on-one connections with their peers in the training. Lessons learned of transitioning leadership training from in-person to an online experience are highlighted.

A Qualitative Analysis of Individual Leadership Behaviors Among Participants in the Maternal and Child Health Public Health Leadership Institute.

OBJECTIVES: To gain insights into how participants in the Maternal and Child Health Public Health Leadership Institute (MCH PHLI) report applying the personal leadership skills gained through the program at the "Self" level of the MCH Leadership Competencies 4.0. METHODS: 112 mid- to senior-level MCH leaders completed the MCH PHLI, which is a year-long intensive leadership training program. At graduation, 111 participants gave < 5-min oral presentations detailing the actions taken and impact created by implementation of the skills gained through the MCH PHLI training. Presentations were recorded, transcribed and then qualitatively analyzed in reference to the "Self" level of the MCH Leadership Competencies 4.0. RESULTS: Participants reported 562 coded examples of activities in which they implemented skills aligning with each competency in the Self-Level of the MCH Leadership Competencies 4.0 and with three other competency areas that emerged as themes from the data: networking, confidence, and career advancement. CONCLUSIONS FOR PRACTICE: This analysis suggests that intensive leadership development strategies focused on the mid-to-senior level leader benefit the individual and their organizations in broad and strategic ways that can be captured and described. Further, these applications of skills can create a virtual "ripple effect" of the workforce development program, by ultimately impacting a far greater number of people. Finally, this type of reflective assignment can be a valuable addition to intensive workforce development programs.

An Examination of the Self-directed Online Leadership Learning Choices of Public Health Professionals: The Maternal and Child Health Public Health Leadership Institute Experience.

OBJECTIVES: To assess the self-selected asynchronous leadership module-based learning choices of public health professionals participating in the Maternal and Child Health Public Health Leadership Institute (MCH PHLI). METHODS: Online module completion and evaluation data were used to determine the topics most utilized by the Fellows; whether the topics and mode of training were acceptable, relevant, and practical; and whether participant characteristics explained any usage patterns. PARTICIPANTS: A total of 109 enrolled Fellows in the MCH PHLI program. MAIN OUTCOME MEASURES: Module frequency of selection by Fellows; Fellows' rating scores in regard to relevance, practicality, and acceptability of module topics. RESULTS: All program titles were highly rated. The 5 most frequently selected module topics were employee engagement (87.2%), talent acquisition strategies (84.4%), employee motivation (79.8%), emotional intelligence (78.9%), and workforce development strategies (68.8%). The least accessed topics focused on cultural competence (15.6%), social marketing (25.7%), effective communication and advocacy (25.7%), family partnerships (25.9%), and creating learning organizations (31.2%). All module topics provided were rated as relevant, practical, and acceptable to these public health leaders. CONCLUSION: Self-directed computer-based learning was rated strongly by the MCH public health leaders in this study. Such an approach can be used to customize training to individual needs and interests. These findings suggest that inclusion of skills that enable public health leaders to effectively work with and through others was of core interest in the MCH PHLI. The finding of higher usage of topics related to workforce management can provide guidance for those developing leadership development programs for maternal and child health professionals. In addition, leadership needs and interests should be assessed regularly to ensure that competency-based leadership development guidelines are adapting to the evolving and complex challenges faced by leaders. While these results were tested in MCH professionals, they may be generalizable to other disciplines within the field of public health.

Moving the needle: a retrospective pre- and post-analysis of improving perceived abilities across 20 leadership skills.

To assess the influence of intensive focused leadership training on self-evaluation of leadership skills among Maternal and Child Health (MCH) professionals enrolled in the Maternal and Child Health Public Health Leadership Institute (MCH PHLI). Senior-level MCH leaders (n = 54) participated in the first two cohorts of the MCH PHLI, a senior-level training program funded through the Maternal and Child Health Bureau. Participants were asked to complete a retrospective pre- and post-test rating inventory at program completion. Participants self-identified their skill level across 20 leadership skills that were the focus of the training program. These skills were derived from the MCH Leadership Competencies, 3.0 and literature reviews, and then divided into two domains: Core leadership skills and Organizational/Institutional leadership skills. Data were analyzed to determine whether participants perceived skill level increased by the end of their training year. A one-sided (upper) paired T Test and a Wilcoxen Signed Rank Sum Test were used to determine statistical significance. Increases in perceived skill levels were found to be statistically significant at the alpha = .01 level for all 20 target skills. The MCH PHLI model of intensive leadership development, incorporating a hybrid approach of onsite and distance-based learning, was broadly effective in building targeted leadership skills as perceived by participants.

The behavioral event interview: avoiding interviewing pitfalls when hiring.

The Management Moment" is a regular column within the Journal of Public Health Management and Practice. Edward L. Baker, MD, MPH, MSc, is serving as The Management Moment Editor. Dr Baker is Director of The North Carolina Institute for Public Health, School of Public Health, at the University of North Carolina at Chapel Hill. This column provides commentary and guidance on timely management issues commonly encountered in public health practice.