Unintended consequences of technology in competency-based education: a qualitative study of lessons learned in an OtoHNS program.

BACKGROUND: Formative feedback and entrustment ratings on assessments of entrustable professional activities (EPAs) are intended to support learner self-regulation and inform entrustment decisions in competency-based medical education. Technology platforms have been developed to facilitate these goals, but little is known about their effects on these new assessment practices. This study investigates how users interacted with an e-portfolio in an OtoHNS surgery program transitioning to a Canadian approach to competency-based assessment, Competence by Design. METHODS: We employed a sociomaterial perspective on technology and grounded theory methods of iterative data collection and analysis to study this OtoHNS program's use of an e-portfolio for assessment purposes. All residents (n = 14) and competency committee members (n = 7) participated in the study; data included feedback in resident portfolios, observation of use of the e-portfolio in a competency committee meeting, and a focus group with residents to explore how they used the e-portfolio and visualize interfaces that would better meet their needs. RESULTS: Use of the e-portfolio to document, access, and interpret assessment data was problematic for both residents and faculty, but the residents faced more challenges. While faculty were slowed in making entrustment decisions, formative assessments were not actionable for residents. Workarounds to these barriers resulted in a "numbers game" residents played to acquire EPAs. Themes prioritized needs for searchable, contextual, visual, and mobile aspects of technology design to support use of assessment data for resident learning. CONCLUSION: Best practices of technology design begin by understanding user needs. Insights from this study support recommendations for improved technology design centred on learner needs to provide OtoHNS residents a more formative experience of competency-based training.

Fighting Fires and Battling the Clock: Advance Care Planning in Family Medicine Residency.

BACKGROUND AND OBJECTIVES: Few family physicians treating patients with life-limiting illness report regularly initiating advance care planning (ACP) conversations about illness understanding, values, or care preferences. To better understand how family medicine training contributes to this gap in clinical care, we asked how family medicine residents learn to engage in ACP in the workplace. METHODS: We coded semistructured interviews with family medicine residents (n=9), reflective memos (n=9), and autoethnographic field notes (n=37) using a constructivist-grounded theory approach. We next used the constant comparative method of grounded theory to develop two composite narratives describing participants' experiences that we then member-checked with participants. RESULTS: We identified six core categories of social process to describe how participants were taught to engage in advance care planning. These social processes included previously unidentified barriers to ACP that were specific to their role as learners. These barriers appeared to lead to cultural avoidance of prognosis, conflation of ACP and goals of care (GOC) conversations, and deferral of difficult conversations to nonprimary care settings. CONCLUSIONS: Family medicine educators should consider developing interventions such as flexible clinic schedules, dedicated ACP time, deliberate observed practice, and structured teaching to address potential barriers identified in this exploratory research. Family medicine leaders may wish to consider directly teaching residents and preceptors about crucial differences between ACP and GOC discussions. Shifting curricular focus toward eliciting values and illness understanding during ACP could help resolve a cultural avoidance of prognosis that limits family medicine residents' attempts to engage in ACP.

The embodiment of practice thresholds: from standardization to stabilization in surgical education.

Surgeons practice their own variations on a procedure. Residents experience shifting thresholds between variations that one surgeon holds firmly as principle and another takes more lightly as preference. Such variability has implications for surgical education, but the impact is not well understood. This is a critical problem to investigate as programs seek to define procedures for competency-based medical education (CBME) and improve learning through deliberate practice. Our study analyzes the emergence of procedural variation in an early-adopter CBME program through a situational analysis of tonsillectomy, a foundation level procedure in this otolaryngology, head and neck surgical program. An earlier phase of the study identified frequent variations (n = 12) on tonsillectomy among co-located surgeons who routinely perform this procedure (n = 6). In the phase reported here we interviewed these surgeons (n = 4) and residents at different stages of training (n = 3) about their experiences of these variations to map the relations of contributing social and material actors. Our results show that even a basic procedure resists standardization. This study contributes a sociomaterial grounded theory of surgical practice as an embodied response to conditions materialized by intra-relations of human and more-than-human actors. Shifting root metaphors about practice in surgical education from standardization to stabilization can help residents achieve stable-for-now embodiments of performance as their practice thresholds continue to emerge.

Scut to Scholarship: Can Operative Notes be Educationally Useful?

OBJECTIVE: Efforts to implement competency-based medical education require new sources of workplace-based evidence of growth in learning. We used qualitative analysis of operative notes to explore procedural variation in a simple surgical procedure. DESIGN: We used a grounded theory-based mixed methods approach to depict intersurgeon procedural variation. Our grounded theory approach to analysis included follow up interviews with surgeons and residents to probe their understandings of the reasons for variation in the dictated notes and the current and potential utility of operative notes as a reliable source of data for learning and assessment. SETTING: Publicly funded tertiary care otolaryngology-head & neck surgery residency program in Ontario, Canada PARTICIPANTS: Using maximum variability sampling, all surgeons performing tonsillectomy in the department (n = 6) contributed operative notes from 65 tonsillectomies, 5 intraoperative observations, and 4 semi-structured interviews. An additional 3 residents from various levels of training contributed semistructured interviews. RESULTS: Intersurgeon procedural variations persist even in simple surgical procedures such as tonsillectomy. Operative notes appear to capture procedural variations in a limited way. Surgeons and resident make informal educational use of the clerical work of writing and assessing operative notes, but optimization will be required to shift such hidden work into the formal educational domain. CONCLUSIONS: The implementation of competency-based medical education requires surgical educators to both eliminate low-yield tasks for learning and to find new opportunities for multiple low-stakes assessment. Analysis of operative notes may become a high-yield strategy for learning and assessment if residents and surgeons are coached to use operative notes more reliably and efficiently.

An Active In-Home Physician Model of Palliative Care and Its Resulting Performance Indicators Related to Home Deaths, Unplanned Emergency Department Visits and Unplanned Hospital Admissions.

BACKGROUND: Limited research has characterized team-based models of home palliative care and the outcomes of patients supported by these care teams. CASE PRESENTATION: A retrospective case series describing care and outcomes of patients managed by the London Home Palliative Care Team between May 1, 2017 and April 1, 2019. CASE MANAGEMENT: The London Home Palliative Care (LHPC) Team care model is based upon 3 pillars: 1) physician visit availability 2) active patient-centered care with strong physician in-home presence and 3) optimal administrative organization. CASE OUTCOMES: In the 18 month study period, 354 patients received care from the London Home Palliative Care Team. Most significantly, 88.4% (n = 313) died in the community or at a designated palliative care unit after prearranged direct transfer; no comparable provincial data is available. 21.2% (n = 75) patients visited an emergency department and 24.6% (n = 87) were admitted to hospital at least once in their final 30 days of life. 280 (79.1%) died in the community. These values are better than comparable provincial estimates of 62.7%, 61.7%, and 24.0%, respectively. CONCLUSION: The London Home Palliative Care (LHPC) Team model appears to favorably impact community death rate, ER visits and unplanned hospital admissions, as compared to accepted provincial data. Studies to determine if this model is reproducible could support palliative care teams achieving similar results.

Demographic and socioeconomic characteristics of Canadian medical students: a cross-sectional study.

BACKGROUND: While the importance of medical students' demographic characteristics in influencing the scope and location of their future practice is recognized, these data are not systematically collected in Canada. This study aimed to characterize and compare the demographics of Canadian medical students with the Canadian population. METHODS: Through an online survey, delivered in 2018, medical students at 14 English-speaking Canadian medical schools provided their age, sex, gender identity, ethnicity, educational background, and rurality of the area they grew up in. Respondents also provided information on parental income, occupation, and education as markers of socioeconomic status. Data were compared to the 2016 Canadian Census. RESULTS: A total of 1388 students responded to the survey, representing a response rate of 16.6%. Most respondents identified as women (63.1%) and were born after 1989 (82.1%). Respondents were less likely, compared to the Canadian Census population, to identify as black (1.7% vs 6.4%) (P < 0.001) or Aboriginal (3.5% vs. 7.4%) (P < 0.001), and have grown up in a rural area (6.4% vs. 18.7%) (P < 0.001). Respondents had higher socioeconomic status, indicated by parental education (29.0% of respondents' parents had a master's or doctoral degree, compared to 6.6% of Canadians aged 45-64), occupation (59.7% of respondents' parents were high-level managers or professionals, compared to 19.2% of Canadians aged 45-64), and income (62.9% of respondents grew up in households with income >$100,000/year, compared to 32.4% of Canadians). Assessment of non-response bias showed that our sample was representative of all students at English-speaking Canadian medical schools with respect to age, though a higher proportion of respondents were female. Additionally, there were no differences between early and late respondents with respect to ethnicity, rurality, and parental income, occupation, and education. CONCLUSIONS: Canadian medical students have different socioeconomic characteristics compared to the Canadian population. Collecting and analyzing these characteristics can inform evidence-based admissions policies.

How Do Thresholds of Principle and Preference Influence Surgeon Assessments of Learner Performance?

OBJECTIVE: The present study asks whether intraoperative principles are shared among faculty in a single residency program and explores how surgeons' individual thresholds between principles and preferences might influence assessment. BACKGROUND: Surgical education continues to face significant challenges in the implementation of intraoperative assessment. Competency-based medical education assumes the possibility of a shared standard of competence, but intersurgeon variation is prevalent and, at times, valued in surgical education. Such procedural variation may pose problems for assessment. METHODS: An entire surgical division (n = 11) was recruited to participate in video-guided interviews. Each surgeon assessed intraoperative performance in 8 video clips from a single laparoscopic radical left nephrectomy performed by a senior learner (>PGY5). Interviews were audio recorded, transcribed, and analyzed using the constant comparative method of grounded theory. RESULTS: Surgeons' responses revealed 5 shared generic principles: choosing the right plane, knowing what comes next, recognizing normal and abnormal, making safe progress, and handling tools and tissues appropriately. The surgeons, however, disagreed both on whether a particular performance upheld a principle and on how the performance could improve. This variation subsequently shaped their reported assessment of the learner's performance. CONCLUSIONS: The findings of the present study provide the first empirical evidence to suggest that surgeons' attitudes toward their own procedural variations may be an important influence on the subjectivity of intraoperative assessment in surgical education. Assessment based on intraoperative entrustment may harness such subjectivity for the purpose of implementing competency-based surgical education.

Training the next generation of Canadian Clinician-Scientists: charting a path to success.

Clinician-scientists are physicians with training in both clinical medicine and research that enables them to occupy a unique niche as specialists in basic and translational biomedical research. While there is widespread acknowledgement of the importance of clinician-scientists in today's landscape of evidence-based medical practice, training of clinician-scientists in Canada has been on the decline, with fewer opportunities to obtain funding. With the increasing length of training and lower financial compensation, fewer medical graduates are choosing to pursue such a career. MD-PhD programs, in which trainees receive both medical and research training, have the potential to be an important tool in training the next generation of clinician-scientists; however, MD-PhD trainees in Canada face barriers that include an increase in medical school tuition and a decrease in the amount of financial support. We examined the available data on MD-PhD training in Canada and identified a lack of oversight, a lack of funding and poor mentorship as barriers experienced by MD-PhD trainees. Specific recommendations are provided to begin the process of addressing these challenges, starting with the establishment of an overseeing national body that would track long-term outcome data for MD-PhD trainees. This national body could then function to implement best practices from individual programs across the country and to provide further mentorship and support for early-career physician-scientists. MD-PhD programs have the potential to address Canada's growing shortage of clinician-scientists, and strengthening MD-PhD programs will help to effect positive change.

"Staying in the Game": How Procedural Variation Shapes Competence Judgments in Surgical Education.

PURPOSE: Emerging research explores the educational implications of practice and procedural variation between faculty members. The potential effect of these variations on how surgeons make competence judgments about residents has not yet been thoroughly theorized. The authors explored how thresholds of principle and preference shaped surgeons' intraoperative judgments of resident competence. METHOD: This grounded theory study included reanalysis of data on the educational role of procedural variations and additional sampling to attend to their impact on assessment. Reanalyzed data included 245 hours of observation across 101 surgical cases performed by 29 participants (17 surgeons, 12 residents), 39 semistructured interviews (33 with surgeons, 6 with residents), and 33 field interviews with residents. The new data collected to explore emerging findings related to assessment included two semistructured interviews and nine focused field interviews with residents. Data analysis used constant comparison to refine the framework and data collection process until theoretical saturation was reached. RESULTS: The core category of the study, called staying in the game, describes how surgeons make moment-to-moment judgments to allow residents to retain their role as operators. Surgeons emphasized the role of principles in making these decisions, while residents suggested that working with surgeons' preferences also played an important role in such intraoperative assessment. CONCLUSIONS: These findings suggest that surgeons' and residents' work with thresholds of principle and preference have significant implications for competence judgments. Making use of these judgments by turning to situated assessment may help account for the subjectivity in assessment fostered by faculty variations.

Rethinking research in the medical humanities: a scoping review and narrative synthesis of quantitative outcome studies.

OBJECTIVES: The rise of medical humanities teaching in medical education has introduced pressure to prove efficacy and utility. Review articles on the available evidence have been criticised for poor methodology and unwarranted conclusions. To support a more nuanced discussion of how the medical humanities work, we conducted a scoping review of quantitative studies of medical humanities teaching. METHODS: Using a search strategy involving MEDLINE, EMBASE and ERIC, and hand searching, our scoping review located 11 045 articles that referred to the use of medical humanities teaching in medical education. Of these, 62 studies using quantitative evaluation methods were selected for review. Three iterations of analysis were performed: descriptive, conceptual, and discursive. RESULTS: Descriptive analysis revealed that the medical humanities as a whole cannot be easily systematised based on simple descriptive categories. Conceptual analysis supported the development of a conceptual framework in which the foci of the arts and humanities in medical education can be mapped alongside their related epistemic functions for teaching and learning. Within the framework, art functioned as expertise, as dialogue or as a means of expression and transformation. In the discursive analysis, we found three main ways in which the relationship between the arts and humanities and medicine was constructed as, respectively, intrinsic, additive and curative. CONCLUSIONS: This review offers a nuanced framework of how different types of medical humanities work. The epistemological assumptions and discursive positioning of medical humanities teaching frame the forms of outcomes research that are considered relevant to curriculum decision making, and shed light on why dominant review methodologies make some functions of medical humanities teaching visible and render others invisible. We recommend the use of this framework to improve the rigor and relevance of future explorations of the efficacy and utility of medical humanities teaching.

"They Have to Adapt to Learn": Surgeons' Perspectives on the Role of Procedural Variation in Surgical Education.

OBJECTIVE: Clinical research increasingly acknowledges the existence of significant procedural variation in surgical practice. This study explored surgeons' perspectives regarding the influence of intersurgeon procedural variation on the teaching and learning of surgical residents. DESIGN AND SETTING: This qualitative study used a grounded theory-based analysis of observational and interview data. Observational data were collected in 3 tertiary care teaching hospitals in Ontario, Canada. Semistructured interviews explored potential procedural variations arising during the observations and prompts from an iteratively refined guide. Ongoing data analysis refined the theoretical framework and informed data collection strategies, as prescribed by the iterative nature of grounded theory research. PARTICIPANTS: Our sample included 99 hours of observation across 45 cases with 14 surgeons. Semistructured, audio-recorded interviews (n = 14) occurred immediately following observational periods. RESULTS: Surgeons endorsed the use of intersurgeon procedural variations to teach residents about adapting to the complexity of surgical practice and the norms of surgical culture. Surgeons suggested that residents' efforts to identify thresholds of principle and preference are crucial to professional development. Principles that emerged from the study included the following: (1) knowing what comes next, (2) choosing the right plane, (3) handling tissue appropriately, (4) recognizing the abnormal, and (5) making safe progress. Surgeons suggested that learning to follow these principles while maintaining key aspects of surgical culture, like autonomy and individuality, are important social processes in surgical education. CONCLUSIONS: Acknowledging intersurgeon variation has important implications for curriculum development and workplace-based assessment in surgical education. Adapting to intersurgeon procedural variations may foster versatility in surgical residents. However, the existence of procedural variations and their active use in surgeons' teaching raises questions about the lack of attention to this form of complexity in current workplace-based assessment strategies. Failure to recognize the role of such variations may threaten the implementation of competency-based medical education in surgery.

A model of engagement in reflective writing-based portfolios: Interactions between points of vulnerability and acts of adaptability.

BACKGROUND: Portfolios are widely used for meeting new accreditation standards in the age of competency-based medicine. However, the method of learning through portfolio has been suggested to be vulnerable. AIM: The aim of this study was to explore conditions affecting the experience of teaching and learning from the perspective of both students and mentors in a reflective writing-based portfolio initiative. METHOD: Using mixed-methods rooted in grounded theory, 139 students and 13 mentors completed questionnaires, 23 students participated in four focus groups and 9 mentors in individual interviews. RESULTS: The overarching theme in our data was student-mentor engagement. Our results confirm previous literature describing portfolio as a vulnerable method of learning, extend this concept by identifying and categorizing specific points of vulnerability, and contribute new knowledge regarding acts of adaptability, which serve to strengthen the student-mentor relationship. CONCLUSION: Engagement is central to the success of portfolio and is shaped by a dynamic interaction between points of vulnerability and acts of adaptability. We propose a model of engagement in portfolio that can be used for faculty development to optimize student-mentor engagement.

Thinking like an expert: surgical decision making as a cyclical process of being aware.

BACKGROUND: Education researchers are studying the practices of high-stake professionals as they learn how to better train for flexibility under uncertainty. This study explores the "Reconciliation Cycle" as the core element of an intraoperative decision-making model of how experienced surgeons assess and respond to challenges. METHODS: We analyzed 32 semistructured interviews using constructivist grounded theory to develop a model of intraoperative decision making. Using constant comparison analysis, we built on this model with 9 follow-up interviews about the most challenging cases described in our dataset. RESULTS: The Reconciliation Cycle constituted an iterative process of "gaining" and "transforming information." The cyclical nature of surgeons' decision making suggested that transforming information requires a higher degree of awareness, not yet accounted by current conceptualizations of situation awareness. CONCLUSIONS: This study advances the notion of situation awareness in surgery. This characterization will support further investigations on how expert and nonexpert surgeons implement strategies to cope with unexpected events.

Thresholds of Principle and Preference: Exploring Procedural Variation in Postgraduate Surgical Education.

BACKGROUND: Expert physicians develop their own ways of doing things. The influence of such practice variation in clinical learning is insufficiently understood. Our grounded theory study explored how residents make sense of, and behave in relation to, the procedural variations of faculty surgeons. METHOD: We sampled senior postgraduate surgical residents to construct a theoretical framework for how residents make sense of procedural variations. Using a constructivist grounded theory approach, we used marginal participant observation in the operating room across 56 surgical cases (146 hours), field interviews (38), and formal interviews (6) to develop a theoretical framework for residents' ways of dealing with procedural variations. Data analysis used constant comparison to iteratively refine the framework and data collection until theoretical saturation was reached. RESULTS: The core category of the constructed theory was called thresholds of principle and preference and it captured how faculty members position some procedural variations as negotiable and others not. The term thresholding was coined to describe residents' daily experiences of spotting, mapping, and negotiating their faculty members' thresholds and defending their own emerging thresholds. CONCLUSIONS: Thresholds of principle and preference play a key role in workplace-based medical education. Postgraduate medical learners are occupied on a day-to-day level with thresholding and attempting to make sense of the procedural variations of faculty. Workplace-based teaching and assessment should include an understanding of the integral role of thresholding in shaping learners' development. Future research should explore the nature and impact of thresholding in workplace-based learning beyond the surgical context.

Adaptation and innovation: a grounded theory study of procedural variation in the academic surgical workplace.

RATIONALE, AIMS AND OBJECTIVES: Surgical research struggles to describe the relationship between procedural variations in daily practice and traditional conceptualizations of evidence. The problem has resisted simple solutions, in part, because we lack a solid understanding of how surgeons conceptualize and interact around variation, adaptation, innovation, and evidence in daily practice. This grounded theory study aims to describe the social processes that influence how procedural variation is conceptualized in the surgical workplace. METHOD: Using the constructivist grounded theory methodology, semi-structured interviews with surgeons (n = 19) from four North American academic centres were collected and analysed. Purposive sampling targeted surgeons with experiential knowledge of the role of variations in the workplace. Theoretical sampling was conducted until a theoretical framework representing key processes was conceptually saturated. RESULTS: Surgical procedural variation was influenced by three key processes. Seeking improvement was shaped by having unsolved procedural problems, adapting in the moment, and pursuing personal opportunities. Orienting self and others to variations consisted of sharing stories of variations with others, taking stock of how a variation promoted personal interests, and placing trust in peers. Acting under cultural and material conditions was characterized by being wary, positioning personal image, showing the logic of a variation, and making use of academic resources to do so. Our findings include social processes that influence how adaptations are incubated in surgical practice and mature into innovations. CONCLUSIONS: This study offers a language for conceptualizing the sociocultural influences on procedural variations in surgery. Interventions to change how surgeons interact with variations on a day-to-day basis should consider these social processes in their design.

Understanding clinical uncertainty: what is going on when experienced surgeons are not sure what to do?

PURPOSE: In clinical settings, uncertainty is part of everyday practice. However, a lack of insight into how experts approach uncertainty limits the ability to explicitly teach and assess it in training. This study explored how experienced surgeons perceived and handled uncertainty during challenging intraoperative situations, to develop a theoretical language supporting both education and research. METHOD: This constructivist qualitative study included observations and interviews during 26 surgical cases. The cases, drawn from seven staff surgeons from various specialties at a medical school, were purposively sampled after being preidentified by the surgeon as "likely challenging." The authors combined template and inductive analyses. In template analysis, an existing theory was used to identify instances of uncertainty in the dataset. Inductive analysis was used to elaborate and refine the concepts. RESULTS: Template analysis confirmed that existing theoretical concepts are relevant to surgery. However, inductive analysis revealed additional concepts and positioned existing concepts within new relationships. Two new theoretical themes were recognizing uncertainty and responding to uncertainty, each with corresponding subthemes. Factors such as the novelty of the situation, difficulty in predicting the outcome, and difficulty deciding the course of action mainly characterize an uncertain situation in surgery according to the participants. CONCLUSIONS: The results offer a refined language for conceptualizing uncertainty in surgery. Although further research could elaborate and test the explanatory power of this language, the authors anticipate that it has implications both for current discussions of surgical safety and for future development of explicit training for effective management of surgical uncertainty.