Patient involvement in assessment of postgraduate medical learners: A scoping review.

CONTEXT: Competency-based assessment of learners may benefit from a more holistic, inclusive, approach for determining readiness for unsupervised practice. However, despite movements towards greater patient partnership in health care generally, inclusion of patients in postgraduate medical learners' assessment is largely absent. METHODS: We conducted a scoping review to map the nature, extent and range of literature examining the inclusion (or exclusion) of patients within the assessment of postgraduate medical learners. Guided by Arskey and O'Malley's framework and informed by Levac et al. and Thomas et al., we searched two databases (MEDLINE® and Embase®) from inception until February 2021 using subheadings related to assessment, patients and postgraduate learners. Data analysis examined characteristics regarding the nature and factor influencing patient involvement in assessment. RESULTS: We identified 41 papers spanning four decades. Some literature suggests patients are willing to be engaged in assessment, however choose not to engage when, for example, language barriers may exist. When stratified by specialty or clinical setting, the influence of factors such as gender, race, ethnicity or medical condition seems to remain consistent. Patients may participate in assessment as a stand-alone group or part of a multi-source feedback process. Patients generally provided high ratings but commented on the observed professional behaviours and communication skills in comparison with physicians who focused on medical expertise. CONCLUSION: Factors that influence patient involvement in assessment are multifactorial including patients' willingness themselves, language and reading-comprehension challenges and available resources for training programmes to facilitate the integration of patient assessments. These barriers however are not insurmountable. While understudied, research examining patient involvement in assessment is increasing; however, our review suggests that the extent which the unique insights will be taken up in postgraduate medical education may be dependent on assessment systems readiness and, in particular, physician readiness to partner with patients in this way.

Conversation Starter: Advancing the Theory of Peer-Assisted Learning.

This Conversation Starter article uses four selected abstracts, one each from the four regional Association of American Medical Colleges (AAMC) Group on Educational Affairs (CGEA) 2018 spring meetings, as a springboard for unpacking the definition of peer-assisted learning (PAL). The aim of this article is to prompt deeper reflection on this phenomenon and, in so doing, to foster scholarly program evaluation of this widely adopted instructional approach. This analysis calls for a more nuanced definition of PAL, one that emphasizes process over structure, one that stimulates examination of "doing" PAL and how this affects the personal and professional development of all involved.

Twelve tips for developing, implementing, and sustaining medical education fellowship programs: Building on new trends and solid foundations.

Medical education fellowship programs (MEFPs) are a form of faculty development contributing to an organization's educational mission and participants' career development. Building an MEFP requires a systematic design, implementation, and evaluation approach which aligns institutional and individual faculty goals. Implementing an MEFP requires a team of committed individuals who provide expertise, guidance, and mentoring. Qualified MEFP directors should utilize instructional methods that promote individual and institutional short and long term growth. Directors must balance the use of traditional design, implementation, and evaluation methodologies with advancing trends that may support or threaten the acceptability and sustainability of the program. Drawing on the expertise of 28 MEFP directors, we provide twelve tips as a guide to those implementing, sustaining, and/or growing a successful MEFP whose value is demonstrated by its impacts on participants, learners, patients, teaching faculty, institutions, the greater medical education community, and the population's health.

Validity evidence of resident competency ratings and the identification of problem residents.

OBJECTIVES: This study examined validity evidence of end-of-rotation evaluations used to measure progress toward mastery of core competencies in residents. In addition, this study investigated whether end-of-rotation evaluations can be used to detect problem residents during their training. METHODS: Historical data for a 4-year period (2009-2012), containing 4986 observations of 291 internal medicine residents, were examined. Residents were observed and assessed by fellows, faculty members and programme directors on nine domains, including the six Accreditation Council for Graduate Medical Education core competencies, as part of their end-of-rotation evaluations. Descriptive statistics were used to collect evidence of the response process. Correlations between competencies and a generalisability study were used to examine the internal structure of the end-of-rotation evaluations. Hierarchical regression was used to estimate the increase in scores across years of training. Scores on end-of-rotation evaluations were compared with trainees identified as problem residents by programme directors. RESULTS: Compared with fellows, faculty and programme directors had significantly greater variability in assigning scores across different competencies. Correlations between competencies ranged from 0.69 to 0.92. The reliability of end-of-rotation evaluations was adequate (fellows, phi coefficient [φ] = 0.68; faculty [including programme directors], φ = 0.71). Mean scores increased by 0.21 points (95% confidence interval 0.18-0.24) per postgraduate year. Mean scores were significantly correlated with classification as a problem resident (r = 0.33, p < 0.001); problem residents also had significantly lower ratings across all competencies during PGY-1 compared with all other residents. CONCLUSIONS: End-of-rotation evaluations are a useful method of measuring the growth in resident performance associated with core competencies when sufficient numbers of end-of-rotation evaluation scores are used. Furthermore, end-of-rotation evaluation scores provide preliminary evidence with which to detect and predict problem residents in subsequent postgraduate training years.

The use of peer optic nerve photographs for teaching direct ophthalmoscopy.

OBJECTIVE: To use a novel teaching exercise to encourage students to practice ophthalmoscopy and to measure the learning effect both subjectively and objectively. DESIGN: Comparative case series. PARTICIPANTS: One hundred thirty-one fourth-year medical students on their 1-week ophthalmology rotations with 89 in the experimental group and 42 in the control group. METHODS: Those in the experimental group had 1 eye dilated and their optic nerve photographed on the first day. The next day, these students received an unlabeled optic nerve photograph belonging to 1 of their peers (typically 8-10 per group) and were given 3 days to identify the student matching the photograph. The students in the control group were simply encouraged to practice ophthalmoscopy on each other without the use of photographs. MAIN OUTCOME MEASURES: Both objective and subjective changes from the beginning to the end of the rotation were measured and compared between the 2 groups. RESULTS: In the 89 students who used peer optic nerve photographs, 75 (84.3%) showed improvement in direct ophthalmoscopy skills over the course of the week. In contrast, only 12 (28.6%) of the 42 control students demonstrated an objective improvement (P<0.001). The subjective confidence levels likewise were more improved in the students who took part in the optic nerve photograph exercise. CONCLUSIONS: These results suggest that the task of matching an unknown optic nerve photograph to the correct eye of a peer leads to increased self-confidence and more proficient use of the direct ophthalmoscope.

Leveraging Mindfulness to Reduce Stress and Improve Quality of Life Among Pharmacy Students.

OBJECTIVE: Methods to improve stress and well-being for health profession trainees are limited. Mindfulness, elevating awareness to the present moment experience with compassion, has been shown to demonstrate effectiveness to enhance well-being. This research leverages techniques from mindfulness to develop and evaluate a credit-bearing longitudinal mindfulness elective, designed to teach mindfulness to improve stress and quality of life (QoL). METHODS: A mindfulness elective was created for pharmacy students. A longitudinal, case-control, survey-based design was used to compare stress and QoL between mindfulness participants and nonparticipant controls. Stress was assessed by the Perceived Stress Scale (PSS) and QoL by the SF-12 v2 Health-Related QoL Scale (SF-12 v2 QoL). RESULTS: Four weeks after course completion, the average PSS score was lower among participants compared to controls (18.58 SD 5.85 vs 20.79 SD 6.31, Cohen's d = 0.36). The Mental Health Component score of SF-12 v2 QoL was higher among participants versus controls (41.94 SD 8.58 vs 36.93 SD 9.59, Cohen's d = 0.55). The Physical Health Component score of SF-12 v2 QoL was lower among participants than the control group (46.13 SD 5.48 vs 48.62 SD 6.53, Cohen's d = 0.41). CONCLUSION: The results indicate small to moderate effect sizes associated with participation in a mindfulness elective, reducing stress and improving mental QoL among pharmacy students. The structure and potential benefits of the course can be extrapolated to other institutions. By offering course credit for mindfulness practice, health profession schools can support student wellness.

Impact of a Regional Grant Program Through the Lens of Social Cognitive Career Theory: A Mixed-Method Evaluation.

PURPOSE: Evaluations of educational grant programs have focused on research productivity, with few examining impacts on grantees or effective program characteristics. This evaluation examined the regional grant program sponsored by Group on Educational Affairs to examine if and how grantees' careers were affected by funding, and if these experiences aligned with program goals. METHOD: In this concurrent, mixed-methods theory-driven evaluation, quantitative and qualitative data were analyzed independently and then integrated to examine complementarity. Quantitative data examined differences among 4 geographic regions and included proposal and grantee characteristics abstracted from administrative records of 52 funded proposals from 2010-2015 grant cycles. Qualitative data from 23 interviews conducted from 2018 to 2019 explored the impact on grantees, with Social Cognitive Career Theory (SCCT) serving as a framework for deductive thematic analysis. To facilitate integration of findings, quantitative data were layered onto each interview to permit exploration of associations between the 2 data types. RESULTS: Although significant regional differences existed in project length and amount of funding, there were few regional differences in grantee experiences. Despite small funding amounts, grants were perceived as career launching pads. The SCCT framework accounted for grantee experiences, including researcher identity formation and subsequent research, but did not capture collaboration phenomena. Integration of the 2 data types identified experience patterns unique to different groups of grantees (e.g., more or less research experience). The diversity among grantees suggests that clarification of program goals and stronger alignment with criteria for funding may be warranted. CONCLUSIONS: This evaluation illuminates why small educational grant programs may or may not impact interest and productivity in research. Implications exist for funders, including clarifying program goals and providing support for less experienced grantees. Future research should explore grantee subsets (e.g., underrepresented in medicine) to further identify what fosters or inhibits careers of medical education scholars.

Developing a simulation-based training curriculum in transesophageal ultrasound with the use of the endobronchial ultrasound-endoscope.

There is an increasing need to focus on how best to train respiratory physicians to perform EUS with bronchoscope-guided fine-needle aspiration biopsy (EUS-B-FNA). At current, training is mostly performed in the clinical environment under expert supervision; however, the advent of simulation-based education now provides a low-risk setting for novice trainees to learn and practice basic endosonography skills from identifying and understanding normal anatomy as well as pathology, maneuvering of endoscope, interpretation of images, and mastering of sampling techniques. In this descriptive educational paper, we used a six-step approach as a framework to describe the development of a structured training program combining EUS-B-FNA with the already well-established certification training program in endobronchial ultrasound transbronchial needle aspiration. This comprehensive training curriculum includes a theoretical course to achieve foundational knowledge, followed by simulation-based training until mastery standards are met, and supervised clinical apprenticeship. All steps should end with an objective assessment to achieve certification. This systematic development will hopefully encourage endosonography leaders and educators to collaborate and implement an evidence-based comprehensive endosonography curriculum that aims to provide the trainee with the essential EUS-B competencies to ensure that lung cancer patients are diagnosed and staged correctly.

Defining Successful Practice Within Health Systems Science Among Entering Residents: A Single-Institution Qualitative Study of Graduate Medical Education Faculty Observations.

PURPOSE: The American Medical Association's Accelerating Change in Medical Education consortium defined health systems science (HSS) as the study of how health care is delivered, how health care professionals work together to deliver that care, and how the health system can improve patient care and health care delivery. This framework is increasingly being incorporated into medical school curricula. Graduate medical education (GME) had previously elevated systems-based practice (SBP) as a core competency, but expectations are defined by specialty-specific milestones. The lack of a shared competency framework between undergraduate medical education (UME) and GME makes it challenging to ensure that entering residents are prepared to implement HSS/SBP concepts in the workplace. The authors explored GME faculty observations of residents exemplifying successful practice across HSS domains to inform targets for UME training and assessment. METHOD: Authors performed a single-institution qualitative study using transcribed phone interviews with eligible Vanderbilt residency program directors, associate program directors, and core faculty. Participants described observations of successful residents within each HSS domain. Two researchers independently coded, discussed, and reconciled deidentified transcripts using inductive-deductive approaches to identify themes. RESULTS: Seventeen faculty participated across specialties (17/39, 45%). Faculty responses emphasize precurricular experiences including professional degrees, work experience, extracurriculars, and medical school exposure. Importantly, successful residents exhibit foundational core workforce characteristics including growth mindset, curiosity, and a desire to learn about systems. GME faculty identified HSS domain-specific skills, noting distinctions among learning environments. Outcomes resulting from residents' application of HSS concepts include delivering high-quality, person-centered care and systems improvements. CONCLUSIONS: Descriptions of successful practice within HSS domains highlight preparatory experiences and core workforce characteristics and outline entry-level HSS behaviors. Conceptualized in a logic model framework, these findings describe key inputs, learning activities, outputs, and outcomes for systems-prepared entering residents bridging the UME-GME transition.

Fundamentals of Scholarly Peer Review: A Workshop for Health Professions Educators on Practicing Scholarly Citizenship.

Introduction: Education scholarship requires peer reviewers. For novice scholars, reviewing is an important developmental activity that cultivates deeper participation in the scholarship community. Yet getting started with reviewing is challenging for those not involved with the educational scholarship community. Beyond scientific expertise, reviewers must have a mentoring mindset, skills in providing constructive feedback, and knowledge of common ethical challenges associated with producing and evaluating scholarship. Methods: Our workshop introduced novice health professions educators to peer reviewing. It included four stimulus presentations about the peer reviewer's mindset and skills, followed by reinforcement activities. The workshop could be adapted to variously sized groups. An 8:1 ratio of participants to facilitators was ideal for activities. Topics covered included considerations before accepting an invitation, the review process, the good citizen approach to reviewing, and reviewer ethics. The session concluded with suggestions for continued development of peer reviewer competencies. The workshop was evaluated using a custom survey. Results: Throughout 2019 and early 2020, 58 health professions educators and trainees participated in the workshop across varied venues. Evaluations were obtained from 33 participants (57%). Nearly all rated the workshop as high quality and valuable to peer reviewer preparation. Most (26 of 33; 75%) gained confidence about their qualifications to serve as reviewers. Eighty percent (28 of 33) believed they could recognize ethical dilemmas. Discussion: This workshop provided a springboard for peer reviewing health professions education scholarship. Participants generally praised the experience for introducing them to the world of peer review and preparing them for it.

How clinical features are presented matters to weaker diagnosticians.

OBJECTIVES: This study aimed to test the extent to which the use of medicalese (i.e. formal medical terminology and semantic qualifiers) alters the test performance of medical graduates; to tease apart the extent to which any observed differences are driven by language difficulties versus differences in medical knowledge; and to assess the impact of varying the language used to present clinical features on the ability of the test to consistently discriminate between candidates. METHODS: Six clinical cases were manipulated in the context of pilot items on the Canadian national qualifying examination. Features indicative of two diagnoses were presented uniformly in lay terms, medical terminology and semantic qualifiers, respectively, and in mixed combinations (e.g. features of one diagnosis were presented using lay terminology and features of the other using medicalese). The rate at which the indicated diagnoses were named was considered as a function of language used, site of training, birthplace and medical knowledge (as measured by overall performance on the examination). RESULTS: In the mixed conditions, Canadian medical graduates were not influenced by the language used to present the cases, whereas international medical graduates (IMGs) were more likely to favour the diagnosis associated with medical terminology relative to that associated with lay terms. This was true regardless of whether the entire sample or only North American-born candidates were considered. Within the IMG cohort, high performers were not influenced by the language manipulation, whereas low performers were. Uniform use of lay terminology resulted in the highest test reliability compared with the other experimental conditions. CONCLUSIONS: The results indicate that the influence of medical terminology is driven more by substandard medical knowledge than by the language issues that challenge some candidates. Implications for both the assessment and education of medical professionals are discussed.

A hypothesis-driven physical examination learning and assessment procedure for medical students: initial validity evidence.

CONTEXT: Diagnostic accuracy is maximised by having clinical signs and diagnostic hypotheses in mind during the physical examination (PE). This diagnostic reasoning approach contrasts with the rote, hypothesis-free screening PE learned by many medical students. A hypothesis-driven PE (HDPE) learning and assessment procedure was developed to provide targeted practice and assessment in anticipating, eliciting and interpreting critical aspects of the PE in the context of diagnostic challenges. OBJECTIVES: This study was designed to obtain initial content validity evidence, performance and reliability estimates, and impact data for the HDPE procedure. METHODS: Nineteen clinical scenarios were developed, covering 160 PE manoeuvres. A total of 66 Year 3 medical students prepared for and encountered three clinical scenarios during required formative assessments. For each case, students listed anticipated positive PE findings for two plausible diagnoses before examining the patient; examined a standardised patient (SP) simulating one of the diagnoses; received immediate feedback from the SP, and documented their findings and working diagnosis. The same students later encountered some of the scenarios during their Year 4 clinical skills examination. RESULTS: On average, Year 3 students anticipated 65% of the positive findings, correctly performed 88% of the PE manoeuvres and documented 61% of the findings. Year 4 students anticipated and elicited fewer findings overall, but achieved proportionally more discriminating findings, thereby more efficiently achieving a diagnostic accuracy equivalent to that of students in Year 3. Year 4 students performed better on cases on which they had received feedback as Year 3 students. Twelve cases would provide a reliability of 0.80, based on discriminating checklist items only. CONCLUSIONS: The HDPE provided medical students with a thoughtful, deliberate approach to learning and assessing PE skills in a valid and reliable manner.

The International Literature on Teaching Faculty Development in English-Language Journals: A Scoping Review and Recommendations for Core Topics.

BACKGROUND: With increasing physician mobility, there is interest in how medical schools and postgraduate medical education institutions across the world develop and maintain the competence of medical teachers. Published reviews of faculty development (FD) have predominantly included studies from the United States and Canada. OBJECTIVE: We synthesized the international FD literature (beyond the US and Canada), focusing on FD type, intended audience, study format, effectiveness, differences among countries, and potential unique features. METHODS: We identified English-language publications that addressed FD for medical faculty for teaching and related activities, excluding US and Canadian publications. RESULTS: A search of 4 databases identified 149 publications, including 83 intervention studies. There was significant growth in international FD publications for the most recent decade, and a sizable number of studies were from developing economies and/or resulted from international collaborations. Focal areas echo those in earlier published reviews, suggesting the international FD literature addresses similar faculty needs and organizational concerns. CONCLUSIONS: The growth in publications in recent years and a higher proportion of reporting on participant reactions, coupled with less frequent reporting of results, transfer to practice, and impact on learners and the organization, suggest this is an evolving field. To enhance international FD, educators and researchers should focus on addressing common needs expressed by faculty, including curriculum design and evaluation, small group teaching, assessing professionalism and providing feedback. Future research should focus on approaches for developing comprehensive institutional FD programs that include communities of learning and practice and evaluating their impact.

Pilot Qualitative Study of Informal Teachers in Interprofessional Collaboration and Practice.

CONTEXT: Nonphysician members of the interprofessional palliative care team often participate in teaching physicians and others in the context of workplace learning due to the interprofessional collaborative nature of the specialty. OBJECTIVES: This pilot study examines the beliefs of the nonphysician members of the interprofessional team about teaching physicians-in-training, the disciplinary training and expertise that informs their teaching, and approaches to teaching in the workplace. METHODS: Semistructured interviews were conducted. All interviews were audio recorded and transcribed verbatim. Initial open coding by two researchers identified the codes, and then the constant comparative method was used to find patterns by axial coding, categories, and themes within the data. RESULTS: Of the 10 health care professionals involved with palliative medical education at one academic medical center, six enrolled in the pilot. Those who participated included chaplains, nurses, a social worker, and a physician assistant. Three major themes were identified from the informal teachers: 1) using professional identity as a foundation for teaching, 2) teaching through experiential learning or debriefing, and 3) teaching to perceived gaps in physician training. CONCLUSION: Nonphysician members of the interprofessional team interacted with physicians-in-training guided by their discipline-based skills and perspectives on patient care. They directed their informal teaching toward perceived educational gaps using reflection and debriefing. Future studies could explore the educational roles of health care professionals across diverse institutions and specialties.

Teaching cardiac examination skills. A controlled trial of two methods.

OBJECTIVE: To determine if structured teaching of bedside cardiac examination skills improves medical residents' examination technique and their identification of key clinical findings. DESIGN: Firm-based single-blinded controlled trial. SETTING: Inpatient service at a university-affiliated public teaching hospital. PARTICIPANTS: Eighty Internal Medicine residents. METHODS: The study assessed 2 intervention groups that received 3-hour bedside teaching sessions during their 4-week rotation using either: (1) a traditional teaching method, "demonstration and practice" (DP) (n=26) or (2) an innovative method, "collaborative discovery" (CD) (n=24). The control group received their usual ward teaching sessions (n=25). The main outcome measures were scores on examination technique and correct identification of key clinical findings on an objective structured clinical examination (OSCE). RESULTS: All 3 groups had similar scores for both their examination technique and identification of key findings in the preintervention OSCE. After teaching, both intervention groups significantly improved their technical examination skills compared with the control group. The increase was 10% (95% confidence interval [CI] 4% to 17%) for CD versus control and 12% (95% CI 6% to 19%) for DP versus control (both P<.005) equivalent to an additional 3 to 4 examination skills being correctly performed. Improvement in key findings was limited to a 5% (95% CI 2% to 9%) increase for the CD teaching method, CD versus control P=.046, equivalent to the identification of an additional 2 key clinical findings. CONCLUSIONS: Both programs of bedside teaching increase the technical examination skills of residents but improvements in the identification of key clinical findings were modest and only demonstrated with a new method of teaching.