Patient Advocacy Assessment in the Medicine Clerkship: A Qualitative Study of Definition, Context, and Impact.

BACKGROUND: Advocacy is a core value of the medical profession. However, patient advocacy (advocacy) is not uniformly assessed and there are no studies of the behaviors clinical supervisors consider when assessing advocacy. OBJECTIVE: To explore how medical students and supervisors characterize advocacy during an internal medicine clerkship, how assessment of advocacy impacted students and supervisors, and elements that support effective implementation of advocacy assessment. DESIGN: A constructivist qualitative paradigm was used to understand advocacy assessment from the perspectives of students and supervisors. PARTICIPANTS: Medical students who completed the internal medicine clerkship at UCSF during the 2018 and 2019 academic years and supervisors who evaluated students during this period. APPROACH: Supervisor comments from an advocacy assessment item in the medicine clerkship and transcripts of focus groups were used to explore which behaviors students and supervisors deem to be advocacy. Separate focus groups with both students and supervisors examined the impact that advocacy assessment had on students' and supervisors' perceptions of advocacy and what additional context was necessary to effectively implement advocacy assessment. KEY RESULTS: Students and supervisors define advocacy as identifying and addressing social determinants of health, recognizing and addressing patient wishes and concerns, navigating the health care system, conducting appropriate evaluation and treatment, and creating exceptional therapeutic alliances. Effective implementation of advocacy assessment requires the creation of non-hierarchical team environments, supervisor role modeling, and pairing assessment with teaching of advocacy skills. Inclusion of advocacy assessment reflects and dictates institutional priorities, shapes professional identity formation, and enhances advocacy skill development for students and their supervisors. CONCLUSIONS: Students and supervisors consider advocacy to be a variety of behaviors beyond identifying and addressing social determinants of health. Effectively implementing advocacy assessment shapes students' professional identity formation, underscoring the critical importance of formally focusing on this competency in the health professions education.

Innovation and Missed Opportunities in Internal Medicine Undergraduate Education During COVID-19: Results from a National Survey.

BACKGROUND: COVID-19 disrupted undergraduate clinical education when medical schools removed students from clinical rotations following AAMC recommendations. Clerkship directors (CDs) had to adapt rapidly and modify clerkship curricula. However, the scope and effects of these modifications are unknown. OBJECTIVE: To examine the effects of the initial phase of COVID-19 on the internal medicine (IM) undergraduate clinical education. DESIGN: A nationally representative web survey. PARTICIPANTS: IM CDs from 137 LCME-accredited US medical schools in 2020. MAIN MEASURES: Items (80) assessed clerkship structure and curriculum, assessment in clerkships, post-clerkship IM clinical experiences, and CD roles and support. The framework of Understanding Crisis Response (Royal Society for Encouragement of Arts, Manufactures, and Commerce) was used to determine whether curricular modifications were "amplified," "restarted," "let go," or "ended." KEY RESULTS: Response rate was 74%. In response to COVID-19, 32% (32/101) of clerkships suspended all clinical activities and 66% (67/101) only in-person. Prior to clinical disruption, students spent a median of 8.0 weeks (IQR: 2) on inpatient and 2.0 weeks (IQR: 4) on ambulatory rotations; during clinical re-entry, students were spending 5.0 (IQR: 3) and 1.0 (IQR: 2) weeks, respectively. Bedside teaching and physical exam instruction were "let go" during the early phase. Students were removed from direct patient care for a median of 85.5 days. The sub-internship curriculum remained largely unaffected. Before the pandemic, 11% of schools were using a pass/fail grading system; at clinical re-entry 47% and during the survey period 23% were using it. Due to the pandemic, 78.2% of CDs assumed new roles or had expanded responsibilities; 51% reported decreased scholarly productivity. CONCLUSIONS: Curricular adaptations occurred in IM clerkships across US medical schools as a result of COVID-19. More research is needed to explore the long-term implications of these changes on medical student education and clinical learning environments.

Roles and Responsibilities of Medicine Subinternship Directors : Medicine Subinternship Director Roles.

BACKGROUND: The internal medicine (IM) subinternship (also referred to as acting internship) plays a crucial part in preparing medical students for residency. The roles, responsibilities, and support provided to subinternship directors have not been described. OBJECTIVE: We sought to describe the current role of IM subinternship directors with respect to their responsibilities, salary support, and reporting structure. DESIGN: Nationally representative, annually recurring thematic survey of IM core clerkship directors with membership in an academic professional association as of September 2017. PARTICIPANTS: A total of 129 core clinical medicine clerkship directors at Liaison Committee on Medical Education fully accredited U.S./U.S.-territory-based medical schools. MAIN MEASURES: Responsibilities, salary support, and reporting structure of subinternship directors. KEY RESULTS: The survey response rate was 83.0% (107/129 medical schools). Fifty-one percent (54/107) of respondents reported overseeing both core clerkship inpatient experiences and/or one or more subinternships. For oversight, 49.1% (28/53) of subinternship directors also reported that they were the clerkship director, 26.4% (14/53) that another faculty member directed all medicine subinternships, and 18.9% (10/53) that each subinternship had its own director. The most frequently reported responsibilities for the subinternship directors were administration, including scheduling, and logistics of student schedules (83.0%, 44/53), course evaluation (81.1%, 43/53), and setting grades 79.2% (42/53). The modal response for estimated FTE per course was 10-20% FTE, with 33.3% (16/48) reporting this level of support and 29.2% (14/54) reporting no FTE support. CONCLUSIONS: The role of the IM subinternship director has become increasingly complex. Since the IM subinternship is critical to preparing students for residency, IM subinternship directors require standard expectations and adequate support. Future studies are needed to determine the appropriate level of support for subinternship directors and to define essential roles and responsibilities.

The Impact of Internal Medicine Clerkship Characteristics and NBME Subject Exams on USMLE Step 2 Clinical Knowledge Exam Performance.

BACKGROUND: Residency program directors will likely emphasize the United States Medical Licensing Exam (USMLE) Step 2 clinical knowledge (CK) exam more during residency application given the recent USMLE Step 1 transition to pass/fail scoring. We examined how internal medicine clerkship characteristics and NBME subject exam scores affect USMLE Step 2 CK performance. DESIGN: The authors used univariable and multivariable generalized estimating equations to determine associations between Step 2 CK performance and internal medicine clerkship characteristics and NBME subject exams. The sample had 21,280 examinees' first Step 2 CK scores for analysis. RESULTS: On multivariable analysis, Step 1 performance (standardized ß = 0.45, p < .001) and NBME medicine subject exam performance (standardized ß = 0.40, p < .001) accounted for approximately 60% of the variance in Step 2 CK performance. Students who completed the internal medicine clerkship last in the academic year scored lower on Step 2 CK (Mdiff = -3.17 p < .001). Students who had a criterion score for passing the NBME medicine subject exam scored higher on Step 2 CK (Mdiff = 1.10, p = .03). There was no association between Step 2 CK performance and other internal medicine clerkship characteristics (all p > 0.05) nor with the total NBME subject exams completed (ß=0.05, p = .78). CONCLUSION: Despite similarities between NBME subject exams and Step 2 CK, the authors did not identify improved Step 2 CK performance for students who had more NBME subject exams. The lack of association of Step 2 CK performance with many internal medicine clerkship characteristics and more NBME subject exams has implications for future clerkship structure and summative assessment. The improved Step 2 CK performance in students that completed their internal medicine clerkship earlier warrants further study given the anticipated increase in emphasis on Step 2 CK.

A is for advocacy: How introducing student advocacy assessment impacts longitudinal integrated clerkship students and clinical supervisors.

To improve health, physicians are increasingly called to advocate. Yet medical schools have only recently focused on health advocacy skill-building. Limited work to date addresses assessing medical student advocacy on behalf of patients. We describe how students and clinical supervisors (CS) in two urban longitudinal integrated clerkships (LIC) experience patient advocacy and how introducing a new advocacy assessment impacts them. Using a thematic approach, we analyzed transcripts of focus groups during 2018-2019. Seventeen of 24 (71%) students and 15 of 21 (71%) CS participated in the focus groups. We describe how students perceive their advocacy role as they accompany the patient, amplify their voice, and facilitate connection. The rationale for advocacy assessment includes that it (1) adds a novel dimension to the written and verbal assessment, (2) drives student learning, (3) aligns with the institutional goal to promote equity, and (4) impacts CS teaching and clinical practice. Challenges are the ambiguity of expectations, pressure to 'perform,' and a moral overlay to advocacy assessment. Findings demonstrate how educational alliances between students and CS and longitudinal relationships between LIC students and patients offer a constructive opportunity for advocacy assessment. We describe suggestions to hone and expand the reach of advocacy assessment.

Moving toward Mastery: Changes in Student Perceptions of Clerkship Assessment with Pass/Fail Grading and Enhanced Feedback.

ProblemClerkship grades contribute to a summative assessment culture in clerkships and can therefore interfere with students' learning. For example, by focusing on summative, tiered clerkship grades, students often discount accompanying feedback that could inform future learning. This case report seeks to explore whether an assessment system intervention which eliminated tiered grades and enhanced feedback was associated with changes in student perceptions of clerkship assessment and perceptions of the clinical learning environment. Intervention: In January 2019, our institution eliminated tiered clerkship grading (honors/pass/fail) for medical students during the core clerkship year and implemented pass/fail clerkship grading along with required twice weekly, work-based assessments for formative feedback. Context: In this single institution, cross-sectional survey study, we collected data from fourth-year medical students one year after an assessment system intervention. The intervention entailed changing from honors/pass/fail to pass/fail grading in all eight core clerkships and implementing an electronic system to record twice-weekly real-time formative work-based assessments. The survey queried student perceptions on the fairness and accuracy of grading and the clinical learning environment-including whether clerkships were mastery- or performance-oriented. We compared responses from students one year after the assessment intervention to those from the class one year before the intervention. Comparisons were made using unpaired, two-tailed t-tests or chi-squared tests as appropriate with Cohen's d for effect size estimation for score differences. Content analysis was used to analyze responses from two open-ended questions about feedback and grading. Impact: Survey response rates were similar before and after intervention (76% (127/168) vs. 72% (118/163), respectively) with no between-group differences in demographics. The after-intervention group showed statistically significant increases in the following factors: "grades are transparent and fair" (Cohen's d = 0.80), "students receive useful feedback" (d = 0.51), and "resident evaluation procedures are fair" (d = 0.40). After-intervention respondents perceived the clerkship learning environment to be more mastery-oriented (d = 0.52), less performance approach-oriented (d = 0.63), and less performance avoid-oriented (d = 0.49). There were no statistical differences in the factors "attending evaluation procedures are fair," "evaluations are accurate," "evaluations are biased," or "perception of stereotype threat." Open-ended questions revealed student recommendations to improve clerkship summary narratives, burden of work-based assessment, and in-person feedback. Lessons Learned: After an assessment system change to pass/fail grading with work-based assessments, we observed moderate to large improvements in student perceptions of clerkship grading and the mastery orientation of the learning environment. Our intervention did not improve perceptions around bias in assessment in clerkships. Other medical schools may consider similar interventions to begin to address student concerns with clerkship assessment and promote a more adaptive learning environment.

Clerkship Roles and Responsibilities in a Rapidly Changing Landscape: a National Survey of Internal Medicine Clerkship Directors.

BACKGROUND: In the rapidly changing landscape of undergraduate medical education (UME), the roles and responsibilities of clerkship directors (CDs) are not clear. OBJECTIVE: To describe the current roles and responsibilities of Internal Medicine CDs. DESIGN: National annual Clerkship Directors in Internal Medicine (CDIM) cross-sectional survey. PARTICIPANTS: One hundred twenty-nine clerkship directors at all Liaison Committee on Medical Education accredited US medical schools with CDIM membership as of September 1, 2017. MAIN MEASURES: Responsibilities of core CDs, including oversight of other faculty, and resources available to CDs including financial support and dedicated time. KEY RESULT: The survey response rate was 83% (107/129). Ninety-four percent of the respondents oversaw the core clerkship inpatient experience, while 47.7% (n = 51) and 5.6% (n = 6) oversaw the outpatient and longitudinal integrated clerkships respectively. In addition to oversight, CDs were responsible for curriculum development, evaluation and grades, remediation, scheduling, student mentoring, and faculty development. Less than one-third of CDs (n = 33) received the recommended 0.5 full-time equivalent (FTE) support for their roles, and 15% (n = 16) had less than 20% FTE support. An average 0.41 FTE (SD .2) was spent in clinical work and 0.20 FTE (SD .21) in administrative duties. Eighty-three percent worked with other faculty who assisted in the oversight of departmental UME experiences, with FTE support varying by role and institution. Thirty-five percent of CDs (n = 38) had a dedicated budget for managing their clerkship. CONCLUSIONS: The responsibilities of CDs have increased in both number and complexity since the dissemination of previous guidelines for expectations of and for CDs in 2003. However, resources available to them have not substantially changed.

Detecting patients with cirrhosis in primary care.

Cirrhosis is a condition that arises as a result of chronic liver damage, typically over many years. It is characterised by fibrosis and nodularity of the liver parenchyma. Cirrhosis interferes with the normal functions of the liver, reducing its ability to produce proteins, which can lead to coagulopathy, low serum albumin and raised bilirubin. The incidence of cirrhosis is rising in the UK, this can primarily be attributed to increasing levels of alcohol consumption and obesity. Mortality from cirrhosis is also rising. Common causes of chronic liver disease include alcohol, non-alcoholic fatty liver disease and chronic viral hepatitis. Nearly half of patients with cirrhosis are asymptomatic. As a result the condition may only be discovered incidentally as a result of abnormalities in liver function tests or imaging of the abdomen performed for other reasons. Alternatively patients may present with signs and symptoms of the complications of cirrhosis e.g. jaundice, ascites, variceal bleeding, hepatic encephalopathy or hepatocellular carcinoma. Detecting patients with cirrhosis in primary care usually relies on identifying common risk factors. Currently, there are no standard criteria for the investigation of patients with suspected cirrhosis. If a patient is suspected of having cirrhosis, most GPs will arrange for blood tests and an ultrasound of the liver to be performed. The gold standard test for the diagnosis of cirrhosis remains a liver biopsy. Staging of liver fibrosis is an important predictor of prognosis and is necessary to guide management.

Structures and Processes of Grading Committees in Internal Medicine Clerkships: Results of a National Survey.

PURPOSE: Clerkship grades are important in the residency selection process but can be influenced by individual bias and grading tendencies. Although clinical competency committees are standard in graduate medical education, in undergraduate medical education, they have not gained widespread traction. This study describes structures and processes of grading committees in internal medicine (IM) clerkships and strategies used to mitigate grading bias. METHOD: From September to December 2022, the Clerkship Directors in Internal Medicine conducted its annual survey of IM core clerkship directors at 140 U.S. and U.S. territory-based medical schools. This study was based on 23 questions about grading committees in IM clerkships. RESULTS: The survey response rate was 80% (n = 112/140). Forty-seven respondents (42%) reported using grading committees in their IM clerkship (median committee size, 7; range, 3-20) (primarily clerkship leadership and faculty). Responsibilities included determining grades for all students (31 [66%]) and students at borderline of failing (30 [64%]), adjudicating on students with professionalism concerns (25 [53%]), and reconciling discordant clinical evaluations (24 [51%]). To mitigate deliberation bias, committees most frequently used multisource assessments (38 [81%]) and adoption of a shared mental model (36 [77%]). Approximately one-third of grading committees "rarely" discussed gender (14 [30%]) and race or ethnicity (15 [32%]), and 7 committees (15%) "never" discussed gender and race or ethnicity. Clerkship directors perceived developing a shared mental model (60 [92%]), promoting consistency (59 [91%]) and transparency (57 [88%]) in the process, mitigating assessment bias (58 [89%]), improving student satisfaction (54 [83%]), and sharing grading responsibility (44 [68%]) as potential benefits. CONCLUSIONS: This study found that grading committees in IM clerkships are not widely used and that existing committees vary in structure and process. These findings highlight an opportunity for medical schools to consider using grading committees to improve grade assignment and address grading inconsistencies.

Changes in Gender and Racial/Ethnic Diversity in US Residency Program Applications From 2018 to 2022.

Background Residency application patterns by gender and race/ethnicity offer important insights about diversity in residency recruitment. It is unknown how the COVID-19 pandemic and virtual interviewing affected these patterns. Objective We hypothesized that the introduction of virtual interviews caused an increase in applications submitted per applicant and that there may be differences by gender and race/ethnicity. Methods We extracted publicly reported Electronic Residency Application Service application data from 2018 to 2022 for 14 residency specialties with 1000 or more applicants in 2022 by self-reported gender and underrepresented in medicine (UIM) status. We compared patterns before and after virtual interviews were introduced in 2021. Results Among 401 480 residency applicants, the average number of applications submitted per applicant increased for all specialties between 2018 and 2022 across gender and race/ethnicity. Across all years, women applied to more programs than men in 5 specialties (dermatology, neurology, obstetrics/gynecology, pediatrics, and surgery), whereas men applied to more programs than women in 3 (anesthesia, family medicine, and physical medicine and rehabilitation). Across all years, non-UIM applicants applied to more programs than UIM applicants in all 14 specialties. There were no clear changes in application patterns by gender and race/ethnicity during in-person versus virtual interview years. Conclusions The average number of applications submitted per applicant increased over time across gender and race/ethnicity. In some specialties, women applied to more programs than men, and in others vice-versa, whereas non-UIM applicants applied to more programs than UIM applicants in all specialties. Virtual interviews did not change these patterns.

Community health education at student-run clinics leads to sustained improvement in patients' hepatitis B knowledge.

While student-run clinics are often important healthcare safety nets for underserved populations, their efficacy for improving patient health knowledge has not been thoroughly explored. From September 2011 to April 2012, we assessed patients' retention of hepatitis B virus (HBV) knowledge after receiving student-led education at two student-run HBV screening and vaccination clinics. Patient education was provided by trained first and second-year medical, nursing, and pharmacy students, aided by a script and interpreters. Patient knowledge of HBV was evaluated at three points: before education, after the initial visit, and at one-month follow-up. Student-led education produced improved knowledge of HBV transmission, prevention, and management, which was retained 1 month after education for 52 patients tracked through time. Mean scores on an HBV knowledge survey improved from 56.4 % (SD = 15.2 %) at baseline to 66.6 % (SD = 15.1 %) after education, and 68.3 % (SD = 15.2 %) after one month. There was a statistically significant difference between the first and second (paired T test, p < 0.001) and the first and third tests (paired T test, p < 0.001), but no difference between the second and third tests (paired T test, p = 0.45). Multivariate analysis demonstrated that retention was correlated with patient educational background but independent of patient age, gender, income, primary language and number of years lived in the United States. Our study suggests that trained health professional students can effectively impart health knowledge that is retained by patients for at least 1 month. These results warrant consideration of student-led educational sessions at SRCs as a promising community health education model.

Patient views of continuity relationships with medical students.

INTRODUCTION: Continuity relationships between students and patients, that occur in a longitudinal integrated clerkship (LIC), enrich medical students' opportunities to learn from patients and provide patient-centered care. Patient preferences for continuity with a primary provider are well-documented, but little is known about patients' experiences of continuity with students. This study examines patients' perception of continuity with and care received by students. METHODS: This qualitative study uses data from semi-structured interviews with 32 patients of LIC students at an academic medical center. Data were analyzed for themes about continuity and experiences of care provided by students. RESULTS: Patients valued relationships with students over time and across settings. Students' contributions to their care included enhanced access to and coordination of care, communication, patient education and wellbeing. Patients with substantial continuity and/or who were moderately or severely ill described their student in a physician-like role more frequently than other patients. Patients appreciated patient-centered attitudes and behaviors in their students. CONCLUSION: Patients value continuity relationships with students, akin to that described between patients and their physicians. Patients described a variety of ways in which students enhanced their care and assumed a physician-like role. These patient perceptions support the concept of mutually beneficial relationships between students and patients.

Strategies for Advancing Equity in Frontline Clinical Assessment.

Educational equity in medicine cannot be achieved without addressing assessment bias. Assessment bias in health professions education is prevalent and has extensive implications for learners and, ultimately, the health care system. Medical schools and educators desire to minimize assessment bias, but there is no current consensus on effective approaches. Frontline teaching faculty have the opportunity to mitigate bias in clinical assessment in real time. Based on their experiences as educators, the authors created a case study about a student to illustrate ways bias affects learner assessment. In this paper, the authors use their case study to provide faculty with evidence-based approaches to mitigate bias and promote equity in clinical assessment. They focus on 3 components of equity in assessment: contextual equity, intrinsic equity, and instrumental equity. To address contextual equity, or the environment in which learners are assessed, the authors recommend building a learning environment that promotes equity and psychological safety, understanding the learners' contexts, and undertaking implicit bias training. Intrinsic equity, centered on the tools and practices used during assessment, can be promoted by using competency-based, structured assessment methods and employing frequent, direct observation to assess multiple domains. Instrumental equity, focused on communication and how assessments are used, includes specific, actionable feedback to support growth and use of competency-based narrative descriptors in assessments. Using these strategies, frontline clinical faculty members can actively promote equity in assessment and support the growth of a diverse health care workforce.

Medical student advising during virtual residency recruitment: results of a national survey of internal medicine clerkship and sub-internship directors.

INTRODUCTION: The residency application process is a critical time for medical students. The COVID-19 pandemic prompted changes to the residency recruitment procedures with the conversion of interviews to a virtual format. For medical school advisors guiding students on an all-virtual residency application process brought uncertainty to their advising practices. Thus, this study aimed to identify advising practices during the 2021 virtual application cycle. METHODS: We administered an IRB-exempt national survey through the Clerkship Directors in Internal Medicine to 186 internal medicine core/co-/associate/assistant clerkship directors and sub-internship directors representing 140 Liaison Committee on Medical Education-accredited U.S./U.S.-territory-based medical schools in spring 2021. The 23-question survey was designed and pilot-tested by faculty-educators and leaders with expertise in undergraduate medical education. Data analysis included paired t- and z-tests and thematic analysis of open-ended questions. RESULTS: The institutional response rate was 67% (93/140) and individual rate 55% (103/186). Half of the respondents felt prepared/very prepared (40% and 13% respectively) for their advising roles. Compared to pre-pandemic cycles, respondents advised a typical student in the middle-third of their class at their institution to apply to more residency programs (mean 24 programs vs 20, p < 0.001) and accept more interviews (mean 14 interviews vs 12, p < 0.001). Sixty-three percent (64/101) of respondents spent more time on student advising; 51% (51/101) reported more students asked them for informal advice. Fifty-nine percent (60/101) of respondents reported their advisees were able to assess a residency program 'somewhat well;' 31% (31/101) expressed that residency recruitment should remain entirely virtual in the future. CONCLUSION: The transition to virtual residency recruitment due to COVID-19 prompted advising practices that may have contributed to application inflation and increased advising workload. Future studies should explore longitudinal outcomes of virtual interviews on student success to guide best practices in how to advise students during residency recruitment.

Reviewing Internal Medicine Clerkship Grading Through a Proequity Lens: Results of a National Survey.

PURPOSE: Equity in assessment and grading has become imperative across medical education. Although strategies to promote equity exist, there may be variable penetrance across institutions. The objectives of this study were to identify strategies internal medicine (IM) clerkship directors (CDs) use to reduce inequities in assessment and grading and explore IM CDs' perceptions of factors that impede or facilitate the implementation of these strategies. METHOD: From October to December 2021, the Clerkship Directors in Internal Medicine of the Alliance for Academic Internal Medicine conducted its annual survey of IM core CDs at 137 U.S. and U.S. territory-based medical schools. This study is based on 23 questions from the survey about equity in IM clerkship assessment and grading. RESULTS: The survey response rate was 73.0% (100 of 137 medical school CDs). Use of recommended evidence-based strategies to promote equity in clerkship assessment and grading varied among IM clerkships. Only 30 respondents (30.0%) reported that their clerkships had incorporated faculty development on implicit bias for clinical supervisors of students; 31 (31.0%) provided education to faculty on how to write narrative assessments that minimize bias. Forty respondents (40.0%) provided guidance to clerkship graders on how to minimize bias when writing final IM clerkship summaries, and 41 (41.0%) used grading committees to determine IM clerkship grades. Twenty-three CDs (23.0%) received formal education by their institution on how to generate clerkship grades and summaries in a way that minimized bias. CONCLUSIONS: This national survey found variability among medical schools in the application of evidence-based strategies to promote equity in assessment and grading within their IM clerkships. Opportunities exist to adopt and optimize proequity grading strategies, including development of programs that address bias in clerkship assessment and grading, reevaluation of the weight of standardized knowledge exam scores on grades, and implementation of grading committees.

A comparison of two methods of teaching reflective ability in Year 3 medical students.

CONTEXT: Little is known about best practices for teaching and learning reflection. We hypothesised that reflective ability scores on written reflections would be higher in students using critical reflection guidelines, or receiving feedback on reflective skill in addition to reflection content, or both, compared with those in students who received only a definition of reflection or feedback on reflection content alone. METHODS: Using a 2 (guidelines) × 2 (feedback) × 2 (time) design, we randomly assigned half of our sample of 149 Year 3 medical students to receive critical reflection guidelines and the other half to receive only a definition of critical reflection. We then randomly divided both groups in half again so that one half of each group received feedback on both the content and reflective ability in their reflections, and the other received content feedback alone. The learners' performance was measured on the first and third written reflections of the academic year using a previously validated scoring rubric. We calculated descriptive statistics for the reflection scores and conducted a repeated-measures analysis of variance with two between-groups factors, guidelines and feedback, and one within-group factor, occasion, using the measure of reflective ability as the dependent variable. RESULTS: We failed to find a significant interaction between guidelines and feedback (F = 0.51, d.f. = 1, 145, p = 0.48). However, the provision of critical reflection guidelines improved reflective ability compared with the provision of a definition of critical reflection only (F = 147.1, d.f. = 1, 145, p < 0.001). Feedback also improved reflective ability, but only when it covered reflective skill in addition to content (F = 6.5, d.f. = 1, 145, p = 0.012). CONCLUSIONS: We found that the provision of critical reflection guidelines improved performance and that feedback on both content and reflective ability also improved performance. Our study demonstrates that teaching learners the characteristics of deeper, more effective reflection and helping them to acquire the skills they need to reflect well improves their reflective ability as measured by performance on reflective exercises.