Virtual Learning and Assessment in Rheumatology Fellowship Training: Objective Structured Clinical Examination Revisited.

OBJECTIVE: With the onset of the COVID-19 pandemic, an annual multi-institutional face-to-face rheumatology objective structured clinical examination (ROSCE) was transformed into a virtual format. The educational goals of the virtual ROSCE (vROSCE) were to reproduce the educational value of the previous in-person ROSCE, providing a valuable formative assessment of rheumatology training activities encompassing the 6 Accreditation Council for Graduate Medical Education (ACGME) core competencies for fellows-in-training (FITs). This article describes the novel design, feasibility, and stakeholder value of a vROSCE. METHODS: Through an established collaboration of 5 rheumatology fellowship training programs, in February 2021, a vROSCE was created and conducted using a Zoom platform. Station development included learning objectives, FIT instructions, faculty proctor instructions, and a checklist by which to provide structured formative feedback. An anonymous, optional web-based survey was sent to FIT participants to evaluate the experience. RESULTS: Twenty-three rheumatology FITs from 5 institutions successfully rotated through 6 stations in the vROSCE. Immediate feedback was given to each FIT using standardized rubrics structured around ACGME core competencies. A total of 65% of FITs (15 of 23) responded to the survey, and 93% of survey respondents agreed or strongly agreed that the vROSCE was a helpful educational activity and identified individualized opportunities for improvement. CONCLUSION: A vROSCE is an innovative, feasible, valuable, and well-received educational technology tool. The vROSCE enriched rheumatology FITs' education and offered collaborative learning experiences across institutions.

Incorporating Telemedicine in Rheumatology Fellowship Training Programs: Needs Assessment, Curricular Intervention, and Evaluation.

OBJECTIVE: To increase the confidence of rheumatology fellows in training (FITs) in delivering virtual care (VC) and prepare them for independent practice, we developed educational materials addressing gaps in their skills. METHODS: We identified gaps in telemedicine skills based on FIT performance in a virtual rheumatology objective structured clinical examination (vROSCE) station on VC delivery using video teleconference technology and survey (survey 1) responses. We created educational materials including videos of "mediocre" and "excellent" VC examples, discussion/reflection questions, and a document summarizing key practices. We measured change in the confidence levels of FITs for delivering VC with a post-intervention survey (survey 2). RESULTS: Thirty-seven FITs (19 first-year, 18 second- plus third-year fellows) from 7 rheumatology fellowship training programs participated in a vROSCE and demonstrated gaps in skills mapping to several Rheumatology Telehealth Competency domains. Confidence levels of FITs improved significantly from survey 1 to survey 2 for 22 of 34 (65%) questions. All participating FITs found the educational materials helpful for learning and reflecting on their own VC practice; 18 FITs (64%) qualified usefulness as "moderately" or "a lot." Through surveying, 17 FITs (61%) reported implementing skills from the instructional videos into VC visits. CONCLUSION: Continually assessing our learners' needs and creating educational materials addressing gaps in training are requisite. Using a vROSCE station, needs assessments, and targeted learning with videos and discussion-guidance materials enhanced the confidence level of FITs in VC delivery. It is imperative to incorporate VC delivery into fellowship training program curricula to ensure breadth in skills, attitudes, and knowledge of new entrants into the rheumatology workforce.

Rheumatology Research Foundation Clinician Scholar Educator Award: Fifteen Years Promoting Rheumatology Educators and Education.

OBJECTIVE: The Rheumatology Research Foundation's Clinician Scholar Educator (CSE) award is a 3-year career development award supporting medical education research while providing opportunities for mentorship and collaboration. Our objective was to document the individual and institutional impact of the award since its inception, as well as its promise to strengthen the subspecialty of rheumatology. METHODS: All 60 CSE Award recipients were surveyed periodically. Fifty-six of those 60 awardees (90%) responded to requests for survey information that included post-award activities, promotions, and further funding. Data were also collected from yearly written progress reports for each grant. RESULTS: Of the total CSE recipients to date, 48 of 60 (80%) are adult rheumatologists, 11 of 60 (18%) are pediatric rheumatologists, and 1 is an adult and pediatric rheumatologist. Two-thirds of survey respondents spend up to 30% of their total time in educational activities, and one-third spend greater than 30%. Thirty-one of the 60 CSE recipients (52%) have published a total of 86 medical education papers. Twenty-six of 52 (50%) had received an academic promotion following the award. Eleven awardees earned advanced degrees. CONCLUSION: We describe the creation and evolution of a grant program from a medical subspecialty society foundation and the impact on producing education research, individual identity formation, and ongoing support for educators. This community of rheumatology scholar educators now serves as an important resource at the national level for the American College of Rheumatology and its membership. We believe that this grant may serve as a model for other medical societies that want to promote education scholarship and leadership within their specialties.

Fellow As Teacher Curriculum: Improving Rheumatology Fellows' Teaching Skills During Inpatient Consultation.

OBJECTIVE: Enhancing rheumatology fellows' teaching skills in the setting of inpatient consultation may have a broad positive impact. Such efforts may improve fellows' clinical skills and overall patient care. Most importantly, effective resident-fellow teaching interactions may not only increase residents' knowledge of rheumatology but may influence their career choice. However, a number of barriers to the resident-fellow teaching interaction have been identified, including fellows' teaching skills. We developed the Fellow As Clinical Teacher (FACT) curriculum in order to enhance fellows' teaching skills during inpatient consultation. METHODS: The FACT curriculum was delivered in two 45-minute workshops during the 3-day Winter Symposium of the Carolinas Fellows Collaborative. We evaluated its effect with self-assessment surveys and fellow performance on the objective structured teaching exercise (OSTE) before and after participation in the curriculum. RESULTS: Nineteen fellows from 4 rheumatology training programs participated in the pre- and post-curriculum OSTEs and 18 fellows completed pre- and post-curriculum surveys. OSTE scores improved on 5 of the 8 items assessed, and the total OSTE score improved as well (34.7 versus 29.5; P < 0.01) after the FACT curriculum. Fellows' self-assessment of their teaching skills and intent to teach during consultation also increased after participation in the curriculum. CONCLUSION: The FACT curriculum, focused on teaching during consultation, improved fellows' teaching skills and attitudes toward teaching. Improving and increasing fellow teaching, particularly in the consultation setting, may impact patient care, resident and fellow learning, and teaching skills of future faculty, and could potentially influence residents' career choice.

Expert panel consensus on assessment checklists for a rheumatology objective structured clinical examination.

OBJECTIVE: While several regional fellowship groups conduct rheumatology objective structured clinical examinations (ROSCEs), none have been validated for use across programs. We aimed to establish agreement among subspecialty experts regarding checklist items for several ROSCE stations. METHODS: We administered a 1-round survey to assess the importance of 173 assessment checklist items for 11 possible ROSCE stations. We e-mailed the survey to 127 rheumatology educators from across the US. Participants rated each item's importance on a 5-point Likert scale (1 = not important to 5 = very important). Consensus for high importance was predefined as a lower bound of the 95% confidence interval ≥4.0. RESULTS: Twenty-five individuals (20%) completed the expert panel survey. A total of 133 of the 173 items (77%) met statistical cutoff for consensus to retain. Several items that had population means of ≥4.0 but did not meet the predetermined definition for consensus were rejected. The percentage of retained items for individual stations ranged from 24% to 100%; all items were retained for core elements of patient counseling and radiograph interpretation tasks. Only 24% of items were retained for a rehabilitation medicine station and 60% for a microscope use/synovial fluid analysis station. CONCLUSION: This single-round expert panel survey established consensus on 133 items to assess on 11 proposed ROSCE stations. The method used in this study, which can engage a diverse geographic representation and employs rigorous statistical methods to establish checklist content agreement, can be used in any medical field.

Competency-based goals, objectives, and linked evaluations for rheumatology training programs: a standardized template of learning activities from the Carolinas Fellows Collaborative.

OBJECTIVE: American Council on Graduate Medical Education program requirements mandate that rheumatology training programs have written goals, objectives, and performance evaluations for each learning activity. Since learning activities are similar across rheumatology programs, we aimed to create competency-based goals and objectives (CBGO) and evaluations that would be generalizable nationally. METHODS: Through an established collaboration of the 4 training programs' directors in North Carolina and South Carolina, we collaboratively composed CBGO and evaluations for each learning activity for rheumatology training programs. CBGO and linked evaluations were written using appropriate verbs based on Bloom's taxonomy. Draft documents were peer reviewed by faculty at the 4 institutions and by members of the American College of Rheumatology (ACR) Clinician Scholar Educator Group. RESULTS: We completed templates of CBGO for core and elective rotations and conferences. Templates detail progressive fellow performance improvement appropriate to educational level. Specific CBGO are mirrored in learning activity evaluations. Templates are easily modified to fit individual program attributes, have been successfully implemented by our 4 programs, and have proven their value in 4 residency review committee reviews. CONCLUSION: We propose adoption of these template CBGO by the ACR, with access available to all rheumatology training programs. Evaluation forms that exactly reflect stated objectives ensure that trainees are assessed using standardized measures and that trainees are aware of the learning expectations. The objectives mirrored in the evaluations closely align with the proposed milestones for internal medicine training, and will therefore be a useful starting point for creating these milestones in rheumatology.