The Value of Supporting Education Research in the Academic Environment: The Hospital for Special Surgery Academy of Rheumatology Medical Educators.

PROBLEM: Medical education academies have been instrumental in providing greater recognition of and promotion for clinician-educators. However, producing education scholarship is essential for clinician-scholar-educator career advancement. Grant funding for education research and protected time to produce scholarship are still lacking for interested physicians, in part due to institutional budget constraints and competing priorities. APPROACH: The Hospital for Special Surgery Academy of Rheumatology Medical Educators was founded in 2011 to promote education scholarship through grants awarded to educators interested in research. Educators were asked to submit proposals aimed at the development of new teaching programs and curricular change. Selected applicants received up to $50,000 per year for one year. Grant money was obtained through directed fundraising from donors. Information from annual grant updates and survey responses from grant recipients in 2017 were used to assess the academy's effectiveness. OUTCOMES: Since 2012, 32 grants have been awarded, totaling $954,045 in funding. Recipients have produced national meeting abstracts, posters, oral presentations, and manuscripts and created unique curricula and electronic learning tools for medical students, residents, fellows, faculty, and patients. Four educators with demonstrated interest and research outcomes were identified during the pilot and received additional funding and support from a dedicated education research assistant. NEXT STEPS: The academy and the innovations grants program highlight the talents of under-supported and under-recognized teaching faculty by allowing them to distinguish themselves academically as clinician-scholar-educators. The success of these educators emphasizes the clear advantages of a formalized structure to achieve the hospital's education goals. Next steps include providing support for a rheumatology fellow to develop an education research career rather than one in bench, clinical, or translational research.

Validity and Reliability of Patient Reported Outcomes Measurement Information System Computerized Adaptive Tests in Systemic Lupus Erythematosus.

OBJECTIVE: The aims of this study were to assess the construct validity and the test-retest reliability of Patient Reported Outcomes Measurement Information System (PROMIS) computerized adaptive tests (CAT) in patients with systemic lupus erythematosus (SLE). METHODS: Adults with SLE completed the Medical Outcomes Study Short Form-36, LupusQoL-US version ("legacy instruments"), and 14 selected PROMIS CAT. Using Spearman correlations, PROMIS CAT were compared with similar domains measured with legacy instruments. CAT were also correlated with the Safety of Estrogens in Lupus Erythematosus National Assessment-Systemic Lupus Erythematosus Disease Activity Index (SELENA-SLEDAI) disease activity and the Systemic Lupus International Collaborating Clinics/American College of Rheumatology Damage Index (SDI) scores. Test-retest reliability was evaluated using ICC. RESULTS: There were 204 outpatients with SLE enrolled in the study and 162 completed a retest. PROMIS CAT showed good performance characteristics and moderate to strong correlations with similar domains in the 2 legacy instruments (r = -0.49 to 0.86, p < 0.001). However, correlations between PROMIS CAT and the SELENA-SLEDAI disease activity and SDI were generally weak and statistically insignificant. PROMIS CAT test-retest ICC were good to excellent, ranging from 0.72 to 0.88. CONCLUSION: To our knowledge, these data are the first to show that PROMIS CAT are valid and reliable for many SLE-relevant domains. Importantly, PROMIS scores did not correlate well with physician-derived measures. This disconnect between objective signs and symptoms and the subjective patient disease experience underscores the crucial need to integrate patient-reported outcomes into clinical care to ensure optimal disease management.

Rheumatology Research Foundation Clinician Scholar Educator Award: Fifteen Years Promoting Rheumatology Educators and Education.

OBJECTIVE: The Rheumatology Research Foundation's Clinician Scholar Educator (CSE) award is a 3-year career development award supporting medical education research while providing opportunities for mentorship and collaboration. Our objective was to document the individual and institutional impact of the award since its inception, as well as its promise to strengthen the subspecialty of rheumatology. METHODS: All 60 CSE Award recipients were surveyed periodically. Fifty-six of those 60 awardees (90%) responded to requests for survey information that included post-award activities, promotions, and further funding. Data were also collected from yearly written progress reports for each grant. RESULTS: Of the total CSE recipients to date, 48 of 60 (80%) are adult rheumatologists, 11 of 60 (18%) are pediatric rheumatologists, and 1 is an adult and pediatric rheumatologist. Two-thirds of survey respondents spend up to 30% of their total time in educational activities, and one-third spend greater than 30%. Thirty-one of the 60 CSE recipients (52%) have published a total of 86 medical education papers. Twenty-six of 52 (50%) had received an academic promotion following the award. Eleven awardees earned advanced degrees. CONCLUSION: We describe the creation and evolution of a grant program from a medical subspecialty society foundation and the impact on producing education research, individual identity formation, and ongoing support for educators. This community of rheumatology scholar educators now serves as an important resource at the national level for the American College of Rheumatology and its membership. We believe that this grant may serve as a model for other medical societies that want to promote education scholarship and leadership within their specialties.

Fellow use of medical jargon correlates inversely with patient and observer perceptions of professionalism: results of a rheumatology OSCE (ROSCE) using challenging patient scenarios.

The NYC Rheumatology Objective Structured Clinical Examination (NYC-ROSCE) is held annually to assess fellow competencies. We recently redesigned our OSCE to better assess subspecialty trainee communication skills and professionalism by developing scenarios in which the patients encountered were psychosocially or medically complex. The objective of this study is to identify which types of verbal and non-verbal skills are most important in the perception of professionalism in the patient-physician interaction. The 2012-2013 NYC-ROSCEs included a total of 53 fellows: 55 MD evaluators from 7 NYC rheumatology training programs (Hospital for Special Surgery-Weill Cornell (HSS), SUNY/Downstate, NYU, Einstein, Columbia, Mount Sinai, and North Shore/Long Island Jewish (NSLIJ)), and 55 professional actors/standardized patients participated in 5 stations. Quantitative fellow performance assessments were made on the following: maintaining composure; partnering with the patient; honesty; professionalism; empathy; and accountability. Free-text comments were solicited regarding specific strengths and weaknesses. A total of 53/53 eligible (100 %) fellows were evaluated. MD evaluators rated fellows lower for professionalism than did the standardized patients (6.8 ± 0.6 vs. 7.4 ± 0.8, p = 0.05), suggesting that physicians and patients view professionalism somewhat differently. Fellow self-evaluations for professionalism (6.6 ± 1.2) were concordant with those of the MD evaluators. Ratings of empathy by fellows themselves (6.6 ± 1.0), MD evaluators (6.6 ± 0.7), and standardized patients (6.6 ± 1.1) agreed closely. Jargon use, frequently cited by evaluators, showed a moderate association with lower professionalism ratings by both MD evaluators and patients. Psychosocially challenging patient encounters in the NYC-ROSCE permitted critical assessment of the patient-centered traits contributing to impressions of professionalism and indicate that limiting medical jargon is an important component of the competency of professionalism.

Building a rheumatology education academy: insights from assessment of needs during a rheumatology division retreat.

OBJECTIVE: To implement a rheumatology department education retreat to systematically identify and address the key factors necessary to improve medical education in our division in preparation for developing a rheumatology academy. METHODS: The Hospital for Special Surgery organized a retreat for the Rheumatology Department aimed at (1) providing formal didactics and (2) assessing participants' self-reported skills and interest in education with the goal of directing this information toward formalizing improvement. In a mixed-methods study design, faculty and fellows in the Division of Rheumatology were surveyed online pre- and post-retreat regarding various aspects of the current education program, their teaching abilities, interest and time spent in teaching, divisional resources allocated, and how education is valued. RESULTS: Enthusiasm for teaching was high before and rose further after the retreat. Confidence in abilities was higher than expected before but fell afterward. Many noted that the lack of specific feedback on teaching skills and useful metrics to assess performance prevented the achievement of educational excellence. Most responding felt lack of time, knowledge of how to teach well, and resources prevented them from making greater commitments to educational endeavors and participating fully and effectively in the department's teaching activities. CONCLUSION: While most rheumatology faculty members want to improve as teachers, they know neither where their educational strengths and weaknesses lie nor where or how to begin to change their teaching abilities. The key elements for an academy would thus be an educational environment that elevates the quality of teaching throughout the division and promotes teaching careers and education research, and raises the importance and quality of teaching to equivalence with clinical care and research.

The New York City Rheumatology Objective Structured Clinical Examination: five-year data demonstrates its validity, usefulness as a unique rating tool, objectivity, and sensitivity to change.

OBJECTIVE: Traditional means of testing rheumatology fellows do not adequately assess some skills that are required to practice medicine well, such as humanistic qualities, communication skills, or professionalism. Institution of the New York City Rheumatology Objective Structured Clinical Examination (ROSCE) and our sequential 5 years of experience have provided us with a unique opportunity to assess its usefulness and objectivity as a rheumatology assessment tool. METHODS: Prior to taking the examination, all of the fellows were rated by their program directors. Fellows from the participating institutions then underwent a multistation patient-interactive examination observed and rated by patient actors and faculty raters. Assessments were recorded by all of the participants using separate but overlapping sets of instruments testing the Accreditation Council of Graduate Medical Education (ACGME) core competencies of patient care, interpersonal and communication skills, professionalism, and overall medical knowledge. RESULTS: Although the program directors tended to rate their fellows more highly than the ROSCE raters, typically there was agreement between the program directors and the ROSCE faculty in distinguishing between the highest- and lowest- performing fellows. The ROSCE faculty and patient actor assessments of individual trainees were notable for a high degree of concordance, both quantitatively and qualitatively. CONCLUSION: The ROSCE provides a unique opportunity to obtain a patient-centered assessment of fellows' ACGME-mandated competencies that traditional knowledge-based examinations, such as the rheumatology in-service examination, cannot measure. The ability of the ROSCE to provide a well-rounded and objective assessment suggests that it should be considered an important component of the rheumatology training director's toolbox.