Psychometric Properties of Entrustable Professional Activity-Based Objective Structured Clinical Examinations During Transition From Undergraduate to Graduate Medical Education: A Generalizability Study.

PURPOSE: The objective structured clinical examination (OSCE) assesses clinical competence in health sciences education. There is little research regarding the reliability and validity of using an OSCE during the transition from undergraduate to graduate medical education. The goal of this study was to measure the reliability of a unique 2-rater entrustable professional activity (EPA)-based OSCE format for transition to internship using generalizability theory for estimating reliability. METHOD: During the 2018 to 2022 academic years, 5 cohorts of interns (n = 230) at the University of Iowa Hospital and Clinics participated in a 6-station OSCE assessment delivered during orientation. A univariate and multivariate generalizability study (G study) was conducted on the scores generated by the 3 cases in the orientation OSCE that shared the 2-rater format. This analysis was supplemented with an associated decision study (D study). RESULTS: The univariate G study for the cases that used a simulated patient and a faculty rater demonstrated that this OSCE generated a moderately reliable score with 3 cases. The D study showed that increasing the OCSE to 12 cases yielded a mean score reliable enough (G = 0.76) for making high-stakes normative decisions regarding remediation and readiness to practice. The universe score correlation between 2 types of raters was 0.398. The faculty ratings displayed a larger proportion of universe (true) score variance and yielded a more reliable (G = 0.433) score compared with the standardized patient ratings (G = 0.337). CONCLUSIONS: This study provides insight into the development of an EPA-based OSCE. The univariate G study demonstrated that when using the 2 rater types, this assessment could generate a moderately reliable score with 3 cases. The multivariate G study showed that the 2 types of raters assessed different aspects of clinical skills and faculty raters were more reliable.

Learning from the early careers of master clinicians.

BACKGROUND: Master clinicians are recognized as multidimensional experts in clinical medicine. Studying their formative clinical activities could generate insights to guide medical trainees and early career clinicians. OBJECTIVES: To investigate which early career activities were adopted more commonly by master clinicians than their matched peers and to characterize master clinicians' early career activities across institutions and specialties. SUBJECTS AND METHODS: We surveyed master clinicians at seven medical centres about their early career activities. For master clinicians in the Department of Medicine (DOM), we also surveyed matched internist peers. RESULTS: Of 150 master clinician respondents, 65% were internists (DOM); 35% practiced in other specialties. Compared to their internist peers, there was a trend toward internist master clinicians reading more about their patients' conditions (6.0 vs. 4.8 h per week), reading more case reports (4.0 vs. 2.1 per month), engaging in more frequent teaching duties and devoting less time to research. CONCLUSIONS: The early career activities identified in this study can be adopted by clinicians pursuing clinical excellence and promoted by training programs that seek to foster life-long learning.

Diurnal temperature variation and the implications for diagnosis and infectious disease screening: a population-based study.

OBJECTIVES: Fevers have been used as a marker of disease for hundreds of years and are frequently used for disease screening. However, body temperature varies over the course of a day and across individual characteristics; such variation may limit the detection of febrile episodes complicating the diagnostic process. Our objective was to describe individual variation in diurnal temperature patterns during episodes of febrile activity using millions of recorded temperatures and evaluate the probability of recording a fever by sex and for different age groups. METHODS: We use timestamped deidentified temperature readings from thermometers across the US to construct illness episodes where continuous periods of activity in a single user included a febrile reading. We model the mean temperature recorded and probability of registering a fever across the course of a day using sinusoidal regression models while accounting for user age and sex. We then estimate the probability of recording a fever by time of day for children, working-age adults, and older adults. RESULTS: We find wide variation in body temperatures over the course of a day and across individual characteristics. The diurnal temperature pattern differed between men and women, and average temperatures declined for older age groups. The likelihood of detecting a fever varied widely by the time of day and by an individual's age or sex. CONCLUSIONS: Time of day and demographics should be considered when using body temperatures for diagnostic or screening purposes. Our results demonstrate the importance of follow-up thermometry readings if infectious diseases are suspected.

Let the Program Evaluation Committee SOAR: Applying the Strengths, Opportunities, Aspirations, and Results (SOAR) Framework for Program Evaluation.

OBJECTIVE: Annual program evaluations are important activities of all graduate medical education programs. Although the Accreditation Council for Graduate Medical Education provides general guidelines, there is substantial scope for educational innovation. Strengths, opportunities, aspirations, and results (SOAR) is a strengths-based framework for strategic planning. Because SOAR emphasizes positivity and engagement, it is an appealing framework for evaluating graduate medical education programs. Our objective was to demonstrate the feasibility and acceptability of SOAR in a program evaluation committee of a fellowship program to generate strategic initiatives. METHODS: The authors used the four steps of SOAR within the program evaluation committee in 2022. Interviewers collected positive stories to understand program strengths. Then, rapid ideation was used to translate strengths into opportunities. These opportunities were condensed and refined for fellows to assess how well they align with aspirations. The ones that aligned best with aspirations were prioritized for implementation. Results were monitored through a scorecard based on specific, measurable, achievable, relevant, and time-bound (SMART) goals every month. RESULTS: Of 15 divisional members, 11 participated (73.3%). Five major strengths were identified: supportive environment, variety of cases, scheduling flexibility, integration with larger networks, and multidisciplinary collaboration. These 5 yielded 15 opportunities, which were refined and condensed to 9. Four were selected for implementation: scholarly works accountability group, hybrid-flex curriculum, fellowship weekly huddles, and structured electives. Scorecards have shown successful implementation during a 4-month period. CONCLUSIONS: SOAR is an innovative and feasible approach to program evaluation that uses trainee engagement to translate and synergize existing program strengths into actionable program improvement.

Neuromuscular and gene signaling responses to passive whole-body heat stress in young adults.

This study aimed to investigate whether acute passive heat stress 1) decreases muscle Maximal Voluntary Contraction (MVC); 2) increases peripheral muscle fatigue; 3) increases spinal cord excitability, and 4) increases key skeletal muscle gene signaling pathways in skeletal muscle. Examining the biological and physiological markers underlying passive heat stress will assist us in understanding the potential therapeutic benefits. MVCs, muscle fatigue, spinal cord excitability, and gene signaling were examined after control or whole body heat stress in an environmental chamber (heat; 82 °C, 10% humidity for 30 min). Heart Rate (HR), an indicator of stress response, was correlated to muscle fatigue in the heat group (R = 0.59; p < 0.05) but was not correlated to MVC, twitch potentiation, and H reflex suppression. Sixty-one genes were differentially expressed after heat (41 genes >1.5-fold induced; 20 < 0.667 fold repressed). A strong correlation emerged between the session type (control or heat) and principal components (PC1) (R = 0.82; p < 0.005). Cell Signal Transduction, Metabolism, Gene Expression and Transcription, Immune System, DNA Repair, and Metabolism of Proteins were pathway domains with the largest number of genes regulated after acute whole body heat stress. Acute whole-body heat stress may offer a physiological stimulus for people with a limited capacity to exercise.

Healthcare educational debt in the united states: unequal economic impact within interprofessional team members.

BACKGROUND: Advancing healthcare access and quality for underserved populations requires a diverse, culturally competent interprofessional workforce. However, high educational debt may influence career choice of healthcare professionals. In the United States, health professions lack insight into the maximum educational debt that can be supported by current entry-level salaries. The purpose of this interprofessional economic analysis was to examine whether average educational debt for US healthcare graduates is supportable by entry-level salaries. Additionally, the study explored whether trainees from minoritized backgrounds graduate with more educational debt than their peers in physical therapy. METHODS: The study modeled maximum educational debt service ratios for 12 healthcare professions and 6 physician specialties, incorporating profession-specific estimates of entry-level salary, salary growth, national average debt, and 4 loan repayment scenarios offered by the US Department of Education Office of Student Financial Aid. Net present value (NPV) provided an estimate for lifetime "economic power" for the modeled careers. The study used a unique data source available from a single profession (physical therapy, N = 4,954) to examine whether educational debt thresholds based on the repayment model varied between minoritized groups and non-minoritized peers. RESULTS: High salary physician specialties (e.g. obstetrics/gynecology, surgery) and professions without graduate debt (e.g. registered nurse) met debt ratio targets under any repayment plan. Professions with strong salary growth and moderate debt (e.g. physician assistant) required extended repayment plans but had high career NPV. Careers with low salary growth and high debt relative to salary (e.g. physical therapy) had career NPV at the lowest range of modeled professions. 29% of physical therapy students graduated with more debt than could be supported by entry-level salaries. Physical therapy students from minoritized groups graduated with 10-30% more debt than their non-minoritized peers. CONCLUSIONS: Graduates from most healthcare professions required extended repayment plans (higher interest) to meet debt ratio benchmarks. For several healthcare professions, low debt relative to salary protected career NPV. Students from minoritized groups incurred higher debt than their peers in physical therapy.

Reimagining the Rheumatology Fellowship Interview: Using Participatory Design-Thinking Process to Create a More Applicant-Centered Experience.

OBJECTIVE: Design thinking is a creative problem-solving process used to better understand users' needs and experiences so that a product or service can be improved. Its emphasis on empathy, iterative prototyping, and participatory collaboration make it an ideal methodology for innovation in medical education. We apply this framework to the virtual rheumatology fellowship interview process so that interviews can become more applicant centered. METHODS: This educational quality improvement project uses a design-thinking framework to identify opportunities and challenges for rheumatology fellowship applicants. The investigators use the 5-step process (Empathize, Define, Ideate, Prototype, Test) and incorporate rapid qualitative analysis of semistructured interviews to innovate the interview experience. The iterative and collaborative nature of this process has empowered participants to codesign an applicant-centered interview experience. RESULTS: Interviews with fellowship applicants (n = 9), fellow physicians (n = 4), and faculty members (n = 3) identified three major dynamics of the interview process: (1) Is it a safe environment to ask questions? (2) How do I exchange information effectively? and (3) How do I fit all these data into the bigger picture? Creative brainstorming techniques at a series of three workshops yielded four prototypes emphasizing customization, hybridization, facilitation, and preparation. A finalized applicant-centered interview template was devised in preparation for the 2023-2024 application season. CONCLUSION: Design thinking has yielded insights into three important dynamics that drive applicant experiences. These insights allow for a redesign of processes so that virtual interviews can be more applicant centered. This framework allows for further iterations and modifications as the needs of applicants and programs evolve over time.

Professional identity formation among undergraduate pre-medical students: a scoping review protocol.

BACKGROUND: Professional Identity formation is the process by which learners internalize a profession's values, behaviors, and perceptions. With respect to physicians, this occurs at multiple levels of medical education, including the undergraduate, graduate, and continuing medical education stages. Professional identity formation likely starts even earlier, during the undergraduate pre-medical years but, to date, no known scoping or systematic review has been conducted on this topic. The objective of this scoping review is to systematically map the literature on professional identity formation among undergraduate pre-medical students. METHODS: This review protocol has been designed following the Arksey and O'Malley framework. We will search MEDLINE, CINAHL, Embase, and Scopus, as well as relevant grey literature, conference proceedings, and citations of selected articles. Inclusion criteria are articles (1) written in the English language, (2) involving undergraduate pre-medical students in the USA and Canada, and (3) containing original data about professional identity formation. Two independent reviewers will evaluate the titles, abstracts, and full articles for eligibility. A third reviewer will help resolve any disputes. Once the full text of articles are obtained, data will be abstracted using a standardized form. A narrative summary of findings will then be conducted, as well as a consultation exercise with university pre-medical students, pre-med advisors, and first-year medical students. DISCUSSION: By conducting this scoping review, we expect to gain a better understanding of how the experiences of undergraduate pre-medical students impact their professional identity formation. These findings will help to identify gaps in the literature, to better characterize professional identity formation in the specific context of the undergraduate pre-medical track, and to outline potential approaches to facilitate professional identity formation among undergraduate pre-medical students. SYSTEMATIC REVIEW REGISTRATION: The protocol is registered with the Open Science Framework ( htps://osf.io/nfzxc ).

Pivoting during a Pandemic: Reimagining Intern Orientation EPA-Based OSCE.

OBJECTIVES: Objective structured clinical examinations (OSCEs) provide reliable and standardized means for assessing the performance of specific clinical skills. Our previous experience with entrustable professional activity-based multidisciplinary OSCEs suggests that this exercise offers just-in-time baseline information regarding critical intern skills. The coronavirus disease 2019 pandemic forced medical education programs to reimagine such educational experiences. For the safety of all of the participants, the Internal Medicine and Family Medicine residency programs pivoted from an in-person OSCE to a hybrid model (combination of in-person and virtual encounters) while maintaining the goals of the OSCEs administered in previous years. Here, we describe an innovative hybrid approach to redesigning and implementing the existing OSCE model while maximizing risk mitigation. METHODS: In total, 41 interns from Internal Medicine and Family Medicine participated in the 2020 hybrid OSCE. Five stations allowed for clinical skills assessment. Faculty completed skills checklists with global assessments and simulated patients completed communication checklists with global assessments. Interns, faculty, and simulated patients completed a post-OSCE survey. RESULTS: Informed consent, handoffs, and oral presentation were the lowest performing stations (29.2%, 53.6%, 53.6%, respectively) as assessed by faculty skill checklists. One hundred percent of interns (41/41) indicated that immediate faculty feedback was the most valuable part of the exercise, and all of the participating faculty believed that the format was efficient, allowing ample time to provide feedback and complete checklists. Eighty-nine percent of simulated patients cited they would be willing to participate again if the same assessment were to be held during the pandemic. The limitations of the study included the lack of demonstration of physical examination maneuvers by interns. CONCLUSIONS: A hybrid OSCE to assess interns' baseline skills during orientation using Zoom technology could be delivered successfully and safely during the pandemic without compromising the program's goals or satisfaction.

Empathy levels among health professional students at a large midwestern public university - a cross-sectional study.

BACKGROUND: Empathic care is considered extremely important by patients and providers alike but there is still an ample need for assessing empathy among healthcare students and professionals and identifying appropriate educational interventions to improve it. This study aims to assess empathy levels and associated factors among students at different healthcare colleges at the University of Iowa. METHODS: An online survey was delivered to healthcare students, including nursing, pharmacy, dental, and medical colleges (IRB ID #202,003,636). The cross-sectional survey included background questions, probing questions, college-specific questions, and the Jefferson Scale of Empathy-Health Professionals Student version (JSPE-HPS). To examine bivariate associations, Kruskal Wallis and Wilcoxon rank sum tests were used. A linear model with no transformation was used in the multivariable analysis. RESULTS: Three hundred students responded to the survey. Overall JSPE-HPS score was 116 (± 11.7), consistent with other healthcare professional samples. There was no significant difference in JSPE-HPS score among the different colleges (P = 0.532). CONCLUSION: Controlling for other variables in the linear model, healthcare students' view of their faculty's empathy toward patients and students' self-reported empathy levels were significantly associated with students' JSPE-HPS scores.

Longitudinal clinical reasoning theme embedded across four years of a medical school curriculum.

OBJECTIVES: The acquisition of clinical reasoning (CR) skills is essential for future healthcare providers as they advance through their education. There is growing consensus that CR skills should be longitudinally integrated into undergraduate curriculum for acquisition/application of these skills. However, only a minority of schools reported having CR focused teaching sessions, citing a lack of curricular time and faculty expertise as the largest barriers. We describe the design and implementation of this theme and report the effects of its early introduction in Phase One as measured by the Diagnostic Thinking Inventory (DTI). METHODS: The Carver College of Medicine developed and implemented a longitudinal four-year clinical reasoning theme (CRT) with a special emphasis on introducing concepts in the preclinical years (Phase One). Educational strategies used to implement the theme relied on following principles: 1) new skills are best acquired in context of application; 2) contextual learning stimulates transfer of knowledge; and 3) knowledge of pathophysiology is necessary but alone is not sufficient to develop CR skills. A patient-centered CR schema served as the framework for developing the theme. Specific focus areas, pedagogies and assessment strategies were established for each of the three phases. RESULTS: The cohort with CRT demonstrated a significant increase in total DTI score after theme implementation compared to the cohort without. CONCLUSIONS: A formal 4-year longitudinal CR theme is feasible, allowing for integration of pathophysiology, social determinants of health, and clinical skills. Early introduction of CR concepts as assessed by DTI showed improvement in student reasoning skills post-intervention.

The Doctors in the House: Examining Physician Legislators in the US Congress from 2011 to 2020.

OBJECTIVES: Legislation dictating federal healthcare policy is drafted largely by members of the US Senate and House of Representatives. As such, their personal and professional backgrounds play important roles in setting the national healthcare agenda. We examine the professional and legislative records of the 28 federal physician legislators with voting privileges between 2011 and 2020. METHODS: Two researchers compiled the names of every federal legislator in both the US Senate and the US House of Representatives who served at any time between 2011 and 2020. The researchers used publicly available records to abstract information regarding their professional and legislative records. Data were then analyzed using descriptive statistics. RESULTS: The majority of the 28 federal physician legislators are Doctor of Medicine graduates (96%), Republican (86%), represent southern states (71%), were in private practice before serving as legislators (78.5%), and have not previously held elected positions as legislators (57%). Approximately 15% of the bills that they sponsor are related to health policy. Obstetrics/Gynecology, Surgery, and Family Medicine are the most common specialties. On average, it takes 25 years from medical school graduation to election to their federal legislative position. Approximately half represent states, or districts within states, in which they attended medical school or completed residency. CONCLUSIONS: To engage in meaningful healthcare policy advocacy, professional organizations must support and encourage leadership training for physicians, increase the geographic and professional diversity of physician legislators, prioritize the election of physicians from both political parties, and inculcate deep and lasting professional relationships to physicians in Congress.

Diagnostic delays in infectious diseases.

OBJECTIVES: Diagnostic delays are a major source of morbidity and mortality. Despite the adverse outcomes associated with diagnostic delays, few studies have examined the incidence and factors that influence diagnostic delays for different infectious diseases. The objective of this study was to understand the relative frequency of diagnostic delays for six infectious diseases commonly seen by infectious diseases (ID) consultants and to examine contributing factors for these delays. METHODS: A 25-item survey to examine diagnostic delays in six infectious diseases was sent to all infectious diseases physicians in the Emerging Infections Network (EIN) who provide care to adult patients. Diseases included (1) tuberculosis, (2) non-tuberculous mycobacterial infections, (3) syphilis, (4) epidural abscess, (5) infective endocarditis, and (6) endemic fungal infections (e.g., histoplasmosis, blastomycosis). RESULTS: A total of 533 of 1,323 (40%) EIN members responded to the survey. Respondents perceived the diagnosis not being considered initially and the appropriate test not being ordered as the two most important contributors to diagnostic delays. Unusual clinical presentations and not consulting ID physicians early enough were also reported as a contributing factor to delays. Responses recorded in open-text fields also indicated errors related to testing as a likely cause of delays; specifically, test-related errors included ordering the wrong laboratory test, laboratory delays (specialized labs not available at the facility), and lab processing delays. CONCLUSIONS: Diagnostic delays commonly occur for the infectious diseases we considered. The contributing factors we identified are potential targets for future interventions to decrease diagnostic delays.

Reducing Delays in Diagnosing Primary Immunodeficiency Through the Development and Implementation of a Clinical Decision Support Tool: Protocol for a Quality Improvement Project.

BACKGROUND: Primary immunodeficiencies (PIs) are a set of heterogeneous chronic disorders characterized by immune dysfunction. They are diagnostically challenging because of their clinical heterogeneity, knowledge gaps among primary care physicians, and continuing shortages of clinically trained immunologists. As a result, patients with undiagnosed PIs are at increased risk for recurrent infections, cancers, and autoimmune diseases. OBJECTIVE: The aim of this research is to develop and implement a clinical decision support (CDS) tool for the identification of underlying PIs. METHODS: We will develop and implement a CDS tool for the identification of underlying PIs among patients who receive primary care through a health care provider at the University of Iowa Hospitals and Clinics. The CDS tool will function through an algorithm that is based on the Immune Deficiency Foundation's 10 Warning Signs for Primary Immunodeficiency. Over the course of a year, we will use Lean Six Sigma principles and the Define, Measure, Analyze, Improve, and Control (DMAIC) framework to guide the project. The primary measure is the number of newly diagnosed PI patients per month. Secondary measures include the following: (1) the number of new patients identified by the CDS as being at high risk for PI, (2) the number of new PI cases in which immunoglobulin replacement or rotating antibiotics are started, (3) the cost of evaluation of each patient identified by the CDS tool as being at high risk for PIs, (4) the number of new consults not diagnosed with a PI, and (5) patient satisfaction with the process of referral to the Immunology Clinic. RESULTS: This study was determined to not be Human Subjects Research by the Institutional Review Board at the University of Iowa. Data collection will begin in August 2021. CONCLUSIONS: The development and implementation of a CDS tool is a promising approach to identifying patients with underlying PI. This protocol assesses whether such an approach will be able to achieve its objective of reducing diagnostic delays. The disciplined approach, using Lean Six Sigma and the DMAIC framework, will guide implementation to maximize opportunities for a successful intervention that meets the study's goals and objectives as well as to allow for replication and adaptation of these methods at other sites. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): PRR1-10.2196/32635.

Genomic and Epigenomic Evaluation of Electrically Induced Exercise in People With Spinal Cord Injury: Application to Precision Rehabilitation.

OBJECTIVE: Physical therapists develop patient-centered exercise prescriptions to help overcome the physical, emotional, psychosocial, and environmental stressors that undermine a person's health. Optimally prescribing muscle activity for people with disability, such as a spinal cord injury, is challenging because of their loss of volitional movement control and the deterioration of their underlying skeletal systems. This report summarizes spinal cord injury-specific factors that should be considered in patient-centered, precision prescription of muscle activity for people with spinal cord injury. This report also presents a muscle genomic and epigenomic analysis to examine the regulation of the proliferator-activated receptor γ coactivator 1α (PGC-1α) (oxidative) and myostatin (hypertrophy) signaling pathways in skeletal muscle during low-frequency (lower-force) electrically induced exercise versus higher-frequency (higher-force) electrically induced exercise under constant muscle recruitment (intensity). METHODS: Seventeen people with spinal cord injury participated in 1 or more unilateral electrically induced exercise sessions using a lower-force (1-, 3-, or 5-Hz) or higher-force (20-Hz) protocol. Three hours after the exercise session, percutaneous muscle biopsies were performed on exercised and nonexercised muscles for genomic and epigenomic analysis. RESULTS: We found that low-frequency (low-force) electrically induced exercise significantly increased the expression of PGC-1α and decreased the expression of myostatin, consistent with the expression changes observed with high-frequency (higher-force) electrically induced exercise. Further, we found that low-frequency (lower-force) electrically induced exercise significantly demethylated, or epigenetically promoted, the PGC-1α signaling pathway. A global epigenetic analysis showed that >70 pathways were regulated with low-frequency (lower-force) electrically induced exercise. CONCLUSION: These novel results support the notion that low-frequency (low-force) electrically induced exercise may offer a more precise rehabilitation strategy for people with chronic paralysis and severe osteoporosis. Future clinical trials are warranted to explore whether low-frequency (lower-force) electrically induced exercise training affects the overall health of people with chronic spinal cord injury.

Diagnostic Reasoning by Expert Clinicians: What Distinguishes Them From Their Peers?

Objectives Expert clinicians (ECs) are defined in large part as a group of physicians recognized by their peers for their diagnostic reasoning abilities. However, their reasoning skills have not been quantitatively compared to other clinicians using a validated instrument. Methods We surveyed Internal Medicine physicians at the University of Iowa to identify ECs. These clinicians were administered the Diagnostic Thinking Inventory, along with an equivalent number of their peers in the general population of internists. Scores were tabulated for structure and thinking, as well as four previously identified elements of diagnostic reasoning (data acquisition, problem representation, hypothesis generation, and illness script search and selection). We compared scores between the two groups using the two-sample t-test. Results Seventeen ECs completed the inventory (100%). Out of 25 randomly-selected non-EC internists (IM), 19 completed the inventory (76%). Mean total scores were 187.2 and 175.8 for the EC and the IM groups respectively. Thinking and structure subscores were 91.5 and 95.71 for ECs, compared to 85.5 and 90.3 for IMs (p-values: 0.0783 and 0.1199, respectively). The mean data acquisition, problem representation, hypothesis generation, and illness script selection subscores for ECs were 4.46, 4.57, 4.71, and 4.46, compared to 4.13, 4.38, 4.45, and 4.13 in the IM group (p-values: 0.2077, 0.4528, 0.095, and 0.029, respectively). Conclusions ECs have greater proficiency in searching for and selecting illness scripts compared to their peers. There were no statistically significant differences between the other scores and subscores. These results will help to inform continuing medical education efforts to improve diagnostic reasoning.

Development and Test-Item Analysis of a Freely Available 1900-Item Question Bank for Rheumatology Trainees.

Background Tests composed of multiple-choice questions are an established tool to help evaluate knowledge of medical content. Within the field of rheumatology, there is an absence of free and easily-accessible sets of multiple-choice questions that have been rigorously evaluated and analyzed. Objective To develop a question bank composed of multiple-choice questions that evaluate trainee knowledge of rheumatology, as well as to investigate the psychometric properties (reliability, discrimination indices, difficulty indices) of items within the question bank. Methods Multiple-choice questions were drafted according to a strict methodology devised by the investigators. Between January and December 2020, questions were administered in sets of 20-25 questions to test-takers who were either current trainees or had recently graduated from training programs. Performance was evaluated through descriptive statistics (mean, median, range, standard deviation) and test-item statistics (difficulty index, discrimination index, reliability). Results Investigators drafted 1900 multiple choice questions within 45 sections each composed of 20 to 25 questions each. These questions were administered to 32 participants. The mean discrimination index was 0.57 (standard deviation: 0.22) and mean difficulty index was 0.38 (standard deviation: 0.23). Reliability indices for the 45 sections ranged from 0.45 to 0.85 (mean: 0.613, standard deviation: 0.09). The overall reliability index for the entire item bank was greater than 0.95. Conclusion The investigators developed a 1900-item question bank composed of items that have sufficient difficulty and discrimination indices to be used for low- and moderate-stakes settings. A rigorous methodology was employed to create the first freely-accessible reliable tool for the assessment of rheumatology knowledge. This tool can be purposed for both summative and formative evaluation in multiple settings and platforms.

Successful implementation of a longitudinal skill-based teaching curriculum for residents.

BACKGROUND: Despite significant teaching responsibilities and national accreditation standards, most residents do not receive adequate instruction in teaching methods. Published reports of residents-as-teachers programs vary from brief one-time exposures to curricula delivered over several months. A majority of interventions described are one or two-day workshops with no clear follow-up or reinforcement of skills. A three-year longitudinal teaching skills curriculum was implemented with these goals: 1) deliver an experiential skill-based teaching curriculum allowing all residents to acquire, practice and implement specific skills; 2) provide spaced skills instruction promoting deliberate practice/reflection; and 3) help residents gain confidence in their teaching skills. METHODS: One hundred percent of internal medicine residents (82/82) participated in the curriculum. Every 10 weeks residents attended a topic-specific experiential skills-based workshop. Each workshop followed the same pedagogy starting with debriefing/reflection on residents' deliberate practice of the previously taught skill and introduction of a new skill followed by skill practice with feedback. Every year, participants completed: 1) assessment of overall confidence in each skill and 2) retrospective pre-post self-assessment. A post-curriculum survey was completed at the end of 3 years. RESULTS: Residents reported improved confidence and self-assessed competence in their teaching skills after the first year of the curriculum which was sustained through the three-year curriculum. The curriculum was well received and valued by residents. CONCLUSIONS: A formal longitudinal, experiential skills-based teaching skills curriculum is feasible and can be delivered to all residents. For meaningful skill acquisition to occur, recurrent continuous skill-based practice with feedback and reflection is important.