The peritoneal dialysis orders objective structured clinical examination (OSCE): A formative assessment for nephrology fellows.

BACKGROUND: Peritoneal dialysis (PD) management is a fundamental nephrology skill, especially with the recent emphasis on home dialysis. We report a prospective multicentre cohort study of a formative objective structured clinical examination (OSCE) assessing competence in managing PD-associated bacterial peritonitis, using the unified model of construct validity. METHODS: The OSCE was developed by the principal investigators and reviewed by two subject matter experts. The test committee (eight nephrologists and one PD nurse) assessed test item difficulty/relevance and determined passing score. There were 22 test items (7 evidence-based/standard-of-care questions). Passing score was 16/22 (73%). No item had median relevance less than 'important', and all were easy to medium difficulty. Content validity index was 0.91. Preliminary validation (16 board-certified volunteers): mean score was 19 ± 2, with 94% (15/16) passing. Kappa = 0.85 [95% confidence interval (CI) 0.77-0.94]. Cronbach's α = 0.70. RESULTS: Eighty-seven fellows (16 programmes) were tested; 67% passed. Fellows scored significantly less than validators: 17 ± 3 versus 19 ± 2, p < 0.001 [95% CI 1.2-3.6]. Eighty-six per cent of evidence-based/standard-of-care questions were answered correctly by validators versus 54% by fellows; p < 0.001. Ninety-three per cent of fellows recognized that sufficient criteria were present to diagnose peritonitis, but only 17% correctly indicated all three. Seventy-seven per cent recognized peritonitis-associated ultrafiltration failure, but only 17% prescribed 21 days of antibiotic treatment for gram-negative peritonitis. Eighty-five per cent of fellows surveyed agreed/strongly agreed that the OSCE was useful in self-assessing proficiency. Second-year in-training examination and OSCE scores were positively correlated (Pearson's r = 0.57, p < 0.00). CONCLUSIONS: The OSCE may be used to formatively assess fellow proficiency in managing PD-associated peritonitis.

Assessing Nephrology Fellows' Skills in Communicating About Kidney Replacement Therapy and Kidney Biopsy: A Multicenter Clinical Simulation Study on Breaking Bad News.

RATIONALE & OBJECTIVE: Interpersonal communication skills and professionalism competencies are difficult to assess among nephrology trainees. We developed a formative "Breaking Bad News" simulation and implemented a study in which nephrology fellows were assessed with regard to their skills in providing counseling to simulated patients confronting the need for kidney replacement therapy (KRT) or kidney biopsy. STUDY DESIGN: Observational study of communication competency in the setting of preparing for KRT for kidney failure, for KRT for acute kidney injury (AKI), or for kidney biopsy. SETTING & PARTICIPANTS: 58 first- and second-year nephrology fellows assessed during 71 clinical evaluation sessions at 8 training programs who participated in an objective structured clinical examination of simulated patients in 2017 and 2018. PREDICTORS: Fellowship training year and clinical scenario. OUTCOME: Primary outcome was the composite score for the "overall rating" item on the Essential Elements of Communication-Global Rating Scale 2005 (EEC-GRS), as assessed by simulated patients. Secondary outcomes were the score for EEC-GRS "overall rating" item for each scenario, score < 3 for any EEC-GRS item, Mini-Clinical Examination Exercise (Mini-CEX) score < 3 on at least 1 item (as assessed by faculty), and faculty and fellow satisfaction with simulation exercise (via a survey they completed). ANALYTICAL APPROACH: Nonparametric tests of hypothesis comparing performance by fellowship year (primary goal) and scenario. RESULTS: Composite scores for EEC-GRS overall rating item were not significantly different between fellowship years (P = 0.2). Only 4 of 71 fellow evaluations had an unsatisfactory score for the EEC-GRS overall rating item on any scenario. On Mini-CEX, 17% scored < 3 on at least 1 item in the kidney failure scenario; 37% and 53% scored < 3 on at least 1 item in the AKI and kidney biopsy scenarios, respectively. In the survey, 96% of fellows and 100% of faculty reported the learning objectives were met and rated the experience good or better in 3 survey rating questions. LIMITATIONS: Relatively brief time for interactions; limited familiarity with and training of simulated patients in use of EEC-GRS. CONCLUSIONS: The fellows scored highly on the EEC-GRS regardless of their training year, suggesting interpersonal communication competency is achieved early in training. The fellows did better with the kidney failure scenario than with the AKI and kidney biopsy scenarios. Structured simulated clinical examinations may be useful to inform curricular choices and may be a valuable assessment tool for communication and professionalism.

The Acute Dialysis Orders Objective Structured Clinical Examination (OSCE): Fellow Performance on a Formative Assessment of Acute Kidney Replacement Therapy Competence.

BACKGROUND AND OBJECTIVES: Acute kidney replacement therapy (KRT) prescription is a critical nephrology skill. We administered a formative objective structured clinical examination (OSCE) to nephrology fellows to assess acute KRT medical knowledge, patient care, and systems-based practice competencies. DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS: Prospective cohort study of an educational test using the unified model of construct validity. We tested 117 fellows: 25 (four programs) in 2016 and 92 (15 programs) in 2017; 51 first-year and 66 second-year fellows. Using institutional protocols and order sets, fellows wrote orders and answered open-ended questions on a three-scenario OSCE, previously validated by board-certified, practicing clinical nephrologists. Outcomes were overall and scenario pass percentage and score; percent correctly answering predetermined, evidence-based questions; second-year score correlation with in-training examination score; and satisfaction survey. RESULTS: A total of 76% passed scenario 1 (acute continuous RRT): 92% prescribed a ≥20 ml/kg per hour effluent dose; 63% estimated clearance as effluent volume. Forty-two percent passed scenario 2 (maintenance dialysis initiation); 75% correctly prescribed 3-4 mEq/L K+ dialysate and 12% identified the two absolute, urgent indications for maintenance dialysis initiation (uremic encephalopathy and pericarditis). Six percent passed scenario 3 (acute life-threatening hyperkalemia); 20% checked for rebound hyperkalemia with two separate blood draws. Eighty-three percent correctly withheld intravenous sodium bicarbonate for acute hyperkalemia in a nonacidotic, volume-overloaded patient on maintenance dialysis, and 32% passed overall. Second-year versus first-year fellow overall score was 44.4±4 versus 42.7±5 (one-tailed P=0.02), with 39% versus 24% passing (P=0.08). Second-year in-training examination and OSCE scores were not significantly correlated (r=0.15; P=0.26). Seventy-seven percent of fellows agreed the OSCE was useful in assessing "proficiency in ordering" acute KRT. Limitations include lack of a validated criterion test, and unfamiliarity with open-ended question format. CONCLUSIONS: The OSCE can provide quantitative data for formative Accreditation Council for Graduate Medical Education competency assessments and identify opportunities for dialysis curriculum development. PODCAST: This article contains a podcast at https://www.asn-online.org/media/podcast/CJASN/2019_08_08_CJASNPodcast_19_09_.mp3.

The dialysis orders objective structured clinical examination (OSCE): a formative assessment for nephrology fellows.

BACKGROUND: Few quantitative nephrology-specific simulations assess fellow competency. We describe the development and initial validation of a formative objective structured clinical examination (OSCE) assessing fellow competence in ordering acute dialysis. METHODS: The three test scenarios were acute continuous renal replacement therapy, chronic dialysis initiation in moderate uremia and acute dialysis in end-stage renal disease-associated hyperkalemia. The test committee included five academic nephrologists and four clinically practicing nephrologists outside of academia. There were 49 test items (58 points). A passing score was 46/58 points. No item had median relevance less than 'important'. The content validity index was 0.91. Ninety-five percent of positive-point items were easy-medium difficulty. Preliminary validation was by 10 board-certified volunteers, not test committee members, a median of 3.5 years from graduation. The mean score was 49 [95% confidence interval (CI) 46-51], κ = 0.68 (95% CI 0.59-0.77), Cronbach's α = 0.84. RESULTS: We subsequently administered the test to 25 fellows. The mean score was 44 (95% CI 43-45); 36% passed the test. Fellows scored significantly less than validators (P < 0.001). Of evidence-based questions, 72% were answered correctly by validators and 54% by fellows (P = 0.018). Fellows and validators scored least well on the acute hyperkalemia question. In self-assessing proficiency, 71% of fellows surveyed agreed or strongly agreed that the OSCE was useful. CONCLUSIONS: The OSCE may be used to formatively assess fellow proficiency in three common areas of acute dialysis practice. Further validation studies are in progress.

Sharing innovations to enhance renal fellow education in dialysis.

Dialysis care is an integral part of the practice of nephrology. Despite this, education of fellows in providing dialysis often remains rudimentary, relying on a combination of didactics and learning through experience. This runs the risk of training nephrologists who can provide dialysis care without truly being experts on the subject. In this article, a collection of novel or innovative teaching methods is presented that are meant to provide training programs with additional tools with which to improve the training of their fellows in dialysis.

Integrating Quality Improvement Education into the Nephrology Curricular Milestones Framework and the Clinical Learning Environment Review.

The Accreditation Council for Graduate Medical Education requires that trainees show progressive milestone attainment in the practice-based learning and systems-based practice competencies. As part of the Clinical Learning Environment Review, sponsoring hospitals must educate trainees in health care quality improvement, provide them with specialty-specific quality data, and ensure trainee participation in quality improvement activities and committees. Subspecialty-specific quality improvement curricula in nephrology training programs have not been reported, although considerable curricular and assessment material exists for specialty residencies, including tools for assessing trainee and faculty competence. Nephrology-specific didactic material exists to assist nephrology fellows and faculty mentors in designing and implementing quality improvement projects. Nephrology is notable among internal medicine subspecialties for the emphasis placed on adherence to quality thresholds-specifically for chronic RRT shown by the Centers for Medicare and Medicaid Services Quality Incentive Program. We have developed a nephrology-specific curriculum that meets Accreditation Council for Graduate Medical Education and Clinical Learning Environment Review requirements, acknowledges regulatory quality improvement requirements, integrates with ongoing divisional quality improvement activities, and has improved clinical care and the training program. In addition to didactic training in quality improvement, we track trainee compliance with Kidney Disease Improving Global Outcomes CKD and ESRD quality indicators (emphasizing Quality Improvement Program indicators), and fellows collaborate on a yearly multidisciplinary quality improvement project. Over the past 6 years, each fellowship class has, on the basis of a successful quality improvement project, shown milestone achievement in Systems-Based Practice and Practice-Based Learning. Fellow quality improvement projects have improved nephrology clinical care within the institution and introduced new educational and assessment tools to the training program. All have been opportunities for quality improvement scholarship. The curriculum prepares fellows to apply quality improvement principals in independent clinical practice-while showing milestone advancement and divisional compliance with Clinical Learning Environment Review requirements.

Implementation of nephrology subspecialty curricular milestones.

Beginning in the 2014-2015 training year, the US Accreditation Council for Graduate Medical Education (ACGME) required that nephrology Clinical Competency Committees assess fellows' progress toward 23 subcompetency "context nonspecific" internal medicine subspecialty milestones. Fellows' advancement toward the "ready for unsupervised practice" target milestone now is tracked in each of the 6 competencies: Patient Care, Medical Knowledge, Professionalism, Interpersonal Communication Skills, Practice-Based Learning and Improvement, and Systems-Based Practice. Nephrology program directors and subspecialty societies must define nephrology-specific "curricular milestones," mapped to the nonspecific ACGME milestones. Although the ACGME goal is to produce data that can discriminate between successful and underperforming training programs, the approach is at risk to produce biased, inaccurate, and unhelpful information. We map the ACGME internal medicine subspecialty milestones to our previously published nephrology-specific milestone schema and describe entrustable professional activities and other objective assessment tools that inform milestone decisions. Mapping our schema onto the ACGME subspecialty milestone reporting form allows comparison with the ACGME subspecialty milestones and the curricular milestones developed by the American Society of Nephrology Program Directors. Clinical Competency Committees may easily adapt and directly translate milestone decisions reached using our schema onto the ACGME internal medicine subspecialty competency milestone-reporting format.