Effect of Perceived Level of Interaction on Faculty Evaluations of 3rd Year Medical Students.

INTRODUCTION: Several factors are known to affect the way clinical performance evaluations (CPEs) of medical students are completed by supervising physicians. We sought to explore the effect of faculty perceived "level of interaction" (LOI) on these evaluations. METHODS: Our third-year CPE requires evaluators to identify perceived LOI with each student as low, moderate, or high. We examined CPEs completed during the academic year 2018-2019 for differences in (1) clinical and professionalism ratings, (2) quality of narrative comments, (3) quantity of narrative comments, and (4) percentage of evaluation questions left unrated. RESULTS: A total of 3682 CPEs were included in the analysis. ANOVA revealed statistically significant differences between LOI and clinical ratings (p ≤ .001), with mean ratings from faculty with a high LOI significantly higher than from faculty with a moderate or low LOI (p ≤ .001). Chi-squared analysis demonstrated differences based on faculty LOI and whether questions were left unrated (p ≤ .001), quantity of narrative comments (p ≤ .001), and specificity of narrative comments (p ≤ .001). CONCLUSIONS: Faculty who perceive higher LOI were more likely to assign that student higher ratings, complete more of the clinical evaluation and were more likely to provide narrative feedback with more specific, higher-quality comments. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s40670-021-01307-w.

Practical Application of Assessment Principles in the Development of an Innovative Clinical Performance Evaluation in the Entrustable Professional Activity Era.

Educators have been challenged to create assessments that are competency-based and grounded in accepted standards such as the entrustable professional activities (EPAs). The clinical performance evaluation (CPE) is a commonly utilized assessment modality, which allows multiple evaluators to provide feedback on a learner's performance in the clinical workplace. In this paper, we describe the relevant principles that served as a guide as we developed a new CPE for medical students that fully incorporate EPAs. This may help ease the transition for other institutions looking to introduce a new student CPE.

Development and testing of performance measures for pharmacy services.

OBJECTIVE: To report on the status of the pilot work of PQA, a pharmacy quality alliance, to develop and test performance metrics of pharmacy services for use in quality improvement, benchmarking, and pay-for-performance benchmarks. DESIGN: Observational cohort study. SETTING: Three health plans (commercial, Medicare and Medicaid) located in the northeastern United States and one nationwide prescription drug plan. PATIENTS: Pharmacies of health plans with membership ranging from approximately 3,330 to nearly 1.7 million members. INTERVENTION: Pharmaceutical claims data for prescriptions dispensed at community pharmacies were analyzed. MAIN OUTCOME MEASURES: Not applicable. RESULTS: The four plans had pharmacy networks ranging from 653 to 53,153 pharmacies. When using a minimum sample of 30 members per measure, less than 10% of the pharmacies within the plans' networks were evaluable for all measures except the measure of high-risk drugs in the elderly. The measure for high-risk drugs in the elderly had 6,210 evaluable pharmacies in a network of 53,153. The measures for high-risk drugs in the elderly and medication adherence appear to have the greatest potential for use as performance measures in that they show room for improvement and variation among pharmacies. CONCLUSION: The ideal performance measure is relevant, scientifically sound, and feasible. Several of the measures that underwent testing possessed some, if not all, of the properties of an ideal performance measure. Strategies for aggregating data across health and drug plans may be useful for overcoming sample size challenges.