Impact of interactive multi-media learning for physicians in musculoskeletal education - a pilot study.

BACKGROUND: The aim of this educational study was to investigate the use of interactive case-based modules relating to the screening and identification of early-stage inflammatory arthritis in both online technology (OLT) and paper (PF) formats with identical content. METHODS: Forty learners from family medicine or rheumatology residency programs were recruited. Content pertaining to a "Sore Hands, Sore Feet" (SHSF) and Gait Arms Legs Spine (GALS) screening tool modules were selected, reviewed and developed based on a validated curriculum from the World Health Organization and Canadian Curriculum for MSK conditions. Both the SHSF module and GALS screening tool were assessed via a randomized control trial. Assessments were completed during an orientation with all learners; then prior to the intervention (T1); at the end of the module (T2) and 3 months following the modules (T3) to assess retention. Focus groups were conducted to determine learners' satisfaction with the different learning formats. Baseline data was collated, and analysis performed after randomization into the PF (control) and OLT (experimental) groups. Repeated measures ANOVA was used for statistical analyses. RESULTS: Forty participants were recruited and randomized into the PF or OLT group (n = 20 each). At 3 months, there were n = 31 participants for SHSF (PF n = 19, OLT n = 12) and n = 32 for GALS (PF n = 19, OLT n = 13). There was no significant difference between the OLT and PF groups in both analyses. A significant increase in scores from Pre- to Post-Module in SHSF (F (1, 18) = 24.62. p < .0001) and GALS (F (1, 30) = 40.08, p < .0001) were identified to suggest learning occurred with both formats. The repeated measures ANOVA to assess retention revealed a significant decrease in scores from Post-Module to Follow-up for both learning format groups for SHSF (F (1, 29) = 4.68. p = .039), and GALS (F (1, 30) = 18.27. p < .0001) suggesting 3 months may be too long to retain this educational information. CONCLUSIONS: Both formats led to residents' ability to screen, identify and initially manage inflammatory arthritis. The hypothesis is rejected because both OLT and PF groups demonstrated significant learning during the process regardless of format. It is important to emphasize that from T1 (pre-module) to T2 (post-module), the residents demonstrated learning regardless of group to which they were assigned. However, learning retention declined from T2 (post-module) to T3 (three-month follow-up). Regular review of knowledge may be required earlier than 3 months to retain information learned. This study may impact educational strategies in MSK health. TRIAL REGISTRATION: This study did not involve "patients" rather learners and as such it was not registered.

FM-MAP: A Novel In-Training Examination Predicts Success on Family Medicine Certification Examination.

BACKGROUND AND OBJECTIVES: The objective of our study was to assess the correlation between a locally developed In-Training Examination (ITE) and the certification examination in family medicine in Canada. METHODS: The ITE was taken twice yearly, which corresponded for most residents to the fifth, ninth, 17th, and 21st month of training. The results for the ITE were correlated to the CFPC certification examination taken in the 23rd month of residency. RESULTS: The scores on each of the four iterations of the ITE correlated moderately well with performance relating to problem solving skills and knowledge on the certification examination. The ITE showed a trend to an increased correlation with duration in the training program with a Spearman correlation coefficient increasing from 0.45 on the first test to 0.54 on the fourth test. The correlation of the ITE with performance on the component assessing the doctor- patient relationship on the certification examination was poor (r=0.26 on the last test). CONCLUSION: Our in-training examination is a useful predictor of performance in problem solving and knowledge domains of the family medicine expert role on the certification examination.

Evidence of Reliability, Validity, and Practicality for the Canadian Physiotherapy Assessment of Clinical Performance.

Purpose: To investigate the internal consistency, construct validity, and practicality of the Canadian Physiotherapy Assessment of Clinical Performance (ACP), a descriptive measure used by physiotherapy students and their clinical instructors (CIs) at the mid- and endpoints of an internship to describe the students' behaviours as observed in the clinical education setting relative to what might be expected of an entry-level physiotherapist. Methods: This multi-centre study piloted the ACP in 10 university physiotherapy (PT) programmes. Both CIs and students undertaking clinical internships completed the ACP and the current tool, the Physical Therapist Clinical Performance Instrument (PT-CPI; Version 1997). Results: CIs assessing PT students' performance during internships representing a variety of areas of practice completed the ACP at the midpoint (n=132) and the endpoint (n=126) of the internship. The end-of-internship sample consisted of 55 junior, 30 intermediate, and 41 senior students. The ACP demonstrated strong internal consistency: Alpha coefficients for each role ranged from 0.94 to 0.99. Aligned items on the ACP and PT-CPI were significantly correlated (r=0.51-0.84). Senior PT students performed significantly better than intermediate students, who, in turn, performed better than junior students (p<0.0001). Effect sizes for midpoint to final scores on the ACP ranged from medium to large (0.40-0.74). Participants were satisfied with the online education module that provided instruction on how to use and interpret the ACP, as indicated by satisfaction scores and qualitative comments. Conclusions: The ACP is a reliable, valid, and practical measure to assess and describe the PT students' behaviours as observed during clinical education relative to what is expected of an entry-level physiotherapist.

Objectif : Examiner la cohérence interne, la validité de construit et l'utilité pratique de l'évaluation de la performance clinique (EPC) en physiothérapie au Canada, une mesure employée par les étudiants en physiothérapie et leurs instructeurs cliniques en milieu et en fin de stage clinique pour décrire les comportements des étudiants par rapport à ce qu'il y a lieu d'être attendu d'un physiothérapeute débutant. Méthodes : Cette étude multicentrique évaluait la mise à l'essai de l'EPC dans 10 programmes universitaires. Les instructeurs cliniques et les étudiants en stage clinique ont réalisé leurs évaluations à l'aide de l'EPC et de l'outil actuel, le PT-CPI (version 1997). Résultats : Des instructeurs cliniques représentant différents champs de pratique ont complété l'ECP en milieu (n=132) et en fin (n=126) de stage. L'échantillon de fin de stage était composé de 55 étudiants débutants, 30 étudiants intermédiaires et 41 étudiants avancés. L'ECP a démontré un haut degré de cohérence interne, soit des coefficients alpha de 0,94 à 0,99 pour chaque rôle. Une corrélation significative a été observée entre les éléments correspondants de l'ECP et du PT-CPI (r=0.51­0.84). Les étudiants avancés ont obtenu des résultats significativement meilleurs que les étudiants intermédiaires qui ont à leur tour obtenu des résultats significativement meilleurs que les étudiants débutants (p<0.0001). Les différences constatées entre les notes de l'ECP en milieu et en fin de stage révèlent une taille d'effet moyenne à forte (0,40 à 0,74). Les notes de satisfaction et les commentaires qualitatifs indiquent que les participants étaient satisfaits du module de formation en ligne qui expliquait comment utiliser et interpréter l'ACP. Conclusions : L'ECP est une mesure fiable, valide et pratique pour évaluer et décrire les comportements des étudiants observés en formation clinique par rapport à ce qu'il y a lieu d'attendre d'un physiothérapeute débutant.

Canadian Physiotherapy Assessment of Clinical Performance: Face and Content Validity.

PURPOSE: To investigate face and content validity of a draft measure to be used across Canada to assess physiotherapy students' performance in clinical education, through broad consultation with physiotherapy clinical instructors (CIs) across Canada. METHODS: An online survey was used to collect input on the draft measure. In addition to demographics, the questionnaire included questions on the preferred rating scale, the items within the measure that should have their own rating scale, and general impressions. RESULTS: Among the 259 CIs who completed the survey, a discrete rating scale with six anchors and 10 boxes or a continuous-line rating scale with six anchors was preferred. Respondents favoured using one rating scale for each key competency in the Expert role but considered a single rating scale sufficient for assessing the Scholarly Practitioner role. CIs agreed that the proposed measure would allow them to assess a student who was performing poorly or very well. The name Canadian Physiotherapy Assessment of Clinical Performance (ACP) received the most votes in the questionnaire. CONCLUSIONS: CIs' collective preferences on the design, organization, and naming of the measure they will use in evaluating students are reflected in the second draft of the ACP.

Objectif : Examiner l'apparence et la validité du contenu d'un projet de mesure qui servira dans tout le Canada pour évaluer le rendement en formation clinique des étudiants en physiothérapie, grâce à une vaste consultation auprès des enseignants cliniques à l'échelle du Canada. Méthodes : Un sondage en ligne a permis de recueillir des commentaires sur le projet de mesure. Outre les données démographiques, le questionnaire comprenait des questions sur l'échelle de cotation préférée, les éléments de la mesure qui devraient avoir leur propre échelle de cotation et les impressions générales. Résultats : Les 259 enseignants cliniques qui ont répondu au sondage préféraient généralement une échelle de cotation discrète comportant 6 ancres et 10 cases ou une échelle de cotation doublée de 6 ancres. Les répondants préféraient utiliser une échelle de cotation pour chaque compétence clé du rôle d'expert, mais considéraient qu'une seule échelle de cotation était suffisante pour évaluer le rôle d'érudit. Les enseignants cliniques étaient d'accord avec le fait que la mesure proposée leur permettrait d'évaluer les étudiants dont le rendement était mauvais ou très bon. L'appellation « évaluation du rendement clinique (ERC) de la physiothérapie au Canada ¼ a obtenu le plus de votes dans le questionnaire. Conclusions : Les préférences collectives des enseignants cliniques en matière de conception, d'organisation, et d'appellation de la mesure qu'ils utiliseront pour évaluer les étudiants sont prises en compte dans la deuxième version de l'ERC.

Use of Simulation Learning Experiences in Physical Therapy Entry-to-Practice Curricula: A Systematic Review.

PURPOSE: To review the literature on simulation-based learning experiences and to examine their potential to have a positive impact on physiotherapy (PT) learners' knowledge, skills, and attitudes in entry-to-practice curricula. METHOD: A systematic literature search was conducted in the MEDLINE, CINAHL, Embase Classic+Embase, Scopus, and Web of Science databases, using keywords such as physical therapy, simulation, education, and students. RESULTS: A total of 820 abstracts were screened, and 23 articles were included in the systematic review. While there were few randomized controlled trials with validated outcome measures, some discoveries about simulation can positively affect the design of the PT entry-to-practice curricula. Using simulators to provide specific output feedback can help students learn specific skills. Computer simulations can also augment students' learning experience. Human simulation experiences in managing the acute patient in the ICU are well received by students, positively influence their confidence, and decrease their anxiety. There is evidence that simulated learning environments can replace a portion of a full-time 4-week clinical rotation without impairing learning. CONCLUSIONS: Simulation-based learning activities are being effectively incorporated into PT curricula. More rigorously designed experimental studies that include a cost-benefit analysis are necessary to help curriculum developers make informed choices in curriculum design.

Objectif : Examiner les publications sur les expériences d'apprentissage par la simulation afin de déterminer si ces expériences peuvent avoir un effet positif sur les connaissances, l'attitude et les compétences des étudiants en physiothérapie dans un programme de formation au niveau débutant. Méthode : Une recherche systématique a été effectuée dans les publications des bases de données MEDLINE, CINAHL, Embase Classic+Embase, Scopus et Web of Science, à l'aide des mots-clés tels physiothérapie, simulation, éducation et étudiants. Résultats : Un total de 820 résumés ont été examinés et 23 articles ont été inclus dans l'examen systématique. Bien qu'il y ait eu peu d'essais contrôlés randomisés avec des mesures de résultats validées, certaines découvertes au sujet de la simulation peuvent avoir un effet positif sur la conception des programmes de formation au niveau débutant en physiothérapie. L'utilisation de simulateurs pour fournir des données de rétroaction précises peut aider les étudiants à acquérir des compétences spécifiques. Les logiciels de simulation peuvent également optimiser l'expérience d'apprentissage des étudiants. Les expériences de simulation humaine pour la gestion des patients en soins de courte durée dans une unité de soins intensifs sont reçues favorablement par les étudiants, leur donnent plus d'assurance et diminuent leur anxiété. Des preuves portent à conclure que les environnements d'apprentissage simulés peuvent remplacer une partie d'une rotation clinique de 4 semaines à temps plein sans nuire à l'apprentissage. Conclusions : Les activités d'apprentissage par la simulation sont incorporées efficacement aux programmes de physiothérapie. Des études expérimentales plus rigoureuses comprenant une analyse coûts-avantages devront être effectuées pour aider les concepteurs à prendre des décisions éclairées en matière d'élaboration de programmes d'études.

Development of the Canadian Physiotherapy Assessment of Clinical Performance: A New Tool to Assess Physiotherapy Students' Performance in Clinical Education.

PURPOSE: To develop the first draft of a Canadian tool to assess physiotherapy (PT) students' performance in clinical education (CE). Phase 1: to gain consensus on the items within the new tool, the number and placement of the comment boxes, and the rating scale; Phase 2: to explore the face and content validity of the draft tool. METHODS: Phase 1 used the Delphi method; Phase 2 used cognitive interviewing methods with recent graduates and clinical instructors (CIs) and detailed interviews with clinical education and measurement experts. RESULTS: Consensus was reached on the first draft of the new tool by round 3 of the Delphi process, which was completed by 21 participants. Interviews were completed with 13 CIs, 6 recent graduates, and 7 experts. Recent graduates and CIs were able to interpret the tool accurately, felt they could apply it to a recent CE experience, and provided suggestions to improve the draft. Experts provided salient advice. CONCLUSIONS: The first draft of a new tool to assess PT students in CE, the Canadian Physiotherapy Assessment of Clinical Performance (ACP), was developed and will undergo further development and testing, including national consultation with stakeholders. Data from Phase 2 will contribute to developing an online education module for CIs and students.

Objectif : Créer le premier projet d'outil canadien d'évaluation de la formation clinique des étudiants en physiothérapie. Phase 1: dégager un consensus sur les éléments constituants du nouvel outil, le nombre et l'emplacement des encadrés servant aux commentaires et l'échelle d'évaluation; Phase 2: explorer l'apparence et la validité du contenu du projet d'outil. Méthodes : On a utilisé, au cours de la Phase 1, la méthode Delphi et, au cours de la Phase 2, des entrevues cognitives menées auprès de nouveaux diplômés et de formateurs cliniques, ainsi que des entrevues détaillées menées auprès de spécialistes de l'éducation clinique et de la mesure. Résultats : On a dégagé un consensus sur la première version du nouvel outil au cours du cycle 3 du processus Delphi, que 21 participants ont terminé. On a réalisé des entrevues avec 13 formateurs cliniques, 6 nouveaux diplômés et 7 experts. Les nouveaux diplômés et les formateurs cliniques ont pu interpréter l'outil avec précision, qu'ils croyaient pouvoir appliquer à une expérience d'éducation clinique récente, et ont suggéré des améliorations. Les spécialistes ont fourni des conseils marquants. Conclusions : La première version d'un nouvel outil d'évaluation de la formation clinique des étudiants en physiothérapie, l'évaluation du rendement clinique (ERC) en physiothérapie canadienne, a été mis au point et fera l'objet d'autres améliorations et essais, y compris d'une consultation nationale des intervenants. Les données tirées de la Phase 2 contribueront à l'élaboration du module d'éducation en ligne.

How to assess communication, professionalism, collaboration and the other intrinsic CanMEDS roles in orthopedic residents: use of an objective structured clinical examination (OSCE).

BACKGROUND: Assessing residents' understanding and application of the 6 intrinsic CanMEDS roles (communicator, professional, manager, collaborator, health advocate, scholar) is challenging for postgraduate medical educators. We hypothesized that an objective structured clinical examination (OSCE) designed to assess multiple intrinsic CanMEDS roles would be sufficiently reliable and valid. METHODS: The OSCE comprised 6 10-minute stations, each testing 2 intrinsic roles using case-based scenarios (with or without the use of standardized patients). Residents were evaluated using 5-point scales and an overall performance rating at each station. Concurrent validity was sought by correlation with in-training evaluation reports (ITERs) from the last 12 months and an ordinal ranking created by program directors (PDs). RESULTS: Twenty-five residents from postgraduate years (PGY) 0, 3 and 5 participated. The interstation reliability for total test scores (percent) was 0.87, while reliability for each of the communicator, collaborator, manager and professional roles was greater than 0.8. Total test scores, individual station scores and individual CanMEDS role scores all showed a significant effect by PGY level. Analysis of the PD rankings of intrinsic roles demonstrated a high correlation with the OSCE role scores. A correlation was seen between ITER and OSCE for the communicator role, while the ITER medical expert and total scores highly correlated with the communicator, manager and professional OSCE scores. CONCLUSION: An OSCE designed to assess the intrinsic CanMEDS roles was sufficiently valid and reliable for regular use in an orthopedic residency program.

CONTEXTE: Évaluer la compréhension et l'application des 6 rôles intrinsèques CanMEDS (communicateur, professionnel, gestionnaire, collaborateur, promoteur de la santé, érudit) chez les résidents pose un défi pour les responsables de la formation médicale postdoctorale. Nous avons émis l'hypothèse selon laquelle un examen clinique objectif structuré (ECOS) conçu pour évaluer plusieurs rôles CanMEDS intrinsèques serait suffisamment fiable et valide. MÉTHODES: L'ECOS comportait 6 stations de 10 minutes, permettant chacune d'évaluer 2 rôles intrinsèques à l'aide de scénarios basés sur des cas (avec ou sans recours à des patients standardisés). Les résidents ont été notés au moyen d'échelles en 5 points et d'une évaluation globale de leur rendement à chacune des stations. La validité convergente a été vérifiée par corrélation avec les rapports d'évaluation en cours de formation (RÉF) des 12 mois précédents et un classement chiffré créé par les directeurs du programme (DP). RÉSULTATS: Vingt-cinq résidents des années 0, 3 et 5 y ont participé. La fiabilité interstation pour les scores totaux aux tests (en pourcentage) a été de 0,87, tandis que la fiabilité pour chacun des rôles de communicateur, collaborateur, gestionnaire et professionnel, a été supérieure à 0,8. Les scores totaux aux tests, les scores aux stations individuelles et les scores pour les rôles CanMEDS individuels ont tous fait état d'un effet significatif selon le niveau des résidents. L'analyse des classements établis par les DP quant aux rôles intrinsèques a révélé une forte corrélation avec les scores au test ECOS pour les rôles. On a observé une corrélation entre les RÉF et l'ECOS pour le rôle de communicateur, tandis que les RÉF pour le rôle d'expert médical et les scores totaux ont été en forte corrélation avec les scores de l'ECOS pour les rôles de communicateur, de gestionnaire et de professionnel. CONCLUSION: Un ECOS conçu pour évaluer les rôles CanMEDS intrinsèques s'est révélé suffisamment valide et fiable pour un usage régulier dans un programme de résidence en orthopédie.

Postgraduate anaesthesiology education: protocol for a scoping review.

INTRODUCTION: The link between education and clinical practice is vital, yet the current state of research suggests there is a substantial gap between medical education research and practice. This too is the case in the domain of anaesthesiology education research, as much of the research focuses on simulation studies, and a narrow range of research methods. The aim of this study is to comprehensively review the existing literature in postgraduate anaesthesiology education research in order to identify key research priorities. The findings from this review will be used to establish a base for developing a strategic research programme in anaesthesia education and practice. METHODS AND ANALYSIS: We will employ the scoping review methodology outlined by Arksey and O'Malley (2005) to comprehensively search the literature pertaining to postgraduate anaesthesiology education. We will search relevant electronic databases (eg, MEDLINE, EMBASE) and grey literature. After conducting calibration exercises, two authors will independently apply inclusion criteria to all titles and abstracts and perform full-text review of all eligible articles. Data to be extracted will include basic information about the study (eg, location, design) as well as detailed information regarding the context of the research and type of education examined. Our results will be used to develop a framework of themes that outline the research currently being conducted and identify gaps in research. ETHICS AND DISSEMINATION: This review is the first step in a strategic research plan in postgraduate anaesthesiology education. We plan to disseminate this research through publications, presentations and meetings with relevant stakeholders. Ethical approval was not sought for this scoping review.

The epidemiology of competence: protocol for a scoping review.

INTRODUCTION: Factors that are important to the competence of healthcare providers have important consequences for quality of healthcare. Although some previous research has discussed risks or supports to the competence of clinicians, a thorough exploration is currently lacking. The purpose of this review is to examine the literature examining risks and supports to clinical competence of healthcare practitioners and trainees engaged in field-based education. In this study, field-based education refers to teaching or training in a real-world/work-based setting. METHODS AND ANALYSIS: We will employ an established scoping review methodology. Eligible studies will include those that mention (1) a healthcare professional, (2) competence from field-based education throughout the lifespan/career and (3) a risk or support to competence. Four authors will independently apply the inclusion criteria to all studies, regardless of study design. Our data extraction will include information on study design, location and type of study and we will develop a comprehensive list of risks and supports that are discussed in the literature. ETHICS AND DISSEMINATION: Since this is a review of the literature, ethics approval is not indicated. We will disseminate the findings from this study in publications in peer-reviewed journals as well as presentations at relevant national and international conferences.

Emergency medicine procedural skills: what are residents missing?

OBJECTIVE: This study sought to establish the current state of procedural skills training in Canadian Royal College emergency medicine (EM) residencies. METHODS: A national Web-based survey was administered to residents and program directors of all 13 Canadian-accredited Royal College EM residency programs. Program directors rated the importance and experience required for competence of 45 EM procedural skills. EM residents reported their experience and comfort in performing the same procedural skills. RESULTS: Thirteen program directors and 86 residents responded to the survey (response rate of 100% and 37%, respectively). Thirty-two (70%) procedures were considered important by > 70% of program directors, including all resuscitation and lifesaving airway procedures. Four procedures deemed important by program directors, including cricothyroidotomy, pericardiocentesis, posterior nasal pack for epistaxis, and paraphimosis reduction, had never been performed by the majority of senior residents. Program director opinion was used to categorize each procedure based on performance frequency to achieve competence. Overall, procedural experience correlated positively with comfort levels as indicated by residents. CONCLUSIONS: We established an updated needs assessment of procedural skills training for Canadian Royal College EM residency programs. This included program director opinion of important procedures and the performance frequency needed to achieve competence. However, we identified several important procedures that were never performed by most senior residents despite program director opinion regarding the experience needed for competence. Further study is required to better define objective measures for resident competence in procedural skills.

Admissions file review: applying the multiple independent sampling (MIS) methodology.

PURPOSE: Although multiple independent sampling (MIS) has been adapted for admissions interviews, its application for assessing written materials in the admissions file has been limited. Currently, admissions file review at the University of Toronto medical school involves one rater per file to enable holistic assessment, which may introduce a halo effect-that is, impressions of one component influencing the evaluation of other components. The authors examined whether MIS file review, through which multiple raters evaluate specific file components independently, may reduce this effect. METHOD: The authors selected a stratified random sample of 300 applicant files from the 2010-2011 admissions cycle for rescoring by MIS. They divided each of the 300 applicant files into their four components (academic transcript, autobiographical sketch, personal statement, reference letters) and rebundled them into packages of 38 same-component items (purposely creating some overlap among packages to assess inter-rater reliability). The authors distributed each package to 1 of 36 raters; thus, each rater evaluated only one of four components across many applicants. The authors compared the inter-component reliability and factor analysis of MIS with that of holistic scoring. RESULTS: Ratings were returned for all applicants. Inter-component reliability (Cronbach alpha) was 0.69 for holistic scoring and 0.29 for MIS. Factor analysis showed all components loading heavily onto one factor in the holistic approach and onto three factors in the MIS method. CONCLUSIONS: Using MIS to assess the admissions file may reduce the halo effect and should be considered when evaluating applicants' written submissions.

Using "standardized narratives" to explore new ways to represent faculty opinions of resident performance.

PURPOSE: Most efforts to develop reliable evaluations of clinical competence have been oriented toward deconstructing the requisite competencies into separate scales. However, many are questioning the value of this approach on theoretical and empirical bases. This study uses "standardized narratives" to explore a different approach to assessing resident performance. METHOD: In 2009, based on interviews with 19 experienced clinical faculty from two institutions, 16 narrative profiles were created to represent the range of resident competence that clinical faculty might encounter during supervision. Fourteen clinicians from three institutions independently grouped the profiles into as many categories as necessary to reflect various levels of performance, described their categories, then ranked the individual profiles within each category. Then, in groups of three or four, participants negotiated a final ranking and grouping of the 16 profiles. RESULTS: Despite interesting idiosyncracies in the factors some participants identified as guiding their rankings, there was strong consistency across the 14 clinicians regarding the rankings (single-rater intraclass correlation [ICC] = 0.86) and groupings (single-rater ICC = 0.81) of the profiles. Similarly, across institutions, the four groups were highly consistent in their final negotiated rankings (single-group ICC = 0.91) and groupings (single-group ICC = 0.87) of the profiles. CONCLUSIONS: Faculty showed more consistency in their decisions of what constitutes excellent, competent, and problematic performance in residents than implied by current assessment techniques that require deconstruction of resident competencies. This use of standardized narratives points to interesting opportunities for more authentically codifying faculty opinions of residents.

Who you know or what you know? Effect of examiner familiarity with residents on OSCE scores.

BACKGROUND: Despite the goal of objective structured clinical examinations (OSCEs) to be objective, examiner biases may influence scores. Examiner familiarity with candidates is a potential bias that has not been well studied. METHOD: To determine the effect of familiarity, OSCE scores for 158 internal medicine residents were analyzed by whether examiners were familiar with them, based on previous clinical encounters, and if previous impressions were positive or negative. A hierarchical multivariable analysis of variance was performed to control for resident, examiner, and level of training. RESULTS: Across 480 interactions (50 examiners, 158 residents), multivariable analysis showed that positive familiarity was associated with a significant increase in ratings (+0.37 on a 5-point scale), comparable to the difference between first- and third/fourth-year residents. CONCLUSIONS: Familiarity with candidates is a significant source of examiner bias in OSCE scores. Consideration should be paid to the influence of examiners' previous knowledge of examinees and attempts made to mitigate this bias.