Psychometric validation of the Laval developmental benchmarks scale for family medicine.

BACKGROUND: With the implementation of competency-based education in family medicine, there is a need for summative end-of-rotation assessments that are criterion-referenced rather than normative. Laval University's family residency program therefore developed the Laval Developmental Benchmarks Scale for Family Medicine (DBS-FM), based on competency milestones. This psychometric validation study investigates its internal structure and its relation to another variable, two sources of validity evidence. METHODS: We used assessment data from a cohort of residents (n = 1432 assessments) and the Rasch Rating Scale Model to investigate its reliability, dimensionality, rating scale functioning, targeting of items to residents' competency levels, biases (differential item functioning), items hierarchy (adequacy of milestones ordering), and score responsiveness. Convergent validity was estimated by its correlation with the clinical rotation decision (pass, in difficulty/fail). RESULTS: The DBS-FM can be considered as a unidimensional scale with good reliability for non-extreme scores (.83). The correlation between expected and empirical items hierarchies was of .78, p < .0001.Year 2 residents achieved higher scores than year 1 residents. It was associated with the clinical rotation decision. CONCLUSION: Advancing its validation, this study found that the DBS-FM has a sound internal structure and demonstrates convergent validity.

CanMEDS Competencies in Family Medicine Residents: Can Criterion-Based Assessment Improve the Quality of Teacher Feedback?

This article was migrated. The article was marked as recommended. Background: The Université Laval family medicine program has developed an innovative computerized tool called the criterion-based Competency Assessment Tool (CAT), currently undergoing validity assessment. Methods: This study followed a qualitative design assessing written comments collected in the assessment reports from the cohorts before and after the implementation of the CAT (n pre = 200, n post = 200) in order to ascertain the tool's consequence validity. A deductive thematic content analysis was performed and pre- and post-implementation cohorts were compared. Findings: Overall feedback quality does not appear to have changed between cohorts. When analyzing CanMEDS roles separately, each is covered more often, but related comments appear to be less specific. The new report also seems to enable the teacher to tell more with the same number of words. Conclusions: Perhaps since the items are complete, exhaustive, and detailed enough to be self-explanatory, the tool helps the teacher to cover a wider area of competencies without the need to add many details with narrative comments. Consequence validity does not seem to have been substantially affected by changes in the family medicine resident's competency assessment, but the results do not support the contention that comment quality has improved either.

Interventions for undergraduate and postgraduate medical learners with academic difficulties: A BEME systematic review: BEME Guide No. 56.

Background: Clinical teachers often struggle to report unsatisfactory trainee performance, partly because of a lack of evidence-based remediation options. Objectives: To identify interventions for undergraduate (UG) and postgraduate (PG) medical learners experiencing academic difficulties, link them to a theory-based framework and provide literature-based recommendations around their use. Methods: This systematic review searched MEDLINE, CINAHL, EMBASE, ERIC, Education Source and PsycINFO (1990-2016) combining these concepts: medical education, professional competence/difficulty and educational support. Original research/innovation reports describing intervention(s) for UG/PG medical learners with academic difficulties were included. Data extraction employed Michie's Behavior Change Techniques (BCT) Taxonomy and program evaluation models from Stufflebeam and Kirkpatrick. Quality appraisal used the Mixed Methods Appraisal Tool (MMAT). The authors synthesized extracted evidence by adapting the GRADE approach to formulate recommendations. Results: Sixty-eight articles met the inclusion criteria, most commonly addressing knowledge (66.2%), skills (53.9%) and attitudinal problems (26.2%), or learner personal issues (41.5%). The most common BCTs were Shaping knowledge, Feedback/monitoring, and Repetition/substitution. Quality appraisal was variable (MMAT 0-100%). A thematic content analysis identified 109 interventions (UG: n = 84, PG: n = 58), providing 24 strong, 48 moderate, 26 weak and 11 very weak recommendations. Conclusion: This review provides a repertoire of literature-based interventions for teaching/learning, faculty development, and research purposes.

Value for money in self-regulated procedural simulation.

BACKGROUND: In self-regulated procedural simulation, learners practise on many simulators (e.g. paracentesis), self-regulating their choice of simulators, time and goals. Current needs assessments cannot predict the number of simulators needed to plan cost-effective self-regulated simulation. Knowing the ratios of simulators and participants would allow for better-informed purchase decisions to be made. METHODS: We designed 90-minute sessions of self-regulated procedural simulation for internal medicine residents. In Phase 1, 51 participants (8.5 per group) could use 22 simulators (US$69 925): ultrasound-guided central (n = 6) and peripheral (n = 2) venous catheterisation; thoracocentesis (n = 2); paracentesis (n = 2); lumbar puncture (n = 6); and arthrocentesis (n = 4). We calculated minimal numbers of simulators based on the time that participants used each simulator in order to design a resource-effective Phase 2, with 24 participants (with 12 per group) using 14 simulators (US$48 720) to meet their needs. RESULTS: Calculated from time of use (83 minutes in total), the optimal ratios of simulators expressed for 10 participants were 9.2: 3.7 for jugular and subclavian venous catheterisation (33 minutes); 1.5 for thoracocentesis (13 minutes), 1.0 for femoral venous catheterisation (9 minutes), 1.0 for lumbar puncture (9 minutes), 0.8 for peripheral venous catheterisation (8 minutes), 0.7 for paracentesis (6 minutes) and 0.5 for arthrocentesis (5 minutes). In Phase 2, the usage rate of simulators increased from 35.5% to 76.6%, maintaining the total time of use at 80.4 minutes. CONCLUSIONS: We present a replicable method for the cost-effective planning of self-regulated simulation by measuring the use of simulators. Expressed as ratios of simulators per participant, this information can support purchase decisions and be shared with similar programmes.

Adjusting to duty hour reforms: residents' perception of the safety climate in interdisciplinary night-float rotations.

BACKGROUND: New scheduling models were needed to adjust to residents' duty hour reforms while maintaining safe patient care. In interdisciplinary night-float rotations, four to six residents from most residency programs collaborated for after-hours cross-coverage of most adult hospitalised patients as part of a Faculty-led rotation. Residents worked sixteen 12-hour night shifts over a month. METHODS: We measured residents' perception of the patient safety climate during implementation of night-float rotations in five tertiary hospitals. We surveyed 267 residents who had completed the rotation in 2015-2016 with an online version of the Safety Attitudes Questionnaire. First year residents came from most residency programs, second- and third-year residents came from internal medicine. RESULTS: One-hundred-and-thirty residents completed the questionnaire. Scores did not differ across hospitals and residents' years of training for all six safety-related climate factors: teamwork climate, job satisfaction, perceptions of management, safety climate, working conditions, and stress recognition. CONCLUSION: Simultaneous implementation in five hospitals of a Faculty-led interdisciplinary night-float rotation for most junior residents proved to be logistically feasible and showed similar and reassuring patient safety climate scores.

CONTEXTE: De nouveaux horaires de garde en établissements hospitaliers étaient nécessaires pour s'adapter aux réformes des heures de travail des résidents tout en maintenant des soins sécuritaires pour les patients. Dans les stages cliniques de nuit interdisciplinaires, quatre à six résidents de la plupart des programmes de résidence ont collaboré pour assurer une couverture croisée, après les heures normales de travail, de la plupart des patients adultes hospitalisés. Les résidents ont travaillé seize nuits de 12 heures durant un mois. MÉTHODES: Nous avons mesuré la perception des résidents du climat de travail lié à la sécurité des patients lors de la mise en place de stages de nuit dans cinq hôpitaux universitaires. Nous avons interrogé 267 résidents ayant terminé le stage en 2015-2016 avec une version numérique du Safety Attitudes Questionnaire. Les résidents de première année provenaient de la plupart des programmes de résidence, les résidents de deuxième et troisième années provenaient du programme de médecine interne. RÉSULTATS: 130 résidents ont complété le questionnaire. Les scores ne différaient pas entre les hôpitaux et les années de formation des résidents pour les six facteurs liés à la sécurité des patients: climat de travail en équipe, satisfaction au travail, perceptions des supérieurs, climat de sécurité, conditions de travail et reconnaissance du stress. CONCLUSIONS: La mise en place simultanée, dans cinq hôpitaux, de stages cliniques de nuit réunissant des résidents juniors de la majorité des programmes de résidence fut logistiquement possible et a montré des résultats similaires et rassurants sur le climat de travail lié à la sécurité des patients.

An Integrated Career Coaching and Time-Banking System Promoting Flexibility, Wellness, and Success: A Pilot Program at Stanford University School of Medicine.

Faculty in academic medicine experience multiple demands on their time at work and home, which can become a source of stress and dissatisfaction, compromising success. A taskforce convened to diagnose the state of work-life flexibility at Stanford University School of Medicine uncovered two major sources of conflict: work-life conflict, caused by juggling demands of career and home; and work-work conflict, caused by competing priorities of the research, teaching, and clinical missions combined with service and administrative tasks. Using human-centered design research principles, the 2013-2014 Academic Biomedical Career Customization (ABCC) pilot program incorporated two elements to mitigate work-life and work-work conflict: integrated career-life planning, coaching to create a customized plan to meet both career and life goals; and a time-banking system, recognizing behaviors that promote team success with benefits that mitigate work-life and work-work conflicts. A matched-sample pre-post evaluation survey found the two-part program increased perceptions of a culture of flexibility (P = .020), wellness (P = .013), understanding of professional development opportunities (P = .036), and institutional satisfaction (P = .020) among participants. In addition, analysis of research productivity indicated that over the two-year program, ABCC participants received 1.3 more awards, on average, compared with a matched set of nonparticipants, a funding difference of approximately $1.1 million per person. These results suggest it is possible to mitigate the effects of extreme time pressure on academic medicine faculty, even within existing institutional structures.