Validity Evidence for a Residency Admissions Standardized Assessment Letter for Pediatrics.

Construct: This study aims to provide validity evidence for the standardized Assessment Letter for Pediatrics as a measure of competencies expected of a 1st-year pediatrics resident as part of a pediatric residency admissions process. BACKGROUND: The Narrative Letter of Recommendation is a frequently used tool in the residency admissions process even though it has poor interrater reliability, lacks pertinent content, and does not correlate with residency performance. A newer tool, the Standardized Letter, has shown validity evidence for content and interrater reliability in other specialties. We sought to develop and provide validity evidence for the standardized Assessment Letter for Pediatrics. APPROACH: All 2012 and 2013 applicants invited to interview at the University of Geneva Pediatrics Residency Program provided 2 standardized Assessment Letters. Content for the letter was based on CanMEDS roles and ratings of 6 desired competencies and an overall assessment. Validity evidence was gathered for internal structure (Cronbach's alpha and generalizability), response process (interrater reliability with intraclass correlation), relations to other variables (Pearson's correlation coefficient), and consequences (logistic regression to predict admission). RESULTS: One hundred fourteen faculty completed 142 standardized Assessment Letters for 71 applicants. Average overall assessment was 3.0 of 4 (SD = 0.59). Cronbach's alpha was 0.93. The G-coefficient was 0.59. The decision study projected that four Assessment Letters are needed to attain a G-coefficient of 0.73. Applicant variance (28.5%) indicated high applicant differentiation. The Assessment Letter intraclass coefficient was 0.51, 95% confidence interval (CI) [0.43, 0.59]. Assessment Letter scores were correlated with the structured interview (r = .28), 95% CI [0.05, 0.51]; global rating (r = .36), 95% CI [0.13, 0.58]; and admissions decision (r = .25), 95% CI [0.02, 0.46]. Assessment Letter scores did not predict the admissions decision (odds ratio = 1.67, p = .37) after controlling for the unique contribution of the structured interview and global rating scores. CONCLUSION: Validity evidence supports use of the Assessment Letter for Pediatrics; future studies should refine items to improve predictive validity and explore how to best integrate the Assessment Letter into the residency admissions process.

A Multilevel Analysis of Professional Conflicts in Health Care Teams: Insight for Future Training.

PURPOSE: Without a proper understanding of conflict between health care professionals, designing effective conflict management training programs for trainees that reflect the complexity of the clinical working environment is difficult. To better inform the development of conflict management training, this study sought to explore health care professionals' experiences of conflicts and their characteristics. METHOD: Between 2014 and early 2016, 82 semistructured interviews were conducted with health care professionals directly involved in first-line patient care in four departments of the University Hospitals of Geneva. These professionals included residents, fellows, certified nursing assistants, nurses, and nurse supervisors. All interviews were transcribed verbatim, and conventional content analysis was used to derive conflict characteristics. RESULTS: Six conflict sources were identified. Among these sources, disagreements on patient care tended to be the primary trigger of conflict, whereas sources related to communication contributed to conflict escalation without directly triggering conflict. A framework of workplace conflict that integrates its multidimensional and cyclical nature was subsequently developed. This framework suggests that conflict consequences and responses are interrelated, and might generate further tensions that could affect health care professionals, teams, and organizations, as well as patient care. Findings also indicated that supervisors' responses to contentious situations often failed to meet health care professionals' expectations. CONCLUSIONS: Understanding conflicts between health care professionals involves several interrelated dimensions, such as sources, consequences, and responses to conflict. There is a need to strengthen health care professionals' ability to identify and respond to conflict and to further develop conflict management programs for clinical supervisors.

Improving the residency admissions process by integrating a professionalism assessment: a validity and feasibility study.

The purpose of this study was to provide validity and feasibility evidence in measuring professionalism using the Professionalism Mini-Evaluation Exercise (P-MEX) scores as part of a residency admissions process. In 2012 and 2013, three standardized-patient-based P-MEX encounters were administered to applicants invited for an interview at the University of Geneva Pediatrics Residency Program. Validity evidence was gathered for P-MEX content (item analysis); response process (qualitative feedback); internal structure (inter-rater reliability with intraclass correlation and Generalizability); relations to other variables (correlations); and consequences (logistic regression to predict admission). To improve reliability, Kane's formula was used to create an applicant composite score using P-MEX, structured letter of recommendation (SLR), and structured interview (SI) scores. Applicant rank lists using composite scores versus faculty global ratings were compared using the Wilcoxon signed-rank test. Seventy applicants were assessed. Moderate associations were found between pairwise correlations of P-MEX scores and SLR (r = 0.25, P = .036), SI (r = 0.34, P = .004), and global ratings (r = 0.48, P < .001). Generalizability of the P-MEX using three cases was moderate (G-coefficient = 0.45). P-MEX scores had the greatest correlation with acceptance (r = 0.56, P < .001), were the strongest predictor of acceptance (OR 4.37, P < .001), and increased pseudo R-squared by 0.20 points. Including P-MEX scores increased composite score reliability from 0.51 to 0.74. Rank lists of applicants using composite score versus global rating differed significantly (z = 5.41, P < .001). Validity evidence supports the use of P-MEX scores to improve the reliability of the residency admissions process by improving applicant composite score reliability.

[Community immersion program at Geneva medical school: a fifteen-year experience]. / Immersion en communauté au cours des etudes de médecine a Genève: un bilan a quinze ans.

Over the past decades there have been many new developments in medical education due to new public health challenges and to new learning theories. Medical schools throughout the world have adapted to these challenges in adopting community-based learning activities, an approach that the World Health Organization has promoted. The aim of the present article is to describe the characteristics, as well as the evolution, of such a community-based training program which has been implemented over 15 years at the Faculty of medicine of the University of Geneva and to present some evaluation data addressing students' perception, achievement of learning objectives as well as interactions between students and the community.

Design, implementation and evaluation of a community health training program in an integrated problem-based medical curriculum: a fifteen-year experience at the University of Geneva Faculty of Medicine.

BACKGROUND: In the literature the need for relevance in medical education and training has been stressed. In the last 40 years medical schools have been challenged to train doctors competent to respond to community health needs. In the mid-90s the University of Geneva Faculty of Medicine introduced an integrated medical curriculum. In this initiative a particular emphasis was put in introducing a 6-year longitudinal and multidisciplinary Community Health Program (CHP). OBJECTIVES: The aims of the present article are to describe the conception, elaboration and implementation of the CHP as well as its evolution over 15 years and the evaluation of its outcomes. METHODS: The CHP was at its origin elaborated by a small group of highly motivated teachers and later on developed by a multi-disciplinary group of primary care physicians, epidemiologists, public health and bio-ethics specialists, occupational health professionals, lawyers and historians. Evaluation of the program outcomes included educational innovations, new developments of the curriculum and interactions between students and the community. RESULTS: The CHP learning objectives and teaching modalities were defined by the multi-disciplinary group in consensus meetings which triggered a collaborative spirit among teachers and facilitated further developments. The evaluation procedures allowed the monitoring of students' satisfaction which remained high over the years, students' active participation which decreased over time and success at certifying exams which was globally as good as in basic life sciences. The evaluation also assessed outcomes such as educational innovations, new developments of the curriculum and interactions between students and the community. CONCLUSION: As suggested in the literature, our experience shows that the students' direct exposure and practice in the community health environment is an effective training approach to broaden students' education by offering them a community perspective of health and disease.