Social Dynamics of Advice-Seeking: A Network Analysis of Two Residency Programs.

Phenomenon: Residents interact with their peers and supervisors to ask for advice in response to complicated situations occurring during patient care. To provide a deeper understanding of workplace learning, this study explores the structure and dynamics of advice-seeking networks in two residency programs. Approach: We conducted a survey-based social network study. To develop the survey, we conducted focus group discussions and identified three main categories of advice: factual knowledge, clinical reasoning, and procedural skills. We invited a total of 49 emergency medicine and psychiatry residents who had completed at least six months of their training, to nominate their supervisors and peer residents, as their sources of advice, from a roster. Participants identified the number of occasions during the previous month that they turned to each person to seek advice regarding the three broad categories. We calculated the density, centrality, and reciprocity measures for each advice category at each department. Findings: The response rates were 100% (n = 21) and 85.7% (n = 24) in the emergency medicine and psychiatry departments, respectively. The advice network of emergency medicine residents was denser, less hierarchical, and less reciprocated compared to the psychiatry residents' network. In both departments, PGY-1s were the top advice-seekers, who turned to PGY-2s, PGY-3s, and supervisors for advice. The "procedural skills" network had the lowest density in both departments. There was less overlap in the sources of advice for different advice types in the psychiatry department, implying more selectivity of sources. Insights: Complex social structures and dynamics among residents vary by discipline and level of seniority. Program directors can develop tailored educational interventions informed by their departments' specific network patterns to promote a timely and effective advice-seeking culture which in turn, could lead to optimally informed patient care.

The impact of prior performance information on subsequent assessment: is there evidence of retaliation in an anonymous multisource assessment system?

Few studies have engaged in data-driven investigations of the presence, or frequency, of what could be considered retaliatory assessor behaviour in Multi-source Feedback (MSF) systems. In this study, authors explored how assessors scored others if, before assessing others, they received their own assessment score. The authors examined assessments from an established MSF system in which all clinical team members - medical students, interns, residents, fellows, and supervisors - anonymously assessed each other. The authors identified assessments in which an assessor (i.e., any team member providing a score to another) gave an aberrant score to another individual. An aberrant score was defined as one that was more than two standard deviations from the assessment receiver's average score. Assessors who gave aberrant scores were categorized according to whether their behaviour was preceded by: (1) receiving a score or not from another individual in the MSF system (2) whether the score they received was aberrant or not. The authors used a multivariable logistic regression model to investigate the association between the type of score received and the type of score given by that same individual. In total, 367 unique assessors provided 6091 scores on the performance of 484 unique individuals. Aberrant scores were identified in 250 forms (4.1%). The chances of giving an aberrant score were 2.3 times higher for those who had received a score, compared to those who had not (odds ratio 2.30, 95% CI:1.54-3.44, P < 0.001). Individuals who had received an aberrant score were also 2.17 times more likely to give an aberrant score to others compared to those who had received a non-aberrant score (2.17, 95% CI:1.39-3.39, P < 0.005) after adjusting for all other variables. This study documents an association between receiving scores within an anonymous multi-source feedback (MSF) system and providing aberrant scores to team members. These findings suggest care must be given to designing MSF systems to protect against potential downstream consequences of providing and receiving anonymous feedback.

Reopening of a school of dentistry in the era of COVID-19 pandemic, "Step-by-step" approach.

INTRODUCTION: The COVID-19 pandemic has a great impact on all aspects of higher education worldwide. In Iran, after several months of lockdown, all dental schools coped with the challenge of "reopening." School of Dentistry of Tehran University of Medical Sciences (TUMS) planned to facilitate the process of reopening using a "step-by-step" approach. The aim of this study was sharing our experiences of reopening of the school during the pandemic. METHODS: TUMS School of Dentistry planned a "step-by-step" approach of reopening of school to both provide infrastructure of online education for theoretical credits and reassure all staff and students about infection control for practical courses. Also, a two-section survey on students' satisfaction with a variety of aspects of reopening process and the clinical performance of students in different clinical courses was conducted. RESULTS: Findings showed 65.1% of the students, attended the survey, were satisfied with reopening of the school. Also, 86.4% were satisfied with the new online method of education for theoretical courses. Regarding personal protective equipment (PPE) provided for the students, 22.9% and 56.5% of them were satisfied and moderately satisfied, respectively. Additionally, 78.2% of the students were satisfied with the newly developed online assessment method. CONCLUSION: Although reopening of schools during COVID-19 pandemic seems enigmatic for all authorities and students, the "step-by-step" plan designed by TUMS School of dentistry including emotional and infrastructural prerequisites might be a reliable method for all people, who are in charge of management of dental school in uncertain crisis.

Exploring clinical educators' perceptions of role modeling after participating in a role modeling educational program.

INTRODUCTION: Role modeling significantly affects medical students' values, attitudes, and professional behaviors. Role modeling is a complex and multidimensional process that necessitates accurate perception and adequate knowledge and skills. However, most clinical educators do not have adequate awareness and knowledge about the positive and negative effects of role modeling. We aimed to explore clinical educators' perceptions of role modeling after participating in a role modeling educational program. METHODS: This qualitative study was conducted on eighteen clinical educators who were voluntarily participated in a three-month role modeling educational program. Data were collected using reflection paper writing and were analyzed through conventional content analysis. RESULTS: Data analysis resulted in the development of three main categories, namely closer attention to role modeling and effort for its promotion, deliberate effort to display role modeling, and creating a positive environment to increase the effectiveness of role modeling. CONCLUSIONS: Clinical educators have limited role modeling knowledge and skills, and hence, educational programs are required to improve their role modeling knowledge, attitudes, and skills. Role modeling educational programs should aim at developing educators' awareness and knowledge about role modeling, develop their reflection and self-control abilities, and sensitize them to the importance of deliberate use of role modeling.

Enhancement of role modelling in clinical educators: A randomized controlled trial.

Introduction: Role models have an important impact on the professional behavior of medical students. Previous investigations help us understand how to improve role modelling. However, studies aiming at enhancing role modelling among clinical educators are very limited and generally lack comprehensive evaluations of the designed programs. We intended to gather robust evidence on the effectiveness of a longitudinal program for enhancing role modelling.Methods: Clinical educators were divided into intervention and control groups. The longitudinal program, developed based on the exposure phase of the 'Positive Doctor Role Modelling' framework, was delivered during three months of onsite and online sessions. The effectiveness of the program was assessed in three levels of reaction, learning, and behavior.Results: In the intervention group (N = 18), the mean score of satisfaction was 4.7 (SD = 0.5), and the learning (awareness about role modelling) improved significantly after the program (3.33-4.34), comparing to the control group (3.53-3.63). There was no significant difference in terms of behavior improvement between the two groups, before and after the program.Conclusions: Our findings indicate that our longitudinal faculty development program on role modelling was highly appreciated by clinical educators, and improved their awareness and deliberate role modelling.

The utility of mini-Clinical Evaluation Exercise in undergraduate and postgraduate medical education: A BEME review: BEME Guide No. 59.

Background: This BEME review aims at exploring, analyzing, and synthesizing the evidence considering the utility of the mini-CEX for assessing undergraduate and postgraduate medical trainees, specifically as it relates to reliability, validity, educational impact, acceptability, and cost.Methods: This registered BEME review applied a systematic search strategy in seven databases to identify studies on validity, reliability, educational impact, acceptability, or cost of the mini-CEX. Data extraction and quality assessment were carried out by two authors. Discrepancies were resolved by a third reviewer. Descriptive synthesis was mainly used to address the review questions. A meta-analysis was performed for Cronbach's alpha.Results: Fifty-eight papers were included. Only two studies evaluated all five utility criteria. Forty-seven (81%) of the included studies met seven or more of the quality criteria. Cronbach's alpha ranged from 0.58 to 0.97 (weighted mean = 0.90). Reported G coefficients, Standard error of measurement, and confidence interval were diverse and varied based on the number of encounters and the nested or crossed design of the study. The calculated number of encounters needed for a desirable G coefficient also varied greatly. Content coverage was reported satisfactory in several studies. Mini-CEX discriminated between various levels of competency. Factor analyses revealed a single dimension. The six competencies showed high levels of correlation with statistical significance with the overall competence. Moderate to high correlations between mini-CEX scores and other clinical exams were reported. The mini-CEX improved students' performance in other examinations. By providing a framework for structured observation and feedback, the mini-CEX exerts a favorable educational impact. Included studies revealed that feedback was provided in most encounters but its quality was questionable. The completion rates were generally above 50%. Feasibility and high satisfaction were reported.Conclusion: The mini-CEX has reasonable validity, reliability, and educational impact. Acceptability and feasibility should be interpreted given the required number of encounters.

Psychometric properties of the mini peer assessment tools (Mini-PAT) in emergency medicine residents.

Background: A few studies have been done regarding the validity and reliability of the Mini-Peer Assessment Tool across various specialties. This study was conducted to determine the reliability, content and construct validity of Mini-Peer Assessment Tool to assess the competency of emergency medicine residents. Methods: This study was carried out to investigate the psychometric properties of the mini-PAT tool to evaluate the professional competencies of emergency medicine residents in educational hospitals affiliated to Tehran University of Medical Sciences. The initial Mini-Peer Assessment Tool was translated into Persian. After that, the content validity index and content validity ratio determined by consulting 12 professors of emergency medicine. The construct validity was determined with exploratory factor analysis and investigation of the correlation coefficient on 31 self and 248 peer assessment cases. The reliability of the mini peer assessment tool was determined by internal consistency and item deletion by using Cronbach's alpha coefficient. Reliability was also assessed by determining the agreement between the two tools of self-assessment and peer assessment by using the diagram Bland and Altman. Results: The results showed content validity ratio (CVR) of the items ranged from 0.56 to 0.83, and the content validity index (CVI) of the items ranged from 0.72 to 0.90. The reliability of the self-assessment and peer-assessment tools were 0.83 and 0.95 respectively and there was a relative agreement between the self-assessment method and the peer assessment method. Finally, the tool underwent exploratory factor analysis resulting extraction into two factors namely 'clinical competencies' and 'human interactions' in the peer assessment tool. In the self-assessment tool, the factors of 'good practice' and 'technical competence' were extracted. Conclusion: The results of the present study provided evidence of the adequacy of content validity, reliability of the contextually customized mini-peer assessment tool in assessing the competencies of emergency medicine residents.

Developing a competency framework for Health Professions Education at doctoral level: The first step toward a competency based education.

Introduction: The aim of this study was to develop the competency framework for Doctoral graduates in Health Professions Education, which is the first step in implementing an outcome-based curriculum. Method: The present study conducted in three phases. First, viewpoints of 100 stakeholders, including students, graduates, faculty members, and educational managers were explored by using Nominal Group technique, survey, and telephone interviews. The extracted codes classified in subcategories and categories through an inductive content analysis approach. In phase two, the findings converted to competency components, sub-domains and competency domains in working groups. Finally, the content validity of the developed components evaluated through qualitative and quantitative approaches. Results: 535 codes, 12 sub-categories, and 7 categories extracted from stakeholders' viewpoints, which were refined to 65 competency components, 12 sub-domains and 7 competency domains in working groups. The final competency framework includes 54 competency components in seven competency domains of a "professional expert", "research and scholarship", "teaching", "interdisciplinary collaboration", "leadership and management", "professionalism", "personal and professional development". Conclusion: Competency framework is essential for the development of a comprehensive and integrated curriculum. It would be possible to implement a competency-based curriculum in the Doctoral program of Health Professions Education by developing the competency framework.

A decade of reform in medical education: Experiences and challenges at Tehran University of Medical Sciences.

PURPOSE: In this paper, we present the major curricular reform in MD program of Tehran University of Medical Sciences, the oldest and the largest medical university in Iran, initiated about a decade ago. MATERIALS AND METHODS: Following a comprehensive program evaluation, many of the basic challenges of the traditional curriculum were revealed, namely, lack of pre-defined competencies for graduates, over-reliance on teacher-centered teaching methods, over-emphasis on knowledge base in student assessments, and focusing solely on biomedical aspects of patient care. In 2010, a vision statement for reform was created and approved by the University Council. The new curriculum was launched in 2011. RESULTS: The changes included: revising the content of the courses, assimilating horizontal and vertical integration, emphasizing clinical skills, encouraging active involvement in patient management, providing more opportunity for supervised practice, integrating behavioral and psychosocial topics into the curriculum, incorporating interactive teaching methods, assessing students' higher levels of cognition, and strengthening workplace assessments. To evaluate the changes, data were continuously collected and analyzed from the beginning. CONCLUSIONS: Changing the curriculum of an MD program is a laborious task which should be planned and undertaken carefully and cautiously. It is an endless, yet invaluable and satisfying endeavor toward better future.

Introducing a model for optimal design of sequential objective structured clinical examinations.

In a sequential OSCE which has been suggested to reduce testing costs, candidates take a short screening test and who fail the test, are asked to take the full OSCE. In order to introduce an effective and accurate sequential design, we developed a model for designing and evaluating screening OSCEs. Based on two datasets from a 10-station pre-internship OSCE and considering three factors, namely, the number of stations, the criteria for selecting the stations, and the cut-off score, several hypothetical tests were proposed. To investigate their accuracy, the positive predictive value (PPV), the pass rate, and the negative predictive value (NPV) were calculated. Also, a "desirable" composite outcome was defined as PPV = 100 %, pass rate ≥50 %, and NPV ≥25 %. Univariate and multiple logistic regression analyses were conducted to estimate the effects of independent factors on the occurrence of the desirable outcome. In half of the screening tests no false positive result was detected. Most of the screening OSCEs had acceptable levels of pass rate and NPV. Considering the desirable composite outcome 20 screening OSCEs could have successfully predicted the results of the corresponding full OSCE. The multiple regression analysis indicated significant contributions for the selection criteria (p values = 0.019) and the cut-off score (p values = 0.017). In order to have efficient screening OSCEs with the lowest probability of the error rate, careful selection of stations with high values of discrimination or item total correlation, and use of a relatively stringent cut-off score should be considered.

Standard setting in medical education: fundamental concepts and emerging challenges.

The process of determining the minimum pass level to separate the competent students from those who do not perform well enough is called standard setting. A large number of methods are widely used to set cut-scores for both written and clinical examinations. There are some challenging issues pertaining to any standard setting procedure. Ignoring these concerns would result in a large dispute regarding the credibility and defensibility of the method. The goal of this review is to provide a basic understanding of the key concepts and challenges in standard setting and to suggest some recommendations to overcome the challenging issues for educators and policymakers who are dealing with decision-making in this field.

Social Network Analysis in Undergraduate and Postgraduate Medical Education: A Scoping Review.

PURPOSE: Social network analysis (SNA) is a theoretical framework and analytical approach used to study relationships among individuals and groups. While SNA has been employed by many disciplines to understand social structures and dynamics of interpersonal relationships, little is known about its use in medical education. Mapping and synthesizing the scope of SNA in undergraduate and postgraduate medical education can inform educational practice and research. METHOD: This scoping review was based on searches conducted in Medline, Embase, Scopus, and ERIC in December 2020 and updated in March 2022. After removal of duplicates, the search strategy yielded 5,284 records, of which 153 met initial inclusion criteria. Team members conducted full-text reviews, extracted relevant data, and conducted descriptive and thematic analyses to determine how SNA has been used as a theoretical and analytical approach in undergraduate and postgraduate medical education. RESULTS: Thirty studies, from 11 countries, were retained. Most studies focused on undergraduate medical students, primarily in online environments, and explored students' friendships, information sharing, and advice seeking through SNA. Few studies included residents and attending staff. Findings suggested that SNA can be a helpful tool for monitoring students' interactions in online courses and clinical clerkships. SNA can also be used to examine the impact of social networks on achievement, the influence of social support and informal learning outside the classroom, and the role of homophily in learning. In clinical settings, SNA can help explore team dynamics and knowledge exchange among medical trainees. CONCLUSIONS: While SNA has been underutilized in undergraduate and postgraduate medical education, findings indicate that SNA can help uncover the structure and impact of social networks in the classroom and the clinical setting. SNA can also be used to help design educational experiences, monitor learning, and evaluate pedagogical interventions. Future directions for SNA research in medical education are described.

Assessing the reliability of the borderline regression method as a standard setting procedure for objective structured clinical examination.

BACKGROUND: One of the methods used for standard setting is the borderline regression method (BRM). This study aims to assess the reliability of BRM when the pass-fail standard in an objective structured clinical examination (OSCE) was calculated by averaging the BRM standards obtained for each station separately. MATERIALS AND METHODS: In nine stations of the OSCE with direct observation the examiners gave each student a checklist score and a global score. Using a linear regression model for each station, we calculated the checklist score cut-off on the regression equation for the global scale cut-off set at 2. The OSCE pass-fail standard was defined as the average of all station's standard. To determine the reliability, the root mean square error (RMSE) was calculated. The R (2) coefficient and the inter-grade discrimination were calculated to assess the quality of OSCE. RESULTS: The mean total test score was 60.78. The OSCE pass-fail standard and its RMSE were 47.37 and 0.55, respectively. The R (2) coefficients ranged from 0.44 to 0.79. The inter-grade discrimination score varied greatly among stations. CONCLUSION: The RMSE of the standard was very small indicating that BRM is a reliable method of setting standard for OSCE, which has the advantage of providing data for quality assurance.

Understanding and Embracing Culture in International Faculty Development.

Introduction: Research on international faculty development programs (IFDPs) has demonstrated many positive outcomes; however, participants' cultural backgrounds, beliefs, and behaviors have often been overlooked in these investigations. The goal of this study was to explore the influences of culture on teaching and learning in an IFDP. Method: Using interpretive description as the qualitative methodology, the authors conducted semi-structured interviews with 15 Fellows and 5 Faculty of a US-based IFDP. The authors iteratively performed a constant comparative analysis to identify similar patterns and themes. Transformative Learning Theory informed the analysis and interpretation of the results. Results: This research identified three themes related to the influences of culture on teaching and learning. First, cultural differences were not seen as a barrier to learning; instead, they tended to act as a bridge to cultural awareness and network building. Second, some cultural differences produced a sense of unease and uncertainty, which led to adaptations, modifications, or mediation. Third, context mattered, as participants' perspectives were also influenced by the program culture and their professional backgrounds and experiences. Discussion: The cultural diversity of health professions educators in an IFDP did not impede learning. A commitment to future action, together with the ability to reflect critically and engage in dialectical discourse, enabled participants to find constructive solutions to subtle challenges. Implications for faculty development included the value of enhanced cultural awareness and respect, explicit communication about norms and expectations, and building on shared professional goals and experiences.

Validity of constructed-response situational judgment tests in training programs for the health professions: A systematic review and meta-analysis protocol.

BACKGROUND: Situational judgments tests have been increasingly used to help training programs for the health professions incorporate professionalism attributes into their admissions process. While such tests have strong psychometric properties for testing professional attributes and are feasible to implement in high-volume, high-stakes selection, little is known about constructed-response situational judgment tests and their validity. METHODS: We will conduct a systematic review of primary published or unpublished studies reporting on the association between scores on constructed-response situational judgment tests and scores on other tests that measure personal, interpersonal, or professional attributes in training programs for the health professions. In addition to searching electronic databases, we will contact academics and researchers and undertake backward and forward searching. Two reviewers will independently screen the papers and decide on their inclusion, first based on the titles and abstracts of all citations, and then according to the full texts. Data extraction will be done independently by two reviewers using a data extraction form to chart study details and key findings. Studies will be assessed for the risk of bias and quality by two reviewers using the "Quality In Prognosis Studies" tool. To synthesize evidence, we will test the statistical heterogeneity and conduct a psychometric meta-analysis using a random-effects model. If adequate data are available, we will explore whether the meta-analytic correlation varies across different subgroups (e.g., race, gender). DISCUSSION: The findings of this study will inform best practices for admission and selection of applicants for training programs for the health professions and encourage further research on constructed-response situational judgment tests, in particular their validity. TRIAL REGISTRATION: The protocol for this systematic review has been registered in PROSPERO [CRD42022314561]. https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42022314561.